A method for detecting temperature anomaly in high-altitude meteorological detection based on quadratic Bezier curve fitting

Abstract

The present invention relates to a method for detecting temperature anomaly in high-altitude meteorological detection by quadratic Bezier curve fitting, comprising: obtaining high-altitude meteorological observation data; forming an array T according to time data, forming an array A according to temperature data; performing anomaly marking; constructing an array B for storing fitted temperature data, performing Bezier fitting, and obtaining fitted temperature data; calculating the deviation between the temperature data and the fitted temperature data, and updating the anomaly marking; constructing an array D for storing the second fitting temperature data, performing Bezier fitting based on the array B, and obtaining the second fitting temperature data; calculating the deviation between B and D, and updating the anomaly marking; calculating the possibility of each temperature data being abnormal, and outputting a score array representing the probability of temperature anomaly. The present invention gradually enhances the anomaly detection of data from coarse granularity to fine granularity, provides a reference for distinguishing abnormal data for data quality control personnel, is conducive to unifying the judgment standard of abnormal data, avoiding misjudgment and wrong judgment, and improving the work efficiency of quality control personnel.

Description

Technical Field

[0001] The invention relates to a method for detecting temperature anomaly in high-altitude meteorological detection using quadratic Bezier curve fitting, and belongs to the technical field of high-altitude temperature detection. Background Art

[0002] High-altitude meteorological observation involves the use of instruments to detect meteorological elements such as temperature, pressure, humidity, wind direction, and wind speed from the ground to an altitude of 30,000 meters. High-altitude meteorological observation is a fundamental part of meteorological observation, providing accurate and timely high-altitude meteorological intelligence and data for weather forecasting, climate analysis, scientific research, and international exchange.

[0003] High-altitude meteorological detection involves launching sounding instruments aloft on balloons (commonly known as balloon sounding). This method offers the advantages of in-situ data acquisition, high accuracy, high altitude detection, and weather-independent performance. However, due to issues such as sounding instrument failures, wireless communication anomalies, and the explosion of sounding balloons at certain altitudes, high-altitude meteorological data often contains anomalies. If these anomalies are used without processing, subsequent analysis and research will deviate significantly from the actual situation, reducing the credibility of the analysis and research results.

[0004] Conventional quality control methods for high-altitude meteorological data are well-established and primarily include extreme value checks, inter-element consistency checks (temporal, spatial, and horizontal consistency), temperature inversion checks, wind shear checks, and statics checks. These quality control steps are effective for removing obvious anomalies, but they are somewhat crude for fine-grained anomaly detection in second-level high-altitude meteorological data. Building upon these previous methods, a series of new quality control methods have emerged, including improved statics checks, superadiabatic temperature lapse rate checks, rigidity checks, dual-weighted outlier checks, piecewise fitting checks, radar raw coordinate checks, Gaussian filtering checks, balloon velocity checks, monotonicity checks, and linear regression quality checks.

[0005] In high-altitude meteorological detection, data anomalies usually take into account data such as temperature, air pressure, humidity, wind speed and wind direction. Among them, temperature is the most important basis for determining data anomalies.

[0006] Currently, the Meteorological Bureau addresses anomalies in high-altitude meteorological data through manual screening, where experienced personnel analyze and label the collected data. However, manual screening wastes human resources and can lead to inconsistent results due to inconsistent judgment criteria. Summary of the Invention

[0007] In order to overcome the above problems, the present invention provides a method for detecting temperature anomalies in high-altitude meteorological detection based on quadratic Bezier curve fitting. This method gradually enhances data anomaly detection from coarse granularity to fine granularity, and comprehensively utilizes the tag array and the results of two Bezier fittings to calculate the outlier value array score representing the anomaly probability, providing a reference for abnormal data judgment for data quality control personnel, which is conducive to unifying the judgment criteria of quality control personnel for abnormal data, avoiding misjudgment and wrong judgment, and can also greatly improve the work efficiency of quality control personnel.

[0008] The technical solutions of the present invention are as follows:

[0009] A method for detecting temperature anomalies in high-altitude meteorological detection using quadratic Bezier curve fitting, comprising:

[0010] Acquiring high-altitude meteorological observation data, wherein the high-altitude meteorological observation data includes time data and temperature data;

[0011] Form array T based on time data, and form array A based on temperature data;

[0012] Perform extreme value-based anomaly marking based on arrays T and A and store them in the Tag array;

[0013] Construct array B for storing fitted temperature data, perform Bezier fitting based on arrays T and A, obtain fitted temperature data, and store it in array B;

[0014] Calculate the deviation between the temperature data and the fitted temperature data, and update the abnormal tag in the Tag array based on the deviation and the threshold;

[0015] Construct an array D for storing the second fitting temperature data, perform Bezier fitting based on the first Bezier fitting array B, obtain the second fitting temperature data, and store it in array D;

[0016] Calculate the deviation between the fitted temperature data and the second fitted temperature data, and update the abnormal flag in the Tag array according to the deviation and the threshold;

[0017] Calculate the probability of each temperature data anomaly and output a score array representing the probability of temperature anomaly.

[0018] Furthermore, the high-altitude meteorological observation data is obtained by the following method:

[0019] Read the source data files generated by the L-band high-altitude meteorological detection system;

[0020] Parse the source data file and extract the time and temperature data therein.

[0021] Furthermore, the length of arrays T and A are both n+2, where n is the number of data points. The head and tail elements of arrays T and A are as follows:

[0022] A[0]=GT0;

[0023] T[0]=T[n]-1;

[0024] A[n+1]=A[n]+A[n]-A[n-1];

[0025] T[n+1]=T[n]+1;

[0026] Among them, GT0 is the ground temperature.

[0027] Furthermore, arrays T and A are marked as abnormal based on extreme values, specifically:

[0028] Obtaining segmented extreme values ​​of historical high-altitude meteorological observation data, wherein the segmented extreme values ​​include a maximum temperature value min_temp and a minimum temperature value max_temp, and obtaining a range maximum value range_min and a range minimum value range_max of a measuring instrument;

[0029] Based on arrays T and A, set the Tag annotation according to the following formula:

[0030]

[0031] Among them, Tag 0 indicates normal temperature, and Tag 1 indicates abnormal temperature;

[0032] The difference between A[1] and the ground temperature GT0 is used to assign a value to Tag[1]. The formula is as follows:

[0033]

[0034] Furthermore, an array B is constructed to store the fitted temperature data. Bezier fitting is performed based on arrays T and A to obtain the fitted temperature data. Specifically:

[0035] Construct array B to store fitting temperature data;

[0036] Use Bezier fitting to perform the following operations on each point A[k] to be determined, where 1≤k≤n:

[0037]

[0038] P1=(A[k],T[k]);

[0039]

[0040] Wherein, 1≤i≤n, 1≤j≤n, 1≤l≤n, and i, j, k, and l are integers;

[0041] Use P0, P1, and P2 as control points for Bezier curve calculation;

[0042] Calculate the intermediate variable t:

[0043]

[0044] Calculate the intermediate variable Pt:

[0045]

[0046] Among them, P0[0], P1[0], P2[0] are the temperature values ​​of the three control points respectively, and P0[1], P1[1], P2[1] are the time values ​​of the three control points respectively;

[0047] Let B[k]=Pt.

[0048] Furthermore, the deviation between the temperature data and the fitted temperature data is calculated, and the abnormal flag in the Tag array is updated according to the deviation and the threshold, as follows:

[0049]

[0050] Among them, threshold1 is the threshold.

[0051] Furthermore, the threshold threshold1 is 0.122.

[0052] Furthermore, an array D for storing the second fitting temperature data is constructed, and Bezier fitting is performed based on the first Bezier fitting array B to obtain the second fitting temperature data, specifically:

[0053] Construct array D to store the second fitting temperature data;

[0054] Use Bezier fitting to perform the following operations on each point B[k] to be determined:

[0055]

[0056] P1=(B[k],T[k]);

[0057]

[0058] Calculate the intermediate variable t:

[0059]

[0060] Calculate the intermediate variable Pt:

[0061]

[0062] Let D[k]=Pt.

[0063] Furthermore, the deviation between the fitted temperature data and the second fitted temperature data is calculated, and the abnormal flag in the Tag array is updated according to the deviation and the threshold, as follows:

[0064]

[0065] Among them, threshold2 is the threshold.

[0066] Furthermore, the probability of each temperature data anomaly is calculated, and the score array representing the probability of temperature anomaly is output, specifically:

[0067] The non-abnormal temperature data is marked as outliers to indicate the possibility of anomaly. The outlier array score formula is as follows:

[0068]

[0069] The present invention has the following beneficial effects:

[0070] The present invention gradually enhances the data anomaly detection from coarse granularity to fine granularity. First, the instrument range extreme value and the segmented extreme value of the historical data are used to filter out anomalies and separate some significant anomalies. On this basis, the first Bezier curve fitting method is used to perform anomaly judgment on non-continuous abnormal data points. However, the first Bezier fitting is easily interfered by continuous anomalies, thereby misjudging continuous anomalies. Therefore, a second Bezier curve fitting is performed on the basis of the first Bezier judgment to avoid misjudgment of abnormal data caused by continuous anomalies. Finally, the present invention comprehensively utilizes the Tag array and the results of the two Bezier fittings to calculate the outlier value array score representing the probability of anomaly, providing a reference for abnormal data judgment for data quality control personnel, which is conducive to unifying the judgment criteria of quality control personnel for abnormal data, avoiding misjudgment and wrong judgment, and can also greatly improve the work efficiency of quality control personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] FIG1 shows an abnormal point detected by using a temperature range according to an embodiment of the present invention.

[0072] FIG2 shows an abnormal point detected by using historical extreme values ​​of temperature intervals according to an embodiment of the present invention.

[0073] FIG3 shows an abnormal point obtained by performing a detection by subtracting the measured value from the fitted value after the first Bezier fitting in an embodiment of the present invention.

[0074] Figure 4 After the second Bezier fitting in the embodiment of the present invention, the first fitting value is detected by taking the difference between the first fitting value and the current fitting value to obtain the abnormal point. DETAILED DESCRIPTION

[0075] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0076] A method for detecting temperature anomalies in high-altitude meteorological detection using quadratic Bezier curve fitting, comprising:

[0077] Acquiring high-altitude meteorological observation data, wherein the high-altitude meteorological observation data includes time data and temperature data;

[0078] Form array T based on time data, and form array A based on temperature data;

[0079] Perform extreme value-based anomaly marking based on arrays T and A and store them in the Tag array;

[0080] Construct array B for storing fitted temperature data, perform Bezier fitting based on arrays T and A, obtain fitted temperature data, and store it in array B;

[0081] Calculate the deviation between the temperature data and the fitted temperature data, and update the abnormal tag in the Tag array based on the deviation and the threshold;

[0082] Construct an array D for storing the second fitting temperature data, perform Bezier fitting based on the first Bezier fitting array B, obtain the second fitting temperature data, and store it in array D;

[0083] Calculate the deviation between the fitted temperature data and the second fitted temperature data, and update the abnormal flag in the Tag array according to the deviation and the threshold;

[0084] Calculate the probability of each temperature data anomaly and output a score array representing the probability of temperature anomaly.

[0085] In one embodiment, the high-altitude meteorological observation data is obtained by the following method:

[0086] Read the source data files generated by the L-band high-altitude meteorological detection system;

[0087] Parse the source data file and extract the time and temperature data therein.

[0088] More specifically, the source data file (ie, S file) generated by the domestic L-band high-altitude meteorological detection system is read and analyzed, and the time and temperature data therein are extracted.

[0089] In one embodiment of the present invention, the lengths of arrays T and A are both n+2, where n is the number of data points. The head and tail elements of arrays T and A are as follows:

[0090] A[0]=GT0;

[0091] T[0]=T[n]-1;

[0092] A[n+1]=A[n]+A[n]-A[n-1];

[0093] T[n+1]=T[n]+1;

[0094] Among them, GT0 is the ground temperature.

[0095] In one embodiment of the present invention, arrays T and A are marked as abnormal based on extreme values, specifically:

[0096] Obtaining segmented extreme values ​​of historical high-altitude meteorological observation data, wherein the segmented extreme values ​​include a maximum temperature value min_temp and a minimum temperature value max_temp, and obtaining a range maximum value range_min and a range minimum value range_max of a measuring instrument;

[0097] Based on arrays T and A, set the Tag annotation according to the following formula:

[0098]

[0099] Among them, Tag 0 indicates normal temperature, and Tag 1 indicates abnormal temperature;

[0100] The difference between A[1] and the ground temperature GT0 is used to assign a value to Tag[1]. The formula is as follows:

[0101]

[0102] In one embodiment of the present invention, an array B for storing fitted temperature data is constructed, and Bezier fitting is performed based on array T and array A to obtain the fitted temperature data, specifically:

[0103] Construct array B to store fitting temperature data;

[0104] Use Bezier fitting to perform the following operations on each point A[k] to be determined, where 1≤k≤n:

[0105]

[0106] That is, P0 is the non-outlier point closest to A[k] from A[0] to A[k-1];

[0107] P1=(A[k],T[k]);

[0108]

[0109] That is, P2 is the non-outlier point closest to A[k] from A[k+1] to A[n+1];

[0110] Wherein, 1≤i≤n, 1≤j≤n, 1≤l≤n, and i, j, k, and l are integers;

[0111] Use P0, P1, and P2 as control points for Bezier curve calculation;

[0112] Calculate the intermediate variable t:

[0113]

[0114] Calculate the intermediate variable Pt:

[0115]

[0116] Among them, P0[0], P1[0], P2[0] are the temperature values ​​of the three control points respectively, and P0[1], P1[1], P2[1] are the time values ​​of the three control points respectively;

[0117] Let B[k]=Pt.

[0118] The introduction of intermediate variables t and Pt facilitates the simplification of the calculation process.

[0119] In one embodiment of the present invention, the deviation between the temperature data and the fitted temperature data is calculated, and the abnormal flag in the Tag array is updated according to the deviation and the threshold value, as follows:

[0120]

[0121] Among them, threshold1 is the threshold.

[0122] In particular, the threshold threshold1 is 0.122.

[0123] In one embodiment of the present invention, an array D for storing the second fitting temperature data is constructed, and Bezier fitting is performed based on the first Bezier fitting array B to obtain the second fitting temperature data, specifically:

[0124] Construct array D to store the second fitting temperature data;

[0125] Use Bezier fitting to perform the following operations on each point B[k] to be determined:

[0126]

[0127] P1=(B[k],T[k]);

[0128]

[0129] Calculate the intermediate variable t:

[0130]

[0131] Calculate the intermediate variable Pt

[0132]

[0133] Let D[k]=Pt.

[0134] In one embodiment of the present invention, the deviation between the fitted temperature data and the second fitted temperature data is calculated, and the abnormal flag in the Tag array is updated according to the deviation and the threshold value, as follows:

[0135]

[0136] Among them, threshold2 is the threshold.

[0137] In particular, the threshold threshold2 is 0.076.

[0138] In one embodiment of the present invention, the probability of each temperature data being abnormal is calculated, and a score array representing the probability of temperature anomaly is output. The score array represents the probability of the temperature being abnormal. The probability ranges from 0 to 1, and the larger the value, the more likely it is abnormal data. The score formula of the abnormal value array is as follows:

[0139]

[0140] In actual use, temperature node anomalies are checked by setting thresholds.

[0141] The following is an explanation of the results of outlier detection using the sounding data of a certain sounding station using the method of the present invention.

[0142] Figures 1(a) and 1(b) show the anomaly points detected by the sounding station using the temperature range, where "x" represents an anomaly point and "·" represents a normal point. (The two figures were taken at 7:00 AM on January 1, 2020, and 7:00 PM on February 5, 2020, respectively.) As can be seen from the figures, using the temperature measurement instrument's range to identify data point anomalies is effective, correctly marking data points that exceed the instrument's range.

[0143] Figures 2(a) and 2(b) show the anomaly points detected by the sounding station using historical extreme values ​​within the temperature bins, where "x" represents an anomaly point and "·" represents a normal point. (The two figures are taken at 7:00 AM on January 1, 2020, and 7:00 PM on February 5, 2020, respectively.) As can be seen from the figures, using historical extreme values ​​within the temperature bins to identify data point anomalies is effective, as the marked anomaly points clearly deviate from the data trajectory during the sounding balloon's ascent.

[0144] Figures 3(a) and 3(b) show the anomalies detected by subtracting the measured and fitted values ​​after the first Bezier fitting of the radiosonde data. "*" indicates an anomaly, and "·" indicates a normal point. (The two figures are taken at 7:00 AM on February 3, 2020, and 7:00 AM on January 2, 2020, respectively.) As can be seen from the figures, the first Bezier fitting method can effectively detect a small number of offset anomalies at 8 and 58 seconds in Figure 3(a) and at 3072 and 3074 seconds in Figure 3(b).

[0145] Figure 4 After the second Bezier fitting of the sounding station data, the first fitting value is used to make a difference between the fitting value and the current fitting value to detect the abnormal points, where "*" is an abnormal point and "·" is a normal point. (Time 7:00 on January 5, 2020). As can be seen from the figure, Figure 4 The continuous and drastic changes from 3345 to 3350 seconds make the first Bezier fitting result easily affected by the continuous abnormal points, resulting in misjudgment of the correct point (the temperature corresponding to 3346 seconds). However, by selecting the temperature corresponding to 3344 seconds as a reference based on the first Bezier judgment, the data point corresponding to 3344 seconds can be restored, reducing the misjudgment of abnormalities.

[0146] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structures made using the contents of the present invention's description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for detecting temperature anomalies in high-altitude meteorological detection based on quadratic Bezier curve fitting. It is characterized in that include: Acquiring high-altitude meteorological observation data, wherein the high-altitude meteorological observation data includes time data and temperature data; Form array T according to time data, and form array A according to temperature data; Based on arrays T and A, perform anomaly marking based on extreme values ​​and store them in the Tag array; Construct an array B for storing the fitted temperature data, perform Bezier fitting based on array T and array A, obtain the fitted temperature data, and store it in array B; Calculate the deviation between the temperature data and the fitted temperature data, and update the abnormal tag in the Tag array according to the deviation and the threshold; Construct an array D for storing the second fitting temperature data, perform Bezier fitting based on the first Bezier fitting array B, obtain the second fitting temperature data, and store it in array D; Calculate the deviation between the fitted temperature data and the second fitted temperature data, and update the abnormal tag in the Tag array according to the deviation and the threshold; Calculate the probability of each temperature data being abnormal and output a score array representing the probability of temperature anomaly; The length of arrays T and A are both n+2, where n is the number of data points. The head and tail elements of arrays T and A are as follows: A[0]=GT 0 ; T[0] = T[n] - 1; A[n+1]=A[n]+A[n]-A[n-1]; T[n+1]=T[n]+1; Among them, GT 0 is the ground temperature; Perform extreme value-based exception marking on arrays T and A, specifically: Obtain the segmented extreme values ​​of the historical high-altitude meteorological observation data, wherein the segmented extreme values ​​include the maximum temperature value min_temp and the minimum temperature value max_temp, and obtain the maximum range value range_min and the minimum range value range_max of the measuring instrument; Based on arrays T and A, set the Tag annotation according to the following formula: Among them, Tag 0 indicates normal temperature, and Tag 1 indicates abnormal temperature; According to A[1] and ground temperature GT 0 The difference is assigned to Tag[1] using the following formula: Calculate the probability of each temperature data anomaly and output the score array representing the probability of temperature anomaly, specifically: The non-abnormal temperature data is annotated with outliers to indicate the possibility of being abnormal. The outlier array score formula is as follows:

2. The method for detecting temperature anomaly in high-altitude meteorological detection by quadratic Bezier curve fitting according to claim 1, It is characterized in that The high-altitude meteorological observation data is obtained by the following method: Read the source data files generated by the L-band high-altitude meteorological detection system; The source data file is parsed to extract the time and temperature data therein.

3. The method for detecting temperature anomaly in high-altitude meteorological detection by quadratic Bezier curve fitting according to claim 2, It is characterized in that Construct an array B for storing the fitted temperature data, perform Bezier fitting based on arrays T and A, and obtain the fitted temperature data, specifically: Construct array B to store fitting temperature data; Use Bezier fitting to perform the following operations on each point A[k] to be determined, where 1≤k≤n: P 1 =(A[k],T[k]); Wherein, 1≤i≤n, 1≤j≤n, 1≤l≤n, and i, j, k, l are integers; P 0 , P 1 , P 2 As control points for Bezier curve calculation; Calculate the intermediate variable t: Calculate the intermediate variable Pt: Among them, P 0 [0],P 1 [0],P 2 [0] are the temperature values ​​of the three control points, P 0 [1],P 1 [1],P 2 [1] are the time values ​​of the three control points; Let B[k]=Pt.

4. The method for detecting temperature anomaly in high-altitude meteorological detection by quadratic Bezier curve fitting according to claim 3, It is characterized in that When calculating the deviation between the temperature data and the fitted temperature data and updating the abnormal tag in the Tag array according to the deviation and the threshold, proceed as follows: Among them, threshold1 is the threshold.

5. The method for detecting temperature anomaly in high-altitude meteorological detection by quadratic Bezier curve fitting according to claim 4, It is characterized in that The threshold threshold1 is 0.

122.

6. The method for detecting temperature anomaly in high-altitude meteorological detection by quadratic Bezier curve fitting according to claim 4, It is characterized in that Construct an array D for storing the second fitting temperature data, perform Bezier fitting based on the first Bezier fitting array B, and obtain the second fitting temperature data, specifically: Construct array D to store the second fitting temperature data; Use Bezier fitting to perform the following operations on each point B[k] to be determined: P 1 =(B[k],T[k]); Calculate the intermediate variable t: Calculate the intermediate variable Pt: Let D[k]=Pt.

7. The method for detecting temperature anomaly in high-altitude meteorological detection by quadratic Bezier curve fitting according to claim 5, It is characterized in that When calculating the deviation between the fitted temperature data and the second fitted temperature data and updating the abnormal tag in the Tag array according to the deviation and the threshold, proceed as follows: Among them, threshold2 is the threshold.

Images