Self-adaptive soil temperature data quality control method

Abstract

The invention discloses a self-adaptive soil temperature data quality control method, which comprises the following steps of: performing interpolation on soil temperature observation data in time to improve time resolution; eEMD decomposition is carried out on the interpolated data to obtain a series of intrinsic mode IMFs, and only the IMFs corresponding to the original time resolution are reserved; calculating a standard deviation delta and a mean value mu of the fourth intrinsic mode IMF4; comparing the IMF4 component of each time point with a standard deviation delta and a mean value mu, and defining suspicious data; and when suspicious data appear at three continuous time points, defining the data at the second time point as error data, and screening out the error data. According to the method, an ensemble empirical mode decomposition method with a self-adaptive characteristic is adopted, and a high-frequency change part in the soil temperature data is extracted, so that normal fluctuation caused by local weather conditions is distinguished from abnormal data, and quality control on the soil temperature data is realized.

Description

Technical Field

[0001] The invention belongs to the field of soil temperature research, and in particular relates to an adaptive soil temperature data quality control method. Background Art

[0002] With the establishment of a nationwide network of soil temperature observation stations, large-scale collection of soil temperature data has become a reality. However, due to the strong spatial heterogeneity of soil, soil temperature usually presents more irregular small-scale anomaly characteristics, resulting in a significant weakening of its spatiotemporal continuity. This brings additional challenges to the quality control of soil temperature observation data. Existing quality control methods are difficult to effectively handle these irregular spatiotemporal variations, which in turn affects the accurate screening of data and seriously restricts the effective use of soil temperature data in my country. In addition, in the direct assimilation process of satellite data, the accuracy of soil temperature data also limits the assimilation effect. Therefore, it is urgent to develop soil temperature quality control methods that can meet the needs of business applications to improve the reliability and utilization of data. Summary of the invention

[0003] The purpose of the present invention is to solve the defects existing in the prior art and provide a quality control method for soil temperature data, which can distinguish normal fluctuations caused by local weather changes from abnormal data, so as to effectively identify and eliminate abnormal data.

[0004] In order to achieve the above object, the technical solution provided by the present invention is as follows:

[0005] An adaptive soil temperature data quality control method comprises the following steps:

[0006] S1, interpolate the soil temperature observation data in time to improve the temporal resolution;

[0007] S2, perform EEMD decomposition on the interpolated data to obtain a series of intrinsic mode IMFs, and only retain the IMFs corresponding to the original time resolution;

[0008] S3, calculate the standard deviation δ and mean μ of the fourth eigenmode IMF4;

[0009] S4. Compare the IMF4 component at each time point with the standard deviation δ and the mean μ to define suspicious data; when suspicious data appear at three consecutive time points, the data at the second time point is defined as erroneous data and is screened out.

[0010] The present invention adopts an ensemble empirical mode decomposition (EEMD) method with adaptive characteristics to extract the high-frequency variation part in the soil temperature data, thereby distinguishing normal fluctuations caused by local weather conditions from abnormal data and achieving quality control of the soil temperature data.

[0011] Furthermore, the specific implementation method of step S1 is as follows: the temporal resolution of the soil temperature observation data is increased from every hour to every 2.5 minutes using the cubic spline interpolation method to eliminate the scale mixing problem in the EEMD process.

[0012] Furthermore, the definition of suspicious data in step S4 is as follows: when IMF4 at the time point satisfies the following condition: |IMF4-μ|≥3δ, the data is defined as suspicious data.

[0013] Furthermore, the expression of EEMD decomposition in step S3 is shown in formula (1). For the time series u of soil temperature data i , can be decomposed into a series of intrinsic mode IMFs:

[0014]

[0015] Among them, the first k IMFs represent the high-frequency part of the vector; R L (k) represents the residual.

[0016] Compared with the prior art, the present invention has the following advantages:

[0017] 1. The present invention fully considers the strong spatial heterogeneity and complex local variation characteristics of soil. Abnormal data often leads to irregular spatiotemporal variation, that is, high-frequency information in the time series. Therefore, by using the ensemble empirical mode decomposition (EEMD) method, the high-frequency variation part in the soil temperature data can be adaptively extracted to distinguish the normal fluctuations caused by local weather conditions from the abnormal data. At the same time, the present invention screens the intrinsic modes after EEMD decomposition, confirms the changes in the time scale represented by the fourth intrinsic mode IMF4, and can effectively confirm the erroneous data.

[0018] 2. Most existing statistical methods require the pre-specification of fluctuation frequencies in order to extract the high-frequency fluctuation part in the vector. However, the high-frequency fluctuation of soil temperature anomalies has the characteristics of random fluctuations, covering fluctuations of multiple time scales, and the soil texture of soil temperature observation stations in different regions is different, so it cannot be effectively separated by a fixed frequency extraction method. To this end, the present invention adopts the EEMD method to simultaneously extract high-frequency fluctuations of multiple time scales through integrated decomposition technology, so as to better identify complex outliers.

[0019] 3. The present invention uses an adaptive fluctuation extraction method, ensemble empirical mode decomposition (EEMD), to perform soil temperature quality control, which can effectively extract abnormal fluctuations in soil temperature data, especially without regional restrictions. Therefore, the method of the present invention can be directly applied to the quality control of soil temperature observation data, effectively identify and eliminate abnormal data, improve the accuracy and reliability of soil temperature data, and ensure its effective use in scientific research and applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The soil temperature data before and after the observation data interpolation in Example 2 of the present invention;

[0021] Figure 2 are the first 10 intrinsic mode IMFs decomposed by EEMD in Example 2 of the present invention;

[0022] Figure 3 It is a time series diagram of IMF4 in Example 2 of the present invention;

[0023] Figure 4 This is the spatial distribution map of the identified erroneous data;

[0024] Figure 5 This is a time series diagram of the identified error data. DETAILED DESCRIPTION

[0025] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Example 1

[0027] The adaptive soil temperature data quality control method of the present invention comprises the following steps:

[0028] Step 1: The temporal resolution of the observation data is increased from every hour to every 2.5 minutes using the cubic spline interpolation method to eliminate the scale mixing problem in the EEMD process.

[0029] Step 2: Perform EEMD decomposition on the interpolated data to obtain a series of intrinsic mode IMFs, and only retain the IMFs corresponding to the original time resolution.

[0030] Step 3: Calculate the standard deviation δ and mean μ of the fourth eigenmode (IMF4).

[0031] Step 4: Compare the IMF4 component at each time point with the standard deviation δ and mean μ. When |IMF4-μ|≥3δ, the data is defined as suspicious data. If suspicious data appears at three consecutive time points, the data at the second time point is finally defined as erroneous data.

[0032] EEMD decomposition can be expressed as follows: for the soil temperature time series u i , can be decomposed into a series of intrinsic mode IMFs:

[0033]

[0034] The first k IMFs represent the high-frequency part of the vector.

[0035] Example 2

[0036] This embodiment selects the soil temperature data from 00:00 on September 1, 2022 to 23:00 on September 15, 2022 as an example, and uses the method in Embodiment 1 for quality control.

[0037] The first step is to interpolate the soil temperature data in time and increase the time resolution from 1 hour to 2.5 minutes to eliminate the EEMD time scale mixing problem. Figure 1 The figure shows the time series of soil temperature at 5cm depth at Hanzhong Station, where the red dots represent the hourly soil temperature before interpolation and the black dots represent the 2.5-minute soil temperature after interpolation. It can be seen from the figure that interpolation does not affect the original data and its trend characteristics.

[0038] Step 2: Perform EEMD decomposition on the interpolated data. The decomposition results are as follows: Figure 2 As shown, the first three intrinsic mode functions (IMF1, IMF2, IMF3) represent virtual scale features with a time scale of less than 1 hour, which are virtual features caused by interpolation and cannot be used to identify erroneous data. IMF4 represents changes with a time scale of 2 hours. At this time, the disturbance caused by the erroneous data is mixed with small-scale disturbances with periodic characteristics. Starting from IMF5, the changes represent a time scale of 4 hours or longer. Due to the randomness of the erroneous data, it can be determined that the temperature changes caused by the erroneous data will not appear in patterns other than IMF5. In subsequent studies, IMF4 will be used for quality control.

[0039] Step 3: Use the decomposed IMF4 to calculate its mean and standard deviation, and when the IMF4 at that moment is greater than or less than the mean plus or minus three times the standard deviation, it is considered to be suspicious data. Figure 3The IMF4 series obtained by decomposition is shown in the figure. The red line indicates the range of plus or minus three standard deviations of the mean, the red dot indicates the moment when the threshold is exceeded, and the dotted line indicates the moment of erroneous data. At this time, Hanzhong Station had suspicious data at three consecutive time points from 0100 to 0300 UTC on the 8th. At this time, the data at 0200 UTC on the 8th was finally defined as erroneous data. This is because the fluctuation caused by erroneous data always occurs at a certain moment, and there will be large fluctuations before and after this moment, while the temperature caused by weather processes such as precipitation always drops sharply at a certain moment, and then remains stable for a period of time.

[0040] Step 4: In order to determine whether the identified data is wrong, the spatial distribution of soil temperature around the station and the change of temperature of the station at that moment over time are given. Figure 4 The spatial distribution of soil temperature at 0200 UTC on September 8, 2021 is shown, corresponding to Figure 3 The moment when the erroneous data was identified in the image. The points in the image represent the location of each soil temperature station, the numbers represent the soil temperature magnitude of the station, and the red points represent stations with erroneous data. The shaded area represents the altitude of the terrain. As can be seen from the image, the soil temperature of the station is significantly lower than the surrounding soil temperature, and there is no significant altitude change in the area, indicating that the data is erroneous. To further confirm the accuracy of these soil temperatures, the temperature changes over time within the station are shown. Figure 5 As shown in the figure, the red circles mark the erroneous data points. Figure 5 It can be seen that the temperature suddenly drops compared to the previous moment, and then rises rapidly, indicating that the abnormality is discontinuous. Therefore, it is considered to be erroneous data.

[0041] The present invention is not limited to the above embodiments, and any technical solutions obtained by equivalent replacement methods fall within the scope of protection required by the present invention.

Claims

1. An adaptive soil temperature data quality control method, characterized in that: The following steps are involved: S1, interpolate the soil temperature observation data in time to improve the temporal resolution; S2, perform EEMD decomposition on the interpolated data to obtain a series of intrinsic mode IMFs, and only retain the IMFs corresponding to the original time resolution; S3, calculate the standard deviation δ and mean μ of the fourth eigenmode IMF4; S4, compare the IMF4 component at each time point with the standard deviation δ and mean μ to define suspicious data; When suspicious data appear at three consecutive time points, the data at the second time point is defined as erroneous data and is screened out.

2. The quality control method according to claim 1, characterized in that: The specific implementation method of step S1 is as follows: the time resolution of soil temperature observation data is increased from every hour to every 2.5 minutes using cubic spline interpolation method.

3. The quality control method according to claim 2, characterized in that: The definition of suspicious data in step S4 is as follows: when IMF4 at the time point satisfies the following condition: |IMF4-μ|≥3δ, the data is defined as suspicious data.

4. The quality control method according to claim 3, characterized in that: The expression of EEMD decomposition in step S3 is shown in formula (1). For the time series u of soil temperature data, i , can be decomposed into a series of intrinsic mode IMFs: Among them, the first k IMFs represent the high-frequency part of the vector; R L (k) represents the residual.

Images