A ground air temperature quality control method based on complex terrain

Abstract

The application discloses a ground air temperature quality control method based on complex terrain, first, a complex terrain correlation index is constructed to measure the complex terrain correlation between stations, and the correlation between the target station and the adjacent station is measured from two aspects of air temperature data correlation and station complex terrain correlation, and a clustering algorithm is used to select the station with high correlation as a reference station; then, a distance, a cosine similarity and the complex terrain correlation index are used to construct a weight index, a particle swarm optimization algorithm is used to optimize the smoothing parameter in the model, and a prediction model is used to regress and predict the air temperature observation value; finally, the predicted value is compared with the observation value of the target station, and the abnormal value in the observation data is identified. The method makes up for the deficiency of the traditional multi-station quality control method in the complex terrain area, can effectively identify the suspicious air temperature data and mark, and further improves the accuracy and integrity of the ground air temperature data in the complex terrain.

Description

Technical Field

[0001] This invention belongs to the field of quality control algorithm technology for surface meteorological data, specifically involving a surface air temperature quality control method based on complex terrain, which can be applied in numerical weather prediction. Background Technology

[0002] Numerical weather forecasting (NWP) is a core technology for weather forecasting and disaster prevention and mitigation. Research shows that data assimilation technology is a prerequisite for ensuring the accuracy of numerical weather forecasting, while quality control of meteorological data is a necessary step in data assimilation.

[0003] Quality control of surface meteorological observation data is divided into single-station quality control and multi-station quality control. Single-station quality control utilizes the time series of meteorological observation data from a single target station or other meteorological elements related to that observation data for quality control. Traditional methods mainly include format tests, extreme value tests, internal consistency tests, and time consistency tests. Multi-station quality control, on the other hand, uses the target station as the center and selects surrounding stations as reference stations. It then uses the meteorological observations from the reference stations to perform regression predictions on the target station's observations, thereby achieving quality control. Multi-station quality control methods mainly include the inverse distance weighting (IDW) method and the spatial regression test (SRT) method. The former assigns weights to reference stations based on the distance between stations, while the latter assigns weights based on the root mean square error of the meteorological observation data from each station. Currently, multi-station quality control methods generally perform well in flat plains areas, but their effectiveness is poor in complex terrain areas such as mountains and hills. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a surface temperature quality control method based on complex terrain, which improves the quality of surface temperature observation data under complex terrain by addressing the shortcomings of current multi-station quality control methods in terms of site selection, weight assignment, and failure to consider terrain factors.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A method for controlling surface air quality based on complex terrain includes the following steps:

[0007] S1. Select the temperature observation time series data of the target station, and the temperature observation time series data of neighboring stations within a set range from the target station during the same period;

[0008] S2. Calculate the similarity distance between the temperature observation time series of each neighboring station and the target station, and the complex terrain correlation index between each neighboring station and the target station. Use the similarity distance and complex terrain correlation index and the K-means clustering algorithm to screen neighboring stations. The screened stations are used as reference stations.

[0009] S3. Calculate the distance between each reference station and the target station, as well as the cosine similarity of the temperature observation data. Combine the complex terrain-related index to construct a weight index to assign weights to the reference stations and build a prediction model.

[0010] S4. Use the prediction model to predict the temperature observation data of the target station, and use the PSO particle swarm optimization algorithm to optimize the model parameters of the prediction model. When the fitness function value is minimized, the prediction result reaches the best state, and the parameter value in this state is selected as the final model parameters.

[0011] S5. Evaluate the prediction results by combining evaluation indicators;

[0012] S6. Calculate the predicted temperature value for each moment. Compare with the observed temperature value Y, if the difference satisfies If the temperature observation data at that moment is deemed to have passed the test, then the data is deemed to be suspicious and marked accordingly; where f represents the quality control parameter and ζ represents the standard error of the observation sequence.

[0013] Furthermore, the constructed prediction model is as follows:

[0014]

[0015] in, Y is the predicted value. i Let w be the temperature observation value of the i-th reference station. i Let d be the weight of the i-th reference station, n be the number of reference stations, and d be the weight of the i-th reference station. i Let cos be the distance between the i-th reference station and the target station. i Let α be the cosine similarity between the temperature observation data of the i-th reference station and the target station. i Let λ be the complex terrain correlation index between the i-th reference station and the target station, and λ be the smoothing factor.

[0016] Furthermore, in S2, the steps for selecting reference stations are as follows:

[0017] S201. Calculate the similarity deviation C between the temperature observation time series data of each neighboring station and the target station. The smaller the value, the higher the similarity. Let x and y be two temperature observation time series data, and m be the length of the time series. The formula for calculating the similarity deviation is as follows:

[0018]

[0019] in:

[0020] In the formula, R xy The shape coefficient, D, reflects the degree of morphological similarity between time series. xyH is a numerical coefficient that reflects the degree of numerical difference between time series. x (k) and H y (k) represents the k-th time value of time series x and y, respectively, and q1 and q2 are the similarity weighting coefficients of shape coefficient and value coefficient, respectively;

[0021] S202. Calculate the complex terrain correlation index α between each neighboring station and the target station. The smaller the correlation index value, the more similar the complex terrain conditions of the two stations are.

[0022] S203. Using the K-means clustering algorithm, based on the two indicators that measure the correlation between neighboring stations and the target station, namely similarity distance and complex terrain correlation index, the neighboring stations are clustered to obtain the classification criteria for the degree of correlation between neighboring stations and the target station. Stations whose similarity distance value and complex terrain correlation index reach the preset range are marked as 1, and stations that do not reach the preset range are marked as 0. The neighboring stations marked as 1 are selected as reference stations.

[0023] Furthermore, in S202, the complex terrain correlation index α between each neighboring station and the target station is calculated, as follows:

[0024] First, obtain the DEM elevation data of the target area, and extract the topographic parameters of each meteorological station in the target area, including elevation β, slope γ, aspect δ, surface roughness ε, and topographic relief θ.

[0025] Then, the extracted terrain parameter data is subjected to max-min normalization;

[0026] Finally, the complex terrain correlation index between neighboring stations and the target station is calculated using the following formula:

[0027]

[0028] In the formula, α ij β is the complex terrain correlation index between stations i and j. i β j The elevations of stations i and j are respectively, γ i γ j The slopes of stations i and j are δ, respectively. i δ j The slope directions of stations i and j are respectively, ε i ε j θ represents the surface roughness at stations i and j, respectively. i θ j The topographic relief values ​​are denoted as i and j, respectively.

[0029] Furthermore, in S3, assuming the length of the temperature observation time series data is m, the formula for calculating the cosine similarity between the reference station and the target station is as follows:

[0030]

[0031] Where cosθ is the cosine similarity between the reference station and the target station, Y j Let Y be the temperature observation value of the target station at time j. ij Let be the temperature observation value at time j of the i-th reference station.

[0032] Furthermore, in S4, the smoothing factor λ of the prediction model is optimized using the PSO particle swarm optimization algorithm, with the following steps:

[0033] S401. First, initialize the position information x of z particles in D-dimensional space. i and speed information v i Each particle contains a smoothing factor λ value;

[0034] S402. Calculate the fitness function, select the prediction error RMSE as the fitness function, and determine the optimal position p of the individual based on the fitness function value. i and the optimal position of the group g i ;

[0035] S403. For each particle, compare its current fitness value with its individual best fitness value, and update its individual best fitness value and position; compare the individual best fitness values ​​across the entire particle swarm to determine the population's optimal fitness value and position.

[0036] S404. Update the position and velocity of each particle. The formula for updating the position and velocity of the i-th particle is as follows:

[0037]

[0038] in, and These are the position and velocity of particle i in the (k+1)th iteration, respectively, in the d-th dimension. and These are the position and velocity of particle i in the d-th dimension during the k-th iteration. It is the optimal position of particle i in dimension d during the k-th iteration. It is the optimal position in the d-th dimension during the k-th iteration of the population, 1≤d≤D, w is the inertia weight, c1 and c2 are learning factors, and r1 and r2 are random numbers in [0,1].

[0039] S405. In each iteration, repeat S403 and S404 until the number of iterations reaches the set maximum number of iterations, and obtain the smoothing factor λ value with the best prediction result.

[0040] The present invention adopts the above technical solution, and compared with the prior art, the present invention has the following beneficial effects:

[0041] This invention proposes a surface air quality control method based on complex terrain. First, it constructs a complex terrain correlation index to measure the correlation between stations in relation to complex terrain. The correlation between neighboring stations and the target station is measured from two perspectives: temperature data correlation and station-to-complex-terrain correlation. A clustering algorithm is used to select stations with high correlation as reference stations, addressing the problem of current multi-station quality control methods using all neighboring stations as reference stations, resulting in poor quality control performance. Then, a weighting index is constructed using distance, cosine similarity, and the complex terrain correlation index to assign weights to the reference stations, addressing the issue that most current multi-station quality control methods use distance weighting and ignore the impact of complex terrain on quality control. Finally, a particle swarm optimization algorithm is used to optimize the smoothing parameters in the prediction model, further improving the quality control effect.

[0042] The present invention proposes a surface temperature quality control method based on complex terrain, which makes up for the shortcomings of traditional multi-station quality control methods in complex terrain areas. It can effectively identify and mark suspicious temperature data, and further improve the accuracy and completeness of surface temperature data under complex terrain. Attached Figure Description

[0043] Figure 1 This is a flowchart of the method of the present invention.

[0044] Figure 2 This is a site distribution map of the method of the present invention.

[0045] Figure 3 This is an analysis chart showing the effect of the method of this invention, the IDW method, and the SRT method on the quality control of surface air temperature data.

[0046] Figure 4 This is an analysis chart of the error detection results of the method of this invention, the IDW method, and the SRT method on ground temperature data. Detailed Implementation

[0047] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0048] The present invention discloses a method for controlling surface air quality based on complex terrain, the process of which is as follows: Figure 1 As shown, the specific steps include:

[0049] S1. Select the time series data of temperature observation at the target station {Y} t The data includes the time series data of temperature observations at neighboring stations within a set range from the target station, t = 1, 2, 3, ..., T, and the data of temperature observations at neighboring stations within the same time range. it Let ,t=1,2,3,…,T,i=1,2,3,…,N}, where t is time, i is the sequence number of the nearest station, and Yt Let Y be the temperature observation value of the target station at time t. it Let T be the temperature observation value at time t of the i-th reference station, where T is the total time and N is the number of neighboring stations.

[0050] S2. Calculate the similarity C between the temperature observation time series of each neighboring station and the target station and the complex terrain correlation index α between each neighboring station and the target station. Use the K-means clustering algorithm to screen the neighboring stations and select n (1≤n≤N) stations with high correlation as reference stations.

[0051] The specific steps for selecting sites as reference sites are as follows:

[0052] S201. Calculate the similarity deviation C between the temperature observation time series data of each neighboring station and the target station. Similarity deviation is a comprehensive indicator that measures both the similarity of time series values ​​and the similarity of time series shapes. By using similarity deviation, the degree of similarity in both the numerical and shape aspects of the temperature observation time series data between neighboring stations and the target station is determined; the smaller the value, the higher the degree of similarity.

[0053] Let x and y be two time series data points of temperature observation, and m be the length of the time series. Then the formula for calculating the similarity deviation is as follows:

[0054]

[0055] in:

[0056] In the formula, R xy The shape coefficient, D, reflects the degree of morphological similarity between time series. xy H is a value coefficient that reflects the degree of numerical difference between time series. x (k) and H y (k) represents the k-th time value of time series x and y, and q1 and q2 are the similarity weighting coefficients of shape coefficient and value coefficient, respectively.

[0057] S202. Calculate the complex terrain correlation index α between each neighboring station and the target station. The complex terrain correlation index α is a novel indicator proposed in this invention to measure the correlation of complex terrain between different stations. The smaller the value of this index, the more similar the complex terrain conditions of the two stations are. The specific calculation process is as follows:

[0058] (1) Obtain DEM elevation data of the target area and use ArcGIS software to extract the topographic parameters of each meteorological station in the target area, including elevation β, slope γ, aspect δ, surface roughness ε and topographic relief θ, in order to construct the complex terrain correlation index α.

[0059] (2) Perform maximum and minimum normalization on the extracted terrain parameter data to keep them under the same dimension. The calculation formula is as follows:

[0060]

[0061] Where x' is the normalized data, x is the extracted terrain parameter data, and x max and x min These are the maximum and minimum values ​​in the data, respectively.

[0062] (3) Calculate the complex terrain correlation index α. Let the two stations be i and j, then the formula for calculating the complex terrain correlation index between the two stations is as follows:

[0063]

[0064] In the formula, α ij β is the complex terrain correlation index between stations i and j. i β j The elevations of stations i and j are respectively, γ i γ j The slopes of stations i and j are δ, respectively. i δ j The slope directions of stations i and j are respectively, ε i ε j θ represents the surface roughness at stations i and j, respectively. i θ j The topographic relief values ​​are denoted as i and j, respectively.

[0065] S203. The correlation between the target station and the neighboring station is measured by two indicators: the similarity deviation C of the temperature observation time series data between the neighboring station and the target station and the correlation index α of the complex terrain between the neighboring station and the target station. This takes into account both the data similarity between the stations and the similarity of the complex terrain between the stations.

[0066] The smaller the similarity deviation, the smaller the correlation index of complex terrain, which means that the neighboring station is more related to the target station; at the same time, the more related the neighboring station is to the target station, the more concentrated the distribution of these two indicators will be.

[0067] The K-means clustering algorithm is used to cluster neighboring stations based on two indicators that measure the correlation between neighboring stations and the target station: similarity distance and complex terrain correlation index. The neighboring stations are labeled to obtain the classification criteria for the degree of correlation between neighboring stations and the target station. Stations with similarity distance values ​​and complex terrain correlation index that reach the preset range (i.e., stations with high correlation) are labeled as 1, and stations that do not reach the preset range (i.e., stations with low correlation) are labeled as 0. The neighboring stations labeled as 1 are selected as reference stations.

[0068] S3. Calculate the distance *d* between each reference station and the target station, and the cosine similarity *cos* of the temperature observation data. Combine these with the complex terrain correlation index *α* from S2 to construct weights, thereby building a prediction model. Use this model to predict the temperature observation data of the target station and obtain the predicted value. This prediction model is a novel model proposed in this invention, specifically as follows:

[0069]

[0070] Unlike current multi-station quality control methods that often use distance to construct weights, this model uses distance, cosine similarity, and a complex terrain correlation index to construct weight indicators for assigning weights to reference stations. It considers not only distance but also the complex terrain conditions between stations, thus providing a more comprehensive measure of the correlation between reference and target stations. The closer the reference and target stations are, the greater their cosine similarity and the more similar their terrain complexity, the greater the weight assigned to that reference station, and vice versa. Where Y... i Let w be the temperature observation value of the i-th reference station. i Let d be the weight of the i-th reference station, n be the number of reference stations, and d be the weight of the i-th reference station. i Let cos be the distance between the i-th reference station and the target station. i Let α be the cosine similarity between the temperature observation data of the i-th reference station and the target station. i λ is the complex terrain correlation index between the i-th reference station and the target station, and λ is the smoothing factor, which plays a moderating role in the prediction results.

[0071] Let the length of the temperature observation time series be m, then the formula for calculating the cosine similarity between the reference station and the target station is as follows:

[0072]

[0073] Where cosθ is the cosine similarity between the reference station and the target station, Y j Let Y be the temperature observation value of the target station at time j. ij Let be the temperature observation value at time j of the i-th reference station.

[0074] S4. Use the PSO particle swarm optimization algorithm to optimize the smoothing factor λ in S3. When the fitness function value is minimized, the prediction result reaches the optimal state. Select the parameter value in this state as the final λ value.

[0075] S401. First, initialize the position information x of z particles in D-dimensional space. i and speed information v i Each particle contains a smoothing factor λ value.

[0076] S402. Calculate the fitness function. In this embodiment, the prediction error RMSE is selected as the fitness function, and the optimal position p of the individual is determined based on the fitness function value. i and the optimal position of the group g i .

[0077] S403. For each particle, compare its current fitness value with its individual best fitness value, and update its individual best fitness value and position; compare the individual best fitness values ​​across the entire particle swarm to determine the population's optimal fitness value and position.

[0078] S404. Update the position and velocity of each particle. The formula for updating the position and velocity of the i-th particle is as follows:

[0079]

[0080] in, and These are the position and velocity of particle i in the d-th dimension (1≤d≤D) during the k-th iteration. It is the optimal position of particle i in dimension d during the k-th iteration. The optimal position in the d-th dimension during the k-th iteration of the population; w is the inertia weight, which controls the particle's search ability; c1 and c2 are learning factors, which control the particle's self-learning ability and information sharing ability; r1 and r2 are random numbers in [0,1].

[0081] S405. In each iteration, repeat S403 and S404 until the number of iterations reaches the set maximum number of iterations, and obtain the smoothing factor λ value with the best prediction result.

[0082] S5. Evaluate the results using evaluation metrics to assess the model's predictive performance. Evaluation metrics include Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Nash Efficiency Coefficient (NSC).

[0083] Specifically, the magnitude of the error between the predicted and actual values, as well as the degree of fit between the predicted and actual values, are evaluated to assess the quality of the prediction results. The better the predictive performance of the model, the better the quality control effect of using the predicted data on the actual data, and the more accurate the results.

[0084] S6. Calculate the predicted temperature value for each moment. Compare with the observed temperature value Y, if the difference satisfies If the temperature observation data at that moment is considered to have passed the test, then the data is considered to be suspicious and marked accordingly; where f represents the quality control parameter, which is related to the actual quality control effect, and ζ represents the standard error of the observation sequence.

[0085] In this embodiment, Taiyuan Station (station number: 53772) and Lanzhou Station (station number: 52889), two stations in complex terrain areas, are selected as representatives for analysis. All observation stations within 200km of the two stations are taken as neighboring stations. Figure 2 This shows the distribution of stations within a 200km radius of the two target stations, Taiyuan Station and Lanzhou Station. Figure 3 This is a chart analyzing the quality control effects of the method of this invention, the IDW method, and the SRT method on surface temperature data from Taiyuan and Lanzhou stations in 2018. Figure 3 It can be seen that, whether at Taiyuan Station or Lanzhou Station, the MAE, RMSE and NSC of the method of this invention are better than those of the IDW method and SRT method. The quality control effect is significantly improved compared with these two methods. The analysis shows that the quality control effect of the method of this invention is better, more accurate and more adaptable under complex terrain. Figure 4 This is an analysis chart showing the error detection results of the method of this invention, along with the IDW and SRT methods, on 2018 surface temperature data from Taiyuan and Lanzhou stations. Figure 4 It can be seen that, whether at Taiyuan Station or Lanzhou Station, the error detection rate of the method of the present invention is better than that of the IDW method and the SRT method. Analysis shows that the method of the present invention can effectively detect and mark suspicious values ​​in ground temperature data under complex terrain.

Claims

1. A method for controlling surface air quality based on complex terrain, characterized in that, Includes the following steps: S1. Select the temperature observation time series data of the target station, and the temperature observation time series data of neighboring stations within a set range from the target station during the same period; S2. Calculate the similarity distance between the temperature observation time series of each neighboring station and the target station, and the complex terrain correlation index between each neighboring station and the target station. Use the similarity distance and complex terrain correlation index and the K-means clustering algorithm to screen neighboring stations. The screened stations are used as reference stations. Calculate the correlation index of complex terrain between each neighboring station and the target station. The process is as follows: First, obtain the DEM elevation data of the target area, and extract the terrain parameters of each meteorological station in the target area, including altitude. ,slope Slope aspect Surface roughness and terrain relief ; Then, the extracted terrain parameter data is subjected to max-min normalization; Finally, the complex terrain correlation index between neighboring stations and the target station is calculated using the following formula: ， In the formula, For the site and Indices related to complex terrain, , Sites and altitude, , Sites and The slope, , Sites and slope direction, , Sites and Surface roughness, , Sites and The degree of topographic relief; S3. Calculate the distance between each reference station and the target station, as well as the cosine similarity of the temperature observation data. Combine the complex terrain-related index to construct a weight index to assign weights to the reference stations and build a prediction model. The constructed prediction model is as follows: ， ， in, For predicted values, For the first Temperature observations from several reference stations, For the first The weight of each reference site, For reference station numbers, For the first The distance between each reference station and the target station For the first Cosine similarity between temperature observation data from one reference station and one target station For the first The correlation index between the complex terrain of the reference station and the target station. It is a smoothing factor; S4. Use the prediction model to predict the temperature observation data of the target station, and use the PSO particle swarm optimization algorithm to optimize the model parameters of the prediction model. When the fitness function value is minimized, the prediction result reaches the best state, and the parameter value in this state is selected as the final model parameters. S5. Evaluate the prediction results by combining evaluation indicators; S6. Calculate the predicted temperature value for each moment. Compared with temperature observations Compare them, if their difference satisfies If the observed temperature data at that moment passes the test, it is deemed valid; otherwise, the data is deemed suspicious and marked accordingly. Here, f represents the quality control parameter. This represents the standard error of the observed sequence.

2. The surface air quality control method based on complex terrain according to claim 1, characterized in that, In S2, the steps for selecting reference stations are as follows: S201. Calculate the similarity dissimilarity of temperature observation time series data between each neighboring station and the target station. The smaller the value, the higher the similarity. , These are two time series data sets of temperature observations. Given the time series length, the formula for calculating the similarity dissimilarity is as follows: ， in: , , In the formula, The shape coefficient reflects the degree of morphological similarity between time series. The coefficients are numerical values ​​that reflect the degree of difference between time series. and Time series and The Time value, and These are the similarity weighting coefficients for the shape coefficient and the value coefficient, respectively; S202. Calculate the complex terrain correlation index between each neighboring station and the target station. The smaller the value of the correlation index, the more similar the complex terrain conditions of the two stations are; S203. Using the K-means clustering algorithm, based on the two indicators that measure the correlation between neighboring stations and the target station, namely similarity distance and complex terrain correlation index, the neighboring stations are clustered to obtain the classification criteria for the degree of correlation between neighboring stations and the target station. Stations whose similarity distance value and complex terrain correlation index reach the preset range are marked as 1, and stations that do not reach the preset range are marked as 0. The neighboring stations marked as 1 are selected as reference stations.

3. The surface air quality control method based on complex terrain according to any one of claims 1-2, characterized in that, In S3, let the length of the temperature observation time series data be... The formula for calculating the cosine similarity between the reference station and the target station is as follows: ， in, The cosine similarity between the reference station and the target station. For the first The target station temperature observation value at each moment. For the first The first reference station Temperature observation values ​​at each moment.

4. The surface air quality control method based on complex terrain according to claim 1, characterized in that, In S4, the smoothing factor of the prediction model is applied using the PSO particle swarm optimization algorithm. The optimization process involves the following steps: S401, First initialize In 3D space Particle position information and speed information Each particle contains a smoothing factor. value; S402. Calculate the fitness function, select the prediction error RMSE as the fitness function, and determine the optimal position of the individual based on the fitness function value. and the optimal position of the group ; S403. For each particle, compare its current fitness value with its individual best fitness value, and update its individual best fitness value and position; compare the individual best fitness values ​​across the entire particle swarm to determine the population's optimal fitness value and position. S404, Update the position and velocity of each particle, the... The formulas for updating the position and velocity of each particle are as follows: ， ， in, and They are particles No. During the nth iteration Dimensional position and velocity, and They are particles No. During the nth iteration Dimensional position and velocity, It is a particle No. During the nth iteration The optimal position of an individual in dimension. It is the first in the population During the nth iteration The optimal position of the dimension. , For inertial weights, , As a learning factor, , for Random numbers within; S405. In each iteration, repeat S403 and S404 until the set maximum number of iterations is reached, thus obtaining the smoothing factor that yields the best prediction results. value.

Images