Intelligent interpolation method for laser radar data missing

Abstract

The invention provides an intelligent interpolation method for laser radar data missing, and the method comprises the steps: firstly obtaining long-time-sequence laser radar data, carrying out the preliminary screening and processing of the data through a quality control program, and then carrying out the classification of possible missing data. According to the method, linear function interpolation is executed on the condition of partial distance gate data missing caused by weather influence, values of missing data points are quickly and effectively estimated by utilizing a linear relation, and the continuity of the data can be well recovered in a scene that the data change is relatively stable. And for the condition of full-distance layer data missing in the time sequence caused by blade shielding and the like, a bidirectional recursion time sequence interpolation model based on LSTM is adopted, the advantages of a deep learning model in the aspect of processing time sequence data are fully played, data in a missing time period are accurately filled through historical data learning and bidirectional recursion prediction, and the accuracy of time sequence interpolation is improved. Therefore, the integrity and availability of the laser radar data are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of laser radar technology, and in particular to an intelligent interpolation method for missing laser radar data. Background Art

[0002] In the development and utilization of wind energy, accurate measurement of wind speed and direction is crucial for the site selection of wind farms, control and optimization of wind turbines, and wind power testing. The wind radar is installed on the top of the wind turbine nacelle. It uses a coherent detection system based on the principle of laser Doppler frequency shift to achieve accurate measurement of the wind vector field from the top of the nacelle to tens to hundreds of meters in front of the nacelle impeller. It can measure the wind speed directly in front of the wind turbine impeller, and its measurement results are not affected by the wind turbine wake, which can be effectively applied to feedforward control and power curve testing, etc.

[0003] However, due to some uncontrollable reasons, lidar data is easily lost. Under the influence of some bad weather, for example, in foggy weather, the radar signal may be scattered and absorbed by the fog, resulting in the loss of wind speed and direction data. When the impeller is blocked, electromagnetic interference and equipment failure occur, the reception and transmission of the signal are disturbed, which will also affect the normal data collection.

[0004] In the wind power business, the most commonly used wind data interpolation method is the MCP (Measurement-Correlation-Prediction) method. This method assumes that the wind field has spatial correlation and the correlation is strong. On the basis of the known wind speed correlation between the target station and the reference station during the synchronization period, the wind speed of the target station is predicted based on the long-term data of the reference station. This method requires the reference station to have long-term and high-complete data. At the same time, under complex terrain, the wind speed of the reference station and the target station is complex and changeable in space, with low or even no correlation. The traditional MCP method has low accuracy or is even ineffective. Summary of the invention

[0005] In order to solve the problems of the prior art, the present invention provides an intelligent interpolation method for missing lidar data. According to the performance of missing lidar data, two solutions to the problem of missing data are designed. The solutions can adapt to different weather conditions and signal interference situations, and provide a strong guarantee for the integrity of lidar data.

[0006] An intelligent interpolation method for missing laser radar data, characterized by comprising the following steps:

[0007] 1) Obtain laser radar related data;

[0008] 2) Use quality control procedures to conduct preliminary screening and processing of data;

[0009] 3) Classify missing data, including missing data in the far end caused by signal attenuation due to bad weather and missing data in a continuous period of time due to leaf shielding or signal interference;

[0010] 4) Data missing interpolation method:

[0011] 4.1) Linear function interpolation: For the missing data in the far end caused by signal attenuation due to bad weather, the linear interpolation function is used for interpolation;

[0012] 4.2) Bidirectional recursive time series interpolation model based on LSTM: For the case of missing data in a continuous time period and discontinuous time series due to blade occlusion or signal interference, a bidirectional recursive time series interpolation model based on LSTM is used. The front sequence is forward propagated to predict the wind speed at time t, and the rear sequence is backward propagated to predict the wind speed at time t. The average of the two model outputs is taken as the final prediction result;

[0013] 5) Missing data filling: Use the predicted values ​​obtained in the previous steps to fill in the missing data, integrate the predicted values ​​with the known data, and ensure the continuity of the data in the time series.

[0014] As a further improvement, in the process of obtaining the lidar related data in step 1), the focus is on collecting the ten-minute statistical wind speed data information of the cabin wind measuring lidar, covering the changes in wind speed in different time periods, and strictly sorting the collected data in chronological order.

[0015] As a further improvement, the preliminary screening and processing of data using a quality control program in step 2) includes two aspects, including screening outliers and data with low efficiency.

[0016] As a further improvement, the missing of the far-end data caused by signal attenuation due to bad weather in step 3) is mainly manifested as missing of the farthest data, and in extreme weather conditions, it is manifested as missing of all data.

[0017] As a further improvement, the complete lack of data in the continuous time period due to blade shading or signal interference in step 3) is mainly manifested in the complete lack of all range gate data, incomplete wind speed time series, and fragmentary lack.

[0018] Further improvement, the linear function interpolation in step 4.1) is specifically that the wind speed data of different distance layers at the same time are described by a linear relationship. A series of wind speed data points of multiple layers of range gates are known, and a linear function y=ax+b is determined by the least squares method, where a and b are coefficients obtained by fitting the least squares method. For the range gate d with missing data, the corresponding wind speed interpolation value is obtained by calculating y(d)=ad+b.

[0019] Further improvement, step 4.2) the pre-sequence is a positive pre-sequence forward propagation LSTM model, X t is the input of the LSTM model at time t. Similarly, X t-1 and X t-2 Input for the LSTM model at time t-1 and t-2; is the output of the LSTM model at time t. Similarly, and The LSTM model output at time t-1 and t-2; and It is the hidden layer information, which is only transmitted in the model and not output;

[0020] Given a set of lidar wind speed values ​​sorted by time, where the horizontal rows represent the data of 1-10 range gates and the vertical rows represent the time series;

[0021]

[0022] The input mode of the rearranged and combined organization model is set as the 6-hour data before the target time point. The wind speed data is sampled at an interval of 10 minutes. Therefore, the length of the 6-hour time queue is 36. At the same time, there are 10 different distance layers in the data, and each distance layer corresponds to a time series with a length of 36. Finally, the size of the model input is 36×10, that is,

[0023]

[0024] The output of the model is the predicted wind speed values ​​of the 10 range gates at time t, so the size is 1×10;

[0025]

[0026] Step 4.2) The post-sequence is a reverse post-sequence back-propagation LSTM model, including data for 6 hours after time t.

[0027] Further improvement: In the LSTM-based bidirectional recursive time series interpolation model described in step 4.2), the following method is used to optimize and improve the model:

[0028] 4.21) Try different sampling intervals and time window lengths to find the parameter settings that best suit your specific dataset;

[0029] 4.22) Introduce regularization technology to prevent model overfitting and improve the generalization ability of the model;

[0030] 4.23) Combine with other meteorological data or features to further enrich the model input information and improve the accuracy of the prediction.

[0031] As a further improvement, in the missing data filling process of step 5), in order to verify the accuracy and reliability of the filling results, a quality assessment is performed on the filled data set.

[0032] As a further improvement, the quality assessment process specifically calculates some statistical indicators, including mean absolute error and root mean square error, and compares them with the original data to determine whether the filled data set is within a reasonable error range, or displays the data changes before and after filling in a visual way to intuitively observe the filling effect.

[0033] The beneficial effects of the present invention are:

[0034] 1. According to the performance of missing LiDAR data, the present invention designs two solutions to solve the problem of missing data. The linear interpolation solution has high execution efficiency when solving a small amount of missing data, and can complete the interpolation operation in a relatively short time, ensuring the timeliness of data processing. For the case where all range gates are missing and the time series is discontinuous, a bidirectional recursive time series interpolation model based on LSTM is adopted, which gives full play to the advantages of deep learning models in processing time series data, performs well in processing complex data missing situations, can adapt to different weather conditions and signal interference situations, and provides a strong guarantee for the integrity of LiDAR data.

[0035] 2. The bidirectional recursive time series interpolation model based on LSTM designed by the present invention can make full use of the previous and next information in the time series data by considering both the positive order preceding sequence and the reverse order following sequence, and enhance the model's ability to capture the trend of wind speed changes. For lidar wind speed data, the change of wind speed is often affected by many factors, which is not only related to the past wind speed state, but also may be affected by future trends. The bidirectional propagation method allows the model to analyze the data from two directions, so as to more comprehensively understand the dynamic characteristics of wind speed.

[0036] 3. The bidirectional recursive time series interpolation model based on LSTM designed by the present invention takes the mean of the outputs of the two LSTM models as the final prediction result, which helps to reduce the uncertainty of single-direction prediction. In practical applications, due to the complexity and uncertainty of the data, single-direction predictions may be affected by noise, outliers or model bias. By combining the prediction results of the forward and reverse order, these influences can be balanced to a certain extent, improving the stability and accuracy of the prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 It is a technical flow chart of the present invention.

[0039] Figure 2 Schematic diagram of missing data classification.

[0040] Figure 3 Schematic diagram of the bidirectional recursive time series interpolation model based on LSTM. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] A specific embodiment of the present invention is as follows Figure 1 As shown, the following steps are included:

[0043] Step 1: Obtain lidar related data.

[0044] In the cabin wind measurement environment, professional data collection equipment and technology are used to collect relevant data obtained by the lidar. The focus is on collecting the ten-minute statistical wind speed data information of the cabin wind measurement lidar, which covers the changes in wind speed in different time periods. In order to facilitate subsequent analysis and processing, the collected data is strictly sorted in time series in chronological order to ensure the logic and coherence of the data, so as to clearly reflect the changes in wind speed over time.

[0045] Step 2: Quality control procedures.

[0046] Quality control of lidar wind speed data mainly includes two aspects. First, strict screening of singular values. Singular values ​​may be data anomalies caused by equipment failure, external interference or other abnormal conditions. These outliers will seriously interfere with the subsequent analysis and interpretation of the data, so they need to be accurately identified through specific algorithms and technical means. Second, screening of low data efficiency. Data efficiency reflects the reliability and availability of data. Low data efficiency may mean that there are a lot of missing, erroneous or inaccurate data. These poor quality data can be removed through screening.

[0047] Step three: classify the missing data.

[0048] The reasons for missing LiDAR data can be divided into two categories. One is that the signal is attenuated due to bad weather, and the echo signal of the long-distance layer cannot be received. It is mainly manifested in the missing of the farthest data, but in extreme weather, all data may be missing. The other is the missing of data in a continuous period of time due to other reasons such as blade shading or signal interference. It is mainly manifested in the complete missing of all range gate data, incomplete wind speed time series, and fragmentary missing. For the case of partial range gate missing (such as Figure 2 In part a of the figure, a linear interpolation function can be used for interpolation. For the case where all range gates are missing and the timing is discontinuous (such as Figure 2 Part b) adopts a bidirectional recursive time series interpolation model based on LSTM.

[0049] Step 4: Missing data interpolation method.

[0050] Linear function interpolation

[0051] There is a strong spatial correlation between wind speed data at different distance layers at the same time, which can usually be described by a simple linear relationship. For example, given a series of wind speed data points at multiple range gates, a linear function y=ax+b is determined by the least squares method, where a and b are coefficients obtained by fitting the least squares method. For range gate d with missing data, the corresponding wind speed interpolation value can be obtained by calculating y(d)=ad+b.

[0052] (2) Bidirectional recursive time series interpolation model based on LSTM

[0053] Long Short-Term Memory (LSTM) is a special type of Recurrent Neural Network (RNN), which is mainly used to process time series data and problems with long-term dependencies.

[0054] The core of LSTM is the cell state, which is like a conveyor belt of information that runs through the entire network. The cell state can save and pass information during the processing of the sequence, allowing LSTM to effectively handle long-term dependencies. The cell state can be regarded as the "memory" part of the network, which can remember long-term information without gradually forgetting it over time like traditional RNNs.

[0055] LSTM contains three main gating structures: input gate, forget gate, and output gate. These gating structures update cell states and generate outputs by controlling the flow of information. The input gate determines how much of the current input information is stored in the cell state. The forget gate determines how much of the cell state at the previous moment is retained to the current moment. The output gate determines how much of the cell state at the current moment is output to the hidden state.

[0056] This technology proposes a bidirectional recursive time series interpolation model based on LSTM, whose structure is as follows Figure 3 shown.

[0057] In the forward propagation LSTM model with a positive preamble, X t is the input of the LSTM model at time t. Similarly, X t-1 and X t-2 Input to the LSTM model at time t-1 and t-2. is the output of the LSTM model at time t. Similarly, and It is the LSTM model output at time t-1 and t-2. and It is the hidden layer information, which is only transmitted in the middle of the model and not output.

[0058] Given a set of lidar series wind speed values ​​sorted by time, the horizontal rows represent the data of 1-10 range gates and the vertical rows represent the time series.

[0059]

[0060] The input mode of the model is rearranged and organized, and the input is set to the 6-hour data before the target time point. The wind speed data is sampled at an interval of 10 minutes, so the length of the 6-hour time queue is 36. At the same time, there are 10 different distance layers in the data, and each distance layer corresponds to a time series of length 36. Finally, the size of the model input is 36×10, that is,

[0061]

[0062] The output of the model is the predicted wind speed values ​​of the 10 range gates at time t, so the size is 1×10.

[0063]

[0064] The difference between the forward propagation LSTM model with the positive pre-sequence and the backward propagation LSTM model with the reverse post-sequence is that the pre-sequence is the data of 6 hours before time t, and the post-sequence is the data of 6 hours after time t. The forward propagation of the pre-sequence predicts the wind speed at time t, and the backward propagation of the post-sequence predicts the wind speed at time t. The average of the outputs of the two models is taken as the final prediction result.

[0065] The characteristics of the LSTM-based bidirectional recursive time series interpolation model: By considering both the positive order preceding sequence and the reverse order following sequence, it can make full use of the previous and next information in the time series data, and enhance the model's ability to capture the trend of wind speed changes. At the same time, by combining the prediction results of the positive order and the reverse order, it helps to reduce the uncertainty of single-direction prediction and improve the stability and accuracy of the prediction.

[0066] In practical applications, the model can be further optimized and improved. For example, different sampling intervals and time window lengths can be tried to find the parameter settings that best suit a specific data set. At the same time, regularization techniques can be introduced to prevent overfitting of the model and improve the generalization ability of the model. In addition, other meteorological data or features, such as temperature, air pressure, humidity, etc., can be combined to further enrich the input information of the model and improve the accuracy of the prediction.

[0067] Step 5: Fill in missing data.

[0068] Use the predicted values ​​obtained in the previous steps to fill in the missing data, integrate the predicted values ​​with the known data, and ensure the continuity of the data in the time series. At the same time, in order to verify the accuracy and reliability of the filling results, the quality of the filled data set can be evaluated. Some statistical indicators, such as mean absolute error, root mean square error, etc., can be calculated and compared with the original data to determine whether the filled data set is within a reasonable error range. In addition, the data changes before and after filling can be displayed in a visual way to intuitively observe the filling effect.

[0069] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, the above is only a preferred embodiment of the present invention. Since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited to this. Any technician familiar with the technical field is within the technical scope disclosed by the present invention. For ordinary technicians in the technical field, changes or replacements that can be easily thought of should be covered within the protection scope of the present invention without departing from the principle of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An intelligent interpolation method for missing laser radar data, characterized in that The following steps are involved: 1) Obtain laser radar related data; 2) Use quality control procedures to conduct preliminary screening and processing of data; 3) Classify missing data, including missing data in the far end caused by signal attenuation due to bad weather and missing data in a continuous period of time due to leaf shielding or signal interference; 4) Data missing interpolation method: 4.1) Linear function interpolation: For the missing data in the far end caused by signal attenuation due to bad weather, the linear interpolation function is used for interpolation; 4.2) Bidirectional recursive time series interpolation model based on LSTM: For the case of missing data in a continuous time period and discontinuous time series due to blade occlusion or signal interference, a bidirectional recursive time series interpolation model based on LSTM is used. The front sequence is forward propagated to predict the wind speed at time t, and the rear sequence is backward propagated to predict the wind speed at time t. The average of the two model outputs is taken as the final prediction result; 5) Missing data filling: Use the predicted values ​​obtained in the previous steps to fill in the missing data, integrate the predicted values ​​with the known data, and ensure the continuity of the data in the time series.

2. The intelligent interpolation method for missing laser radar data according to claim 1, characterized in that: In the process of obtaining the laser radar related data in step 1), the focus is on collecting the ten-minute statistical wind speed data information of the cabin wind measurement laser radar, covering the changes in wind speed in different time periods, and strictly sorting the collected data in chronological order.

3. The intelligent interpolation method for missing laser radar data according to claim 1, characterized in that: The preliminary screening and processing of data using a quality control program in step 2) includes two aspects, including screening outliers and data with low efficiency.

4. The intelligent interpolation method for missing laser radar data according to claim 1, characterized in that: The missing of the far-end data caused by signal attenuation due to bad weather in step 3) is mainly manifested as missing of the farthest data, and in extreme weather conditions, it is manifested as missing of all data.

5. The intelligent interpolation method for missing laser radar data according to claim 1, characterized in that: Step 3) The complete loss of data in a continuous period of time due to blade shading or signal interference is mainly manifested in the complete loss of all range gate data, incomplete wind speed time series, and fragmentary loss.

6. The intelligent interpolation method for missing laser radar data according to claim 1, characterized in that: The linear function interpolation described in step 4.1) is specifically as follows: the wind speed data of different distance layers at the same time are described by a linear relationship. A series of wind speed data points of multiple layers of range gates are known, and a linear function y=ax+b is determined by the least squares method, where a and b are coefficients obtained by fitting the least squares method. For the range gate d with missing data, the corresponding wind speed interpolation value is obtained by calculating y(d)=ad+b.

7. The intelligent interpolation method for missing laser radar data according to claim 1, characterized in that: Step 4.2) The pre-sequence is a positive pre-sequence forward propagation LSTM model, X t is the input of the LSTM model at time t. Similarly, X t-1 and X t-2 Input for the LSTM model at time t-1 and t-2; is the output of the LSTM model at time t. Similarly, and The LSTM model output at time t-1 and t-2; and It is the hidden layer information, which is only transmitted in the model and not output; Given a set of lidar wind speed values ​​sorted by time, where the horizontal rows represent the data of 1-10 range gates and the vertical rows represent the time series; The input mode of the rearranged and combined organization model is set as the 6-hour data before the target time point. The wind speed data is sampled at an interval of 10 minutes. Therefore, the length of the 6-hour time queue is 36. At the same time, there are 10 different distance layers in the data, and each distance layer corresponds to a time series with a length of 36. Finally, the size of the model input is 36×10, that is, The output of the model is the predicted wind speed values ​​of the 10 range gates at time t, so the size is 1×10; Step 4.2) The post-sequence is a reverse post-sequence back-propagation LSTM model, including data for 6 hours after time t.

8. The intelligent interpolation method for missing laser radar data according to claim 1, characterized in that: In the LSTM-based bidirectional recursive time series interpolation model described in step 4.2), the following method is used to optimize and improve the model: 4.21) Try different sampling intervals and time window lengths to find the parameter settings that best suit your specific dataset; 4.22) Introduce regularization technology to prevent model overfitting and improve the generalization ability of the model; 4.23) Combine with other meteorological data or features to further enrich the model input information and improve the accuracy of the prediction.

9. The intelligent interpolation method for missing laser radar data according to claim 1, characterized in that: In step 5), during the missing data filling process, in order to verify the accuracy and reliability of the filling results, the quality of the filled data set is evaluated.

10. The intelligent interpolation method for missing laser radar data according to claim 1, characterized in that: The quality assessment process specifically calculates some statistical indicators, including mean absolute error and root mean square error, and compares them with the original data to determine whether the filled data set is within a reasonable error range, or displays the data changes before and after filling in a visual way to intuitively observe the filling effect.

Images