Data quality control method of wind measurement laser radar

Abstract

The invention relates to the technical field of wind lidar measurement, and discloses a wind lidar data quality control method comprising the following steps: collecting wind lidar echo signals in a period of time, the wind lidar echo signals comprising atmospheric reflection signals and noise signals; carrying out average power spectrum estimation processing on the original echo signal; gaussian filtering processing is carried out on the average power spectrum so as to suppress obvious and burst noise and interference signals, and a calculation result is used for replacing an original power spectrum numerical value; the filtered power spectrum is obtained, a least square method mathematical optimization method is adopted, model parameters of optimal fitting given power spectrum data points are searched, a prediction model adopts a low-order polynomial, and whole range gate sequence data are selected for prediction fitting. Through power spectrum estimation and noise frequency domain statistical characteristic analysis, the signal and the noise can be distinguished more accurately, so that the influence of the noise on a measurement result is effectively reduced.

Description

Technical Field

[0001] The invention relates to the technical field of wind laser radar measurement, and in particular to a data quality control method for a wind laser radar. Background Art

[0002] In order to achieve quantitative intensity measurement of wind lidar, one of the key technologies is to accurately calculate the noise power from the power spectrum data of the wind lidar to remove background noise and achieve data quality control. Unlike traditional weather radars that measure intensity in the time domain, wind lidar extracts information in the frequency domain. Its working principle is to first estimate the power spectrum based on the observation sequence, and then derive all the required meteorological information from the power spectrum. The power spectrum contains information about the scattering intensity and movement speed of the target, so the extraction of all information can be attributed to the reprocessing of the power spectrum.

[0003] The signal received by the radar inevitably contains noise, which means that the power spectrum of the wind lidar is actually the superposition of the power spectrum of the meteorological signal and the power spectrum of the noise signal. When determining the average speed of the target from the power spectrum, no quantitative measurement of the intensity is required, so the noise problem can be ignored. However, if the purpose is to determine the scattering intensity of the target from the power spectrum, the noise power must be determined first.

[0004] Weather radars usually use the return signal power of the long-range gate as the standard for the noise power of each range gate in the radial direction. This method assumes that the return signal of the long-range gate is all noise and the meteorological signal is small enough to be ignored. Although this method is suitable for weather radars, because their detection distance is long, the echo of the long-range gate has attenuated to the point where it is almost undetectable even if it contains meteorological signals. However, this method is not suitable for wind lidar.

[0005] To this end, the present invention provides a data quality control method for a wind laser radar. Summary of the invention

[0006] In view of the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a data quality control method for wind measuring lidar, so as to provide a more accurate noise power calculation means, thereby improving the measurement accuracy of wind measuring lidar under complex meteorological conditions and achieving data quality control.

[0007] In order to achieve the above-mentioned purpose, the present invention provides the following technical solution: a data quality control method for a wind laser radar, comprising: Step 1: Collect the wind laser radar echo signal within a period of time, wherein the wind laser radar echo signal includes the atmospheric reflection signal and the noise signal, and perform average power spectrum estimation processing on the original echo signal; Step 2: Perform Gaussian filtering on the average power spectrum to suppress obvious and sudden noise and interference signals, and use the calculated result to replace the original power spectrum value; Step 3: Obtain the power spectrum after filtering by using the least squares mathematical optimization method to find the model parameters that best fit the given power spectrum data points. The prediction model uses a low-order polynomial and selects the entire range gate sequence data for prediction fitting; Step 4: Obtain the filtered power spectrum minus the solved polynomial noise power, and obtain the required meteorological useful signal based on the result.

[0008] In some implementations, the specific method of performing average power spectrum estimation processing on the original echo signal is: The N-point sample value is taken as the energy-limited signal, and its Fourier transform is obtained. , and then take the square of its magnitude and divide it by N to get Estimation of the average power spectrum of .

[0009] In some implementations, the calculation formula for performing average power spectrum estimation processing on the original echo signal is: In the process of estimating the average power spectrum of the original echo signal, the wind laser radar is used to continuously collect the echo signal within a certain period of time.

[0010] In some embodiments, the preprocessed echo signal is converted into a frequency domain representation by fast Fourier transform, and the power spectrum of the signal in the frequency domain is calculated and averaged over multiple time windows.

[0011] In some embodiments, a suitable window size N and standard deviation of the Gaussian function are selected. ; For each data point, calculate the Gaussian weighted average of the N points around it. For example, when the window size is 5 and the standard deviation is 1, calculate The Gaussian weighted average of is: And use the above calculation results to replace the original power spectrum values.

[0012] In some embodiments, the prediction model adopts a first-order prediction model, and a set of power spectrum data points is , i=1,2,…,n, the target model is a first-order polynomial , solve the target parameters a and b, and define the sum of squared differences as S: To minimize S, find the partial derivatives of a and b and set them equal to zero, that is: Rearranging the two equations, we get: Solve the linear equations to obtain the target parameters a and b, then the polynomial This is the noise power model.

[0013] In some embodiments, when modeling the filtered power spectrum data, a low-order polynomial is used as a prediction model, and it is suitable for describing the basic trend of the data. After solving the optimal parameters of the polynomial model, a fitting curve is generated and compared with the original filtered data, and the fitting effect of the model is evaluated by residual analysis.

[0014] In some embodiments, after filtering, the power spectrum data before and after filtering are compared to verify the noise suppression effect, and the performance of the filter is evaluated by calculating the signal-to-noise ratio index to ensure that it effectively removes unwanted noise components without significantly weakening the useful signal.

[0015] In some embodiments, the calculated wind speed and direction information is formatted into an easy-to-use report form and provided to the end user, including generating images, time series data or statistical analysis reports, and ensuring that the output results are fully verified and accompanied by necessary metadata for subsequent analysis and application.

[0016] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the above-mentioned data quality control method of a wind laser radar.

[0017] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects: First, accurately separate signals and noise: Through power spectrum estimation and noise frequency domain statistical characteristics analysis, signals and noise can be distinguished more accurately, thereby effectively reducing the impact of noise on measurement results.

[0018] Second, improve measurement accuracy: the original power spectrum is corrected through the calculated noise power model, which significantly improves the reliability and consistency of the data. After reducing noise interference, the measurement results of the wind measuring lidar are more accurate, which is very important for applications such as meteorological monitoring, wind energy assessment, and environmental monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1A schematic diagram of a flow chart of a data quality control method for a wind laser radar according to the present invention; Figure 2 This is a result diagram of the average power spectrum estimation in step 1 in an embodiment of the present invention; Figure 3 This is a result diagram of power spectrum estimation after Gaussian filtering in step 2 of the embodiment of the present invention; Figure 4 This is a result diagram of calculating the noise power spectrum model by a specific algorithm in step 3 of the embodiment of the present invention; Figure 5 This is a result diagram of the effective atmospheric signal component after filtering out the noise power spectrum in an embodiment of the present invention; DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0022] See also Figure 1 The present invention provides a data quality control method for a wind laser radar, the method comprising the following steps: Step 1: Collect the wind laser radar echo signal within a period of time, wherein the wind laser radar echo signal includes the atmospheric reflection signal and the noise signal, and perform average power spectrum estimation processing on the original echo signal; Step 2: Perform Gaussian filtering on the average power spectrum to suppress obvious and sudden noise and interference signals, and use the calculated result to replace the original power spectrum value; Step 3: Obtain the power spectrum after filtering by using the least squares mathematical optimization method to find the model parameters that best fit the given power spectrum data points. The prediction model uses a low-order polynomial and selects the entire range gate sequence data for prediction fitting; Step 4: Obtain the filtered power spectrum minus the solved polynomial noise power, and obtain the required meteorological useful signal based on the result.

[0023] It should be noted that the specific method of performing average power spectrum estimation processing on the original echo signal is: The N-point sample value is taken as the energy-limited signal, and its Fourier transform is obtained. , and then take the square of its magnitude and divide it by N to get Estimation of the average power spectrum of , the specific calculation formula is as follows: In the process of estimating the average power spectrum of the original echo signal, the wind laser radar is used to continuously collect echo signals over a certain period of time. These signals are usually composed of signals reflected by aerosols, molecules and other particles when the laser beam propagates in the atmosphere. The echo signal consists of two parts: the useful reflection signal (return from the atmosphere) and the noise signal (including electromagnetic interference, thermal noise and the background noise generated by the equipment itself).

[0024] The preprocessed echo signal is converted into a frequency domain representation, usually through fast Fourier transform (FFT). The purpose of this step is to analyze the frequency components of the signal, calculate the power spectrum of the signal in the frequency domain, and average the power spectrum over multiple time windows. This process helps to smooth out occasional, instantaneous noise interference and improve the signal-to-noise ratio of the signal. The average power spectrum estimation can better capture the frequency characteristics of useful signals while suppressing random noise by focusing on specific frequency components of the signal. The above steps can lay a solid foundation for subsequent data processing and analysis, ensuring that the subsequent filtering and mathematical optimization processes can be based on high-quality input data.

[0025] When performing power spectrum Gaussian filtering on the average power spectrum, the specific steps are as follows: First select a suitable window size N and the standard deviation of the Gaussian function ; For each data point, calculate the Gaussian weighted average of the N points around it. For example, when the window size is 5 and the standard deviation is 1, calculate The Gaussian weighted average of is: The above calculation results are used to replace the original power spectrum values, and the filtered power spectrum data will be used to replace the original power spectrum values. This means that subsequent analysis and processing will be based on the power spectrum after noise suppression. The purpose of performing power spectrum Gaussian filtering on the average power spectrum is to reduce the noise and interference in the wind laser radar echo signal through filtering technology, avoid narrow and strong interference signals being mistaken for atmospheric turbulence signals, and improve the ability to identify spectrum peaks.

[0026] In the process of signal processing, Gaussian filtering is a technique commonly used to smooth data and suppress noise. Its characteristic is that the smoothing of the signal can effectively reduce burst noise while retaining the overall trend of the signal. The average power spectrum is used as input data and processed by a Gaussian filter. The Gaussian filter performs a weighted average on each data point in the power spectrum, and the weight is determined by the Gaussian function. This process can eliminate sharp noise peaks in the power spectrum caused by electronic interference, environmental fluctuations and other factors, while retaining the main characteristics of the atmospheric reflection signal.

[0027] The filtering effect is verified after filtering, and the performance of the filter can be evaluated by calculating the signal-to-noise ratio or other statistical indicators to ensure that it effectively removes unwanted noise components without significantly weakening the useful signal.

[0028] As a preferred embodiment of the present invention, the prediction model adopts a low-order polynomial, including first-order and second-order. Taking the first-order prediction model as an example, assuming that a set of power spectrum data points , i=1,2,…,n, the target model is a first-order polynomial , solve the target parameters a and b, and define the sum of squared differences as S: To minimize S, find the partial derivatives of a and b and set them equal to zero, that is: Rearranging the two equations, we get: Solve the linear equations to obtain the target parameters a and b, then the polynomial The above is a noise power model. The polynomial fitting algorithm is used to calculate the noise power corresponding to the close and long distance gates, which effectively improves the problem of using the return signal of the long distance gate as the standard of noise power, which brings significant errors to the calculation of spectrum parameters.

[0029] When modeling the filtered power spectrum data, a low-order polynomial (such as a first-order or second-order polynomial) is used as the prediction model. The low-order polynomial is simple enough to avoid overfitting and is suitable for describing the basic trend of the data. After solving the optimal parameters of the polynomial model, a fitting curve is generated and compared with the original filtered data. The fitting effect of the model is evaluated by residual analysis (i.e., the difference between the actual value and the fitted value). If the residuals are randomly distributed and have no obvious pattern, it indicates that the model fits well. If necessary, verify the generalization ability of the model through cross-validation or use another set of sample data to ensure its reliability on different data sets.

[0030] More specifically, after subtracting the solved polynomial noise power from the filtered power spectrum, the required meteorological useful signal can be obtained to reduce calculation errors, achieve accurate intensity measurement, and realize data quality control of wind lidar. The power spectrum after subtracting the noise model mainly contains the characteristics of meteorological signals. Through this purification, the signal is more conspicuous and easy to identify reliably. The final data should be quality controlled and verified to ensure that it is suitable for application requirements. Through these steps, ensuring that the meteorological signal extracted from the filtered power spectrum is accurate and reliable not only helps to achieve high-quality data analysis, but also provides a solid foundation for practical applications. And the calculated wind speed and direction information can be formatted into an easy-to-use report form and provided to the end user. This may include generating images, time series data, or statistical analysis reports. Ensure that the output results are fully verified and accompanied by necessary metadata (such as timestamps, locations, and environmental conditions) for subsequent analysis and application.

[0031] In general, the present invention aims to design a data quality control method for a wind laser radar. The detection distance of the wind laser radar is relatively close to that of the weather radar, and its detection sensitivity is higher than that of the weather radar. Under clear sky conditions, it may be reasonable to think that the signals of the long-distance gate of the wind laser radar are all noise. However, when there is weather activity (especially precipitation activity), the meteorological signal contained in the echo of the long-distance gate cannot be ignored. At this time, if the return signal of the long-distance gate is still used as the standard of noise power, it will bring significant errors to the calculation of the spectrum parameters. In addition, if a strong instantaneous clutter interference signal appears in the long-distance gate, it will also cause calculation errors. In view of the irrationality of the current noise power acquisition, in order to realize the quantitative measurement of the intensity of the wind laser radar, it is necessary to explore a new method for determining the noise power to achieve data quality control. Based on the power spectrum estimation method of the wind laser radar and the frequency domain statistical characteristics of the noise, the present invention proposes a new method to accurately calculate the noise power of the wind laser radar and realize data quality control. This method is expected to overcome the limitations of traditional methods, provide a more accurate noise power calculation method, and thus improve the measurement accuracy of the wind laser radar under complex meteorological conditions and realize data quality control. The method designed by the present invention can accurately separate signals and noise: by subtracting the fitted polynomial noise model, the background noise is effectively removed, making the meteorological signal more significant and clear. The calculation accuracy of signal parameters (such as amplitude and frequency) is improved, the measurement error is reduced, and the reliability of the data is improved. Through power spectrum estimation and noise frequency domain statistical characteristics analysis, the signal and noise can be distinguished more accurately, thereby effectively reducing the impact of noise on the measurement results; and the original power spectrum is corrected by the calculated noise power model, which significantly improves the reliability and consistency of the data. After reducing noise interference, the measurement results of the wind laser radar are more accurate, which is very important for applications such as meteorological monitoring, wind energy assessment, and environmental monitoring. And because this method can accurately identify and process a variety of noise patterns, it can adapt to different measurement environments and conditions, increasing the versatility of the system.

[0032] The embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. The embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit, the above functions defined in the method of the present application are executed. It should be noted that the computer-readable medium mentioned above in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of an electrical, magnetic, optical, electromagnetic, infrared segment, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wire segments, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless segments, wire segments, optical cables, RF, etc., or any suitable combination of the above.

[0033] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0034] Those skilled in the art should understand that the above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A data quality control method for a wind laser radar, characterized in that: The method comprises the following steps: Step 1: Collect the wind laser radar echo signal within a period of time, wherein the wind laser radar echo signal includes the atmospheric reflection signal and the noise signal, and perform average power spectrum estimation processing on the original echo signal; Step 2: Perform Gaussian filtering on the average power spectrum to suppress obvious and sudden noise and interference signals, and use the calculated result to replace the original power spectrum value; Step 3: Obtain the power spectrum after filtering by using the least squares mathematical optimization method to find the model parameters that best fit the given power spectrum data points. The prediction model uses a low-order polynomial and selects the entire range gate sequence data for prediction fitting; Step 4: Obtain the filtered power spectrum minus the solved polynomial noise power, and obtain the required meteorological useful signal based on the result.

2. The data quality control method of a wind laser radar according to claim 1, characterized in that: The specific method of processing the average power spectrum of the original echo signal is as follows: The N-point sample value is taken as the energy-limited signal, and its Fourier transform is obtained. , and then take the square of its magnitude and divide it by N to get Estimation of the average power spectrum of .

3. The data quality control method of a wind laser radar according to claim 2 is characterized in that: The calculation formula for the average power spectrum estimation of the original echo signal is: In the process of estimating the average power spectrum of the original echo signal, the wind laser radar is used to continuously collect the echo signal within a certain period of time.

4. The data quality control method of a wind laser radar according to claim 3 is characterized in that: The preprocessed echo signal is converted into frequency domain representation by fast Fourier transform, and the power spectrum of the signal in the frequency domain is calculated and averaged over multiple time windows.

5. The data quality control method of a wind laser radar according to claim 4 is characterized in that: Choose an appropriate window size N and standard deviation of the Gaussian function ; For each data point, calculate the Gaussian weighted average of the N points around it. For example, when the window size is 5 and the standard deviation is 1, calculate The Gaussian weighted average of is: And use the above calculation results to replace the original power spectrum values.

6. The data quality control method of a wind laser radar according to claim 5, characterized in that: The prediction model adopts a first-order prediction model, and a set of power spectrum data points is , i=1,2,…,n, the target model is a first-order polynomial , solve the target parameters a and b, and define the sum of squared differences as S: To minimize S, find the partial derivatives of a and b and set them equal to zero, that is: Rearranging the two equations, we get: Solve the linear equations to obtain the target parameters a and b, then the polynomial This is the noise power model.

7. The data quality control method of a wind laser radar according to claim 6, characterized in that: When modeling the filtered power spectrum data, a low-order polynomial is used as the prediction model, which is suitable for describing the basic trend of the data. After solving the optimal parameters of the polynomial model, a fitting curve is generated and compared with the original filtered data. The fitting effect of the model is evaluated through residual analysis.

8. The data quality control method of a wind laser radar according to claim 7, characterized in that: After filtering, the power spectrum data before and after filtering are compared to verify the noise suppression effect, and the performance of the filter is evaluated by calculating the signal-to-noise ratio index to ensure that it effectively removes unwanted noise components without significantly weakening the useful signal.

9. The data quality control method of a wind laser radar according to claim 8, characterized in that: Format the calculated wind speed and direction information into an easy-to-use report form for end users, including generating images, time series data or statistical analysis reports, and ensure that the output results are fully validated and accompanied by necessary metadata to facilitate subsequent analysis and application.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement a data quality control method for a wind laser radar according to any one of claims 1 to 9.

Images