Acoustic Doppler current profiler data quality control method, device and equipment

Abstract

The embodiment of the invention discloses a data quality control method, device and equipment for an acoustic Doppler flow velocity profiler. The method comprises the following steps: representing original data of the acoustic Doppler flow velocity profiler as superposition of a real flow velocity, measurement noise and an abnormal value; describing the dynamic characteristics of the original data by using a state space model, and determining a corresponding state vector and an observation vector; performing prediction processing of the Kalman filtering according to the prediction step of the standard Kalman filtering; modifying the updating step of the standard Kalman filtering by adopting a Huber weight function and an innovative sequence self-adaptive method to form an updating step of robust self-adaptive Kalman filtering, and updating the Kalman filtering according to the updating step of the robust self-adaptive Kalman filtering; and detecting an abnormal value in the original data, replacing the abnormal value with a predicted value at a corresponding moment after robust adaptive Kalman filtering processing, and outputting the data after quality control. According to the method and the device, the abnormal value and the noise in the ADCP data are effectively controlled, and the robustness and the accuracy of data processing are remarkably improved.

Description

Technical Field

[0001] This application relates to the fields of signal processing and hydrological and water resources monitoring technology, specifically to a method, apparatus, and equipment for controlling the data quality of an acoustic Doppler current profiler. Background Technology

[0002] With the rapid development of hydrological monitoring technologies for oceans, rivers, lakes, and reservoirs, the Acoustic Doppler Current Profiler (ADCP), as a non-contact current measurement device, has been widely used in hydrological and water resources monitoring, oceanographic research, and hydraulic engineering. The ADCP measures water flow velocity at different depths by emitting sound waves and receiving the reflected echoes from suspended particles in the water, utilizing the Doppler effect to obtain complete water flow velocity profile information. However, in practical applications, the raw data acquired by ADCP is often affected by various factors, such as water turbulence, ship swaying, sound wave scattering, electronic noise, and underwater obstacles, leading to unstable data quality and consequently affecting the accuracy and reliability of subsequent analysis results.

[0003] ADCP (Advanced Hydrological Computing Capacitor) is a crucial instrument for hydrological monitoring, and various research methods have been developed for its data quality control. Traditional ADCP data quality control methods mainly rely on simple threshold screening (such as the triple threshold discrimination method based on echo intensity, correlation coefficient, and signal-to-noise ratio), moving average filtering, or median filtering. While these methods are simple to implement, they have significant limitations when processing data under complex hydrological environments. On the one hand, fixed threshold methods are difficult to adapt to changing hydrological conditions and cannot solve the data screening problem under complex hydrological environments. On the other hand, conventional filtering methods struggle to balance data smoothing with detail preservation, often over-smoothing the data and losing important flow details. Furthermore, these methods typically only consider the data characteristics of a single time point or a single depth, lacking a comprehensive utilization of the spatiotemporal correlation of ADCP data.

[0004] Kalman filtering, as a classic state estimation algorithm, has been widely used in many fields. However, standard Kalman filtering is based on the Gaussian noise assumption, is sensitive to outliers, and the covariance matrices of system noise and observation noise need to be predetermined, making it difficult to adapt to the dynamic changes in noise characteristics in the actual working environment of ADCP. Although some improved Kalman filtering algorithms, such as adaptive Kalman filtering and extended Kalman filtering, have been introduced into hydrological data processing, these methods still have insufficient robustness when processing ADCP data containing a large number of outliers.

[0005] Therefore, how to effectively identify outliers in ADCP data and improve the robustness and accuracy of data processing has become an urgent problem to be solved. Summary of the Invention

[0006] In view of the above problems, this application proposes a method, device and equipment for data quality control of acoustic Doppler current profiler, which is used to solve the following problems: existing ADCP data quality control methods are difficult to effectively identify outliers in ADCP data, and the robustness and accuracy of data processing are poor.

[0007] According to one aspect of the embodiments of this application, a method for controlling the data quality of an acoustic Doppler current profiler is provided, comprising: Acquire raw data from the acoustic Doppler current profiler and represent the raw data as a superposition of the actual flow velocity, measurement noise, and outliers; The dynamic characteristics of raw data from an acoustic Doppler current profiler are described using a state-space model, and the corresponding state vector and observation vector are determined. Following the standard Kalman filter prediction steps, Kalman filter prediction processing is performed based on the state vector and the observation vector. By employing Huber weighting functions and an innovative sequence adaptive method, the update steps of the standard Kalman filter are modified to form the update steps of the robust adaptive Kalman filter. The Kalman filter is updated according to the update steps of the robust adaptive Kalman filter. Outliers in the raw data of the acoustic Doppler current profiler are detected, and the predicted values ​​at the corresponding time are replaced with robust adaptive Kalman filters to output quality-controlled acoustic Doppler current profiler data.

[0008] Furthermore, by employing the Huber weight function and an innovative sequence adaptive method, the update steps of the standard Kalman filter are modified to form a robust adaptive Kalman filter update step, which further includes: Calculate the observation innovation sequence and dynamically adjust the observation noise covariance matrix using an innovation sequence adaptive method; The Huber weight function is used for robust processing, equivalent weights are calculated, and a diagonal weight matrix is ​​formed using the equivalent weights. Based on the observation noise covariance matrix, diagonal weight matrix, and robustness factor, the update steps of the standard Kalman filter are modified to form the update steps of the robust adaptive Kalman filter.

[0009] Furthermore, the formula for calculating the observation noise covariance matrix is ​​as follows: (9) in, Let represent the observation noise covariance matrix at time k. Let represent the observation noise covariance matrix at time k-1; Indicates the forgetting factor; Indicates the adaptive factor; This represents the innovation covariance matrix at time k. This represents the observation vector at time k. This represents the observation matrix at time k. Let T represent the prior state estimate at time k; T represents the transpose of the matrix.

[0010] Furthermore, the formula for calculating the equivalent weight is: (7) in, This represents the i-th equivalent weight; Represents the i-th standardized residual; This represents the parameters of the Huber weight function.

[0011] Furthermore, the update step of the robust adaptive Kalman filter can be expressed as follows: (8) in, This represents the Kalman gain at time k; Let represent the prior estimation error covariance at time k; Let represent the observation matrix at time k; T represents the transpose of the matrix; m represents the robustness factor, whose value is updated iteratively. Let represent the observation noise covariance matrix at time k. This represents the posterior state estimate at time k; This represents the prior state estimate at time k; Represents the diagonal weight matrix; This represents the observation vector at time k. Let represent the posterior estimation error covariance at time k; Represents the identity matrix.

[0012] Furthermore, detecting outliers in the raw data from the acoustic Doppler current profiler further includes: For the observed flow velocity at any moment in the raw data of the acoustic Doppler current profiler, the flow velocity difference, signal-to-noise ratio, and correlation coefficient at that moment are compared with the corresponding thresholds to detect whether the observed flow velocity at that moment is an outlier.

[0013] According to another aspect of the embodiments of this application, an acoustic Doppler current profiler data quality control device is provided, comprising: The data acquisition module is suitable for acquiring raw data from the acoustic Doppler current profiler, representing the raw data as a superposition of real flow velocity, measurement noise, and outliers; and using a state-space model to describe the dynamic characteristics of the raw data, determining the corresponding state vector and observation vector. The filtering module is suitable for performing Kalman filtering prediction processing based on the state vector and observation vector according to the prediction steps of the standard Kalman filtering; it modifies the update steps of the standard Kalman filtering by adopting the Huber weight function and an innovative sequence adaptive method to form a robust adaptive Kalman filtering update step; and performs Kalman filtering update processing according to the robust adaptive Kalman filtering update step. The outlier detection module is suitable for detecting outliers in the raw data of the acoustic Doppler current profiler and replacing the outliers with the predicted values ​​at the corresponding time after robust adaptive Kalman filtering, outputting the acoustic Doppler current profiler data after quality control.

[0014] According to another aspect of the embodiments of this application, a computing device is provided, including: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the acoustic Doppler current profiler data quality control method described above.

[0015] According to another aspect of the embodiments of this application, a computer storage medium is provided, wherein at least one executable instruction is stored in the storage medium, the executable instruction causing a processor to perform an operation corresponding to the above-described acoustic Doppler current profiler data quality control method.

[0016] According to another aspect of the embodiments of this application, a computer program product is provided, including at least one executable instruction that causes a processor to perform operations corresponding to the above-described acoustic Doppler current profiler data quality control method.

[0017] According to the technical solution provided in the embodiments of this application, a robust adaptive Kalman filtering algorithm is introduced into the quality control of ADCP data. Robust processing is achieved by introducing a Huber weight function, and an innovative sequence adaptive method is used to dynamically adjust the observation noise covariance matrix, thus realizing effective control of outliers and noise in ADCP data. The robust mechanism automatically identifies and reduces the weight of outliers, making the filter insensitive to outliers. This effectively avoids the problem of large estimation bias in traditional Kalman filtering when outliers are present. It can effectively identify and process outliers in ADCP data, significantly improving the robustness and accuracy of ADCP data processing, and providing a basis for hydrological monitoring and... The analysis provides a more reliable data foundation; an innovative sequence adaptive mechanism is adopted to dynamically adjust the observation noise covariance matrix, enabling the data processing process to automatically and dynamically adapt to changes in noise characteristics under different hydrological environments. It maintains good performance under various complex hydrological conditions without manual intervention, effectively filtering out noise and outliers while preserving the true characteristics of the flow profile, significantly improving the applicability and practicality of the scheme. Furthermore, by comprehensively considering multi-dimensional standards such as velocity differences, signal-to-noise ratio, and correlation coefficients to detect outliers in the raw acoustic Doppler current profiler data, the accuracy of outlier detection is improved, the false positive rate is significantly reduced, and the reliability of data quality control is effectively enhanced.

[0018] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of the embodiments of this application are described below. Attached Figure Description

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating a data quality control method for an acoustic Doppler current profiler according to an embodiment of this application is shown. Figure 2 A schematic diagram illustrating the principle of an acoustic Doppler current profiler data quality control method according to an embodiment of this application is shown. Figure 3 A schematic diagram of the velocity profile corresponding to the raw data from the acoustic Doppler velocity profiler is shown. Figure 4 A schematic diagram of the flow velocity profile after robust adaptive Kalman filtering according to this application is shown; Figure 5 A comparative schematic diagram of the flow velocity time series in the intermediate depth layer is shown; Figure 6 A robust weight distribution diagram is shown; Figure 7 This shows a comparison of the effects of acoustic Doppler current profiler data quality control; Figure 8 A structural block diagram of an acoustic Doppler current profiler data quality control device according to an embodiment of this application is shown; Figure 9 A schematic diagram of the structure of a computing device according to an embodiment of this application is shown. Detailed Implementation

[0020] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0021] Figure 1 A schematic flowchart of an acoustic Doppler current profiler data quality control method according to an embodiment of this application is shown, as follows: Figure 1 As shown, the method includes the following steps: Step S101: Obtain the raw data from the acoustic Doppler current profiler and represent the raw data from the acoustic Doppler current profiler as a superposition of the actual flow velocity, measurement noise, and outliers.

[0022] This application proposes an efficient quality control method for ADCP data based on robust adaptive Kalman filtering. By introducing robust estimation theory and combining it with an adaptive observation noise covariance matrix adjustment mechanism, it can perform efficient and intelligent quality control on the velocity profile data collected by ADCP, effectively improving the reliability and anti-interference ability of the data. At the same time, it can dynamically adapt to the changes in noise characteristics under different hydrological conditions, improve the robustness and accuracy of data processing, and provide high-quality data support for water flow monitoring and analysis.

[0023] In step S101, the raw data from the acoustic Doppler current profiler to be quality controlled is acquired, i.e., the raw ADCP data to be quality controlled is acquired. Based on the acoustic Doppler principle, ADCP measures the water flow velocity at different depths by emitting sound waves and receiving the reflected echoes from suspended particles in the water body using the Doppler effect. Considering the noise and outliers in the measurement process, the raw data from the acoustic Doppler current profiler can be represented as the superposition of the actual flow velocity, measurement noise, and outliers: (1) in, This represents the observed flow velocity in the raw data from the acoustic Doppler current profiler. Indicates the actual flow rate; This indicates measurement noise and outliers.

[0024] Step S102: Use a state-space model to describe the dynamic characteristics of the raw data from the acoustic Doppler current profiler, and determine the corresponding state vector and observation vector.

[0025] In this embodiment, a state-space model is constructed to describe the dynamic characteristics of the raw data from the acoustic Doppler current profiler, and to determine the corresponding state vector and observation vector. The state-space model includes state equations and observation equations, which are as follows: (2) in, This represents the state vector (i.e., the velocity profile) at the k-th time. This represents the state transition matrix at time k-1. This represents the state vector at time k-1. This represents the process noise at time k-1. This represents the observation vector at time k. This represents the observation matrix at time k. This represents the observation noise at time k.

[0026] Step S103: Following the standard Kalman filtering prediction steps, perform Kalman filtering prediction processing based on the state vector and observation vector.

[0027] In this embodiment, the raw acoustic Doppler current profiler data can be subjected to Kalman filtering prediction processing according to the standard Kalman filtering prediction steps. Specifically, Kalman filtering prediction processing is performed based on the state vector and observation vector determined from the raw acoustic Doppler current profiler data. The formula for the standard Kalman filtering prediction steps is as follows: (3) in, This represents the prior state estimate at time k; This represents the state transition matrix at time k-1. This represents the posterior state estimate at time k-1; Let represent the prior estimation error covariance at time k; Let represent the posterior estimation error covariance at time k-1; T represents the transpose of the matrix. Let represent the process noise covariance at time k-1.

[0028] Step S104: The update steps of the standard Kalman filter are modified by using the Huber weight function and an innovative sequence adaptive method to form the update steps of the robust adaptive Kalman filter.

[0029] Kalman filtering is a recursive "prediction-update" loop, with an update step immediately following the prediction step. In this embodiment, the update step of the standard Kalman filter is modified using the Huber weight function and an innovative sequence adaptive method to form a robust adaptive Kalman filter update step. The standard Kalman filter update step is expressed as follows: (4) in, This represents the Kalman gain at time k; Let represent the prior estimation error covariance at time k; Let represent the observation matrix at time k; T represents the transpose of the matrix; Let represent the observation noise covariance matrix at time k. This represents the posterior state estimate at time k; This represents the prior state estimate at time k; This represents the observation vector at time k. Let represent the posterior estimation error covariance at time k; Represents the identity matrix.

[0030] In step S104, the observation innovation sequence can be calculated, and the observation noise covariance matrix is ​​dynamically adjusted using an innovation sequence adaptive method. Huber weights are used for robustness processing, equivalent weights are calculated, and a diagonal weight matrix is ​​formed using these equivalent weights. Based on the observation noise covariance matrix, the diagonal weight matrix, and the robustness factor, the update steps of the standard Kalman filter are modified, thus forming the update steps of the robust adaptive Kalman filter. In other words, the modified Kalman filter update steps are called the robust adaptive Kalman filter update steps. The observation innovation sequence, also known as the innovation sequence, refers to the difference between the actual observed value and the optimal predicted value based on historical information. The observation innovation sequence is not only used for state updates but also forms the basis for filter performance diagnosis, adaptive control, and fault detection.

[0031] Specifically, to improve the robustness of Kalman filtering to outliers, this application introduces robust estimation theory and uses the Huber weight function to weight the observations: (5) in, Represents the standardized residual; This represents the Huber weight function parameter, typically ranging from 1.345 to 2.0. The formula for calculating the standardized residuals is: (6) in, Represents the i-th standardized residual; Represents the i-th observation vector; This represents the i-th component of the observation matrix; This represents the estimate of the i-th prior state; Represents the innovation covariance matrix The i-th diagonal element, , Represents the observation matrix. This represents the prior estimate error covariance at time k. This represents the observation noise covariance matrix.

[0032] Based on standardized residuals, the formula for calculating equivalent weights is: (7) in, This represents the i-th equivalent weight; Represents the i-th standardized residual; This represents the parameters of the Huber weight function.

[0033] These weights are arranged into a diagonal weight matrix. The formula for the update steps of robust adaptive Kalman filtering is as follows: (8) in, This represents the Kalman gain at time k; Let represent the prior estimation error covariance at time k; Let T denote the observation matrix at time k; T denote the transpose of the matrix; m denotes the robustness factor, with an initial value of 1. The values ​​are updated iteratively: ; Let represent the observation noise covariance matrix at time k. This represents the posterior state estimate at time k; This represents the prior state estimate at time k; Represents the diagonal weight matrix; This represents the observation vector at time k. Let represent the posterior estimation error covariance at time k; Represents the identity matrix.

[0034] To enable the filter to adapt to changes in noise characteristics, this application employs an innovative sequence adaptive method to dynamically adjust the observation noise covariance matrix. The formula for calculating the observation noise covariance matrix is ​​as follows: (9) in, Let represent the observation noise covariance matrix at time k. Let represent the observation noise covariance matrix at time k-1; This represents the forgetting factor, which typically ranges from 0.6 to 0.9. This represents the adaptive factor, typically ranging from 0.1 to 0.5; This represents the innovation covariance matrix at time k. This represents the observation vector at time k. This represents the observation matrix at time k. Let T represent the prior state estimate at time k; T represents the transpose of the matrix.

[0035] This application proposes a dynamic update strategy for robustness factor, which can automatically adjust the robustness based on data quality, thereby enhancing the adaptability and robustness of the algorithm.

[0036] Step S105: Perform Kalman filter update processing according to the update steps of robust adaptive Kalman filter.

[0037] This application utilizes a robust adaptive Kalman filtering framework and a robust estimation algorithm based on Huber weights to establish the relationship between the robust adaptive Kalman filtering framework and ADCP data quality control. It constructs a dual-optimized filtering framework that combines robust estimation and adaptive covariance adjustment. Without prior noise statistics, it can achieve ADCP data quality control based on Huber weights and an innovative sequence adaptive mechanism, thus achieving efficient processing of outliers and noise in ADCP data.

[0038] Step S106: Detect outliers in the raw data of the acoustic Doppler current profiler, and replace the outliers with the predicted values ​​at the corresponding time after robust adaptive Kalman filtering, and output the quality-controlled acoustic Doppler current profiler data.

[0039] In this embodiment, a multi-dimensional standard-based outlier detection mechanism is also designed, comprehensively considering velocity differences, signal-to-noise ratio, and correlation coefficients to detect outliers in the raw data of the acoustic Doppler current profiler. Specifically, for the observed velocity at any given moment in the raw data of the acoustic Doppler current profiler, the velocity difference, signal-to-noise ratio, and correlation coefficient corresponding to that moment are compared with corresponding thresholds to detect whether the observed velocity at that moment is an outlier. The outlier detection mechanism can be expressed as follows: (10) in, Indicates outlier; This represents the observed flow velocity in the raw data from the acoustic Doppler current profiler. Indicates the estimated flow rate; The threshold representing the difference in flow rate; Indicates the signal-to-noise ratio; The threshold representing the signal-to-noise ratio; Represents the correlation coefficient; This represents the threshold for the correlation coefficient. For detected outliers, the predicted value is used instead. (11) in, This represents the actual observed value at time k corresponding to the detected outlier; This indicates that the detected outlier corresponds to the predicted value at time k based on the previous time (i.e., the (k-1)th time).

[0040] Figure 2 A schematic diagram illustrating the principle of an acoustic Doppler current profiler data quality control method according to an embodiment of this application is shown, such as... Figure 2 As shown, the process involves inputting raw ADCP data and initializing filter parameters; constructing a state-space model to describe the dynamic characteristics of the raw acoustic Doppler current profiler data; performing Kalman filtering prediction processing on the raw acoustic Doppler current profiler data according to the standard Kalman filtering prediction steps; calculating the observation innovation sequence and dynamically adjusting the observation noise covariance matrix using an innovation sequence adaptive method; initializing filter parameters; performing robust processing using the Huber weight function, calculating equivalent weights, and forming a diagonal weight matrix using the equivalent weights; modifying the standard Kalman filtering update steps based on the observation noise covariance matrix, diagonal weight matrix, and robustness factor to form a robust adaptive Kalman filtering update step, and performing Kalman filtering update processing according to the robust adaptive Kalman filtering update step; detecting outliers; and finally outputting the quality-controlled acoustic Doppler current profiler data.

[0041] This application proposes an ADCP data quality control method based on robust adaptive Kalman filtering. By introducing robust estimation theory and an adaptive noise covariance matrix adjustment mechanism, it achieves effective identification and processing of outliers. At the same time, it dynamically adapts to changes in noise characteristics under different hydrological conditions. It can effectively filter out noise and outliers while preserving the true characteristics of the flow profile, thereby improving the robustness and accuracy of ADCP data processing and providing a more reliable data foundation for hydrological monitoring and analysis.

[0042] To verify the effectiveness of this application in ADCP data quality control, a systematic simulation experiment was designed and a quantitative evaluation was conducted.

[0043] This application constructs a simulated tidal flow field model with 20 depth layers and 100 time sampling points. The simulated flow field exhibits obvious tidal periodic variation characteristics and a typical logarithmic distribution in the vertical direction. To simulate noise and outliers in actual measurements, Gaussian noise with a standard deviation of 0.2 m / s was added to the real velocity field, and outliers, accounting for 5% of the total data points, were randomly inserted, with the magnitude of the outliers being 3 to 5 times that of the normal values. Figure 3 A schematic diagram of the velocity profile corresponding to the raw data from the acoustic Doppler velocity profiler is shown.

[0044] Figure 4 This shows a schematic diagram of the flow velocity profile after robust adaptive Kalman filtering according to this application. Figure 5 A comparative schematic diagram of the velocity time series in the intermediate depth layer is further shown. The blue line represents the original data containing noise and outliers, while the red line represents the filtered result; the amplitude is between [-1, 1] m / s, with a small number of outliers, which can be removed after filtering. By comparing the original data, it is clear that the filtered data is smooth and continuous, indicating that this application can effectively suppress outliers, effectively removing them while preserving the main variation characteristics of the velocity field.

[0045] Figure 6 A robust weight distribution plot is shown, through Figure 6 This diagram visually demonstrates the automatic weight adjustment mechanism for data points of different quality levels. Brighter colors in the diagram indicate higher weights; it's evident that at outlier locations (dark dots in the diagram), this method automatically reduces their weights, thereby minimizing the impact of outliers on state estimation. Figure 7 This image shows a comparison of the effects of acoustic Doppler current profiler data quality control. Figure 7 The comparison between the actual flow velocity reference value, the original measurement value, and the filtered result is presented in a comprehensive manner, further verifying the effectiveness of the proposed method.

[0046] For quantitative evaluation, the root mean square error (RMSE) before and after processing was calculated: the RMSE of the original data was 0.8293 m / s, and the RMSE after filtering decreased to 0.1437 m / s, an improvement rate of 82.67%. In other words, the experiment shows that in data containing 5% outliers, using the method of this application for quality control reduces the root mean square error by 82.67% compared to traditional Kalman filtering. This indicates that this application can significantly improve the quality of ADCP data, providing more accurate velocity information for hydrological analysis.

[0047] Furthermore, sensitivity analysis was performed on the parameters in this method. The results show that when the initial value of the robustness factor is... The Huber weight function parameter c is set within the range of 0.8 to 1.2, the adaptive factor γ is set within the range of 1.3 to 1.8, and the forgetting factor is set within the range of 0.2 to 0.4. The algorithm performs optimally when set within the range of 0.6 to 0.8. This provides guidance for parameter selection in practical applications.

[0048] This application comprehensively considers multiple standards such as flow velocity difference, signal-to-noise ratio, and correlation coefficient to detect outliers in the raw data of acoustic Doppler current profiler. Compared with a single standard detection method, the accuracy of outlier detection can be improved by 15% to 25%, significantly reducing the false positive rate and effectively improving the reliability of data quality control.

[0049] Simulation results fully demonstrate that the robust adaptive Kalman filtering algorithm proposed in this application has significant advantages in ADCP data quality control: on the one hand, it effectively handles outliers through a robust mechanism; on the other hand, it adapts to complex hydrological environments through adaptive noise covariance adjustment. Experimental results show that, compared with existing technologies, this application can more effectively preserve the true characteristics of the flow profile, and while preserving the true characteristics of the flow profile, it effectively filters out noise and outliers, significantly improving data quality and providing a more reliable data foundation for hydrological monitoring and analysis.

[0050] According to the acoustic Doppler current profiler data quality control method provided in this application, a robust adaptive Kalman filtering algorithm is introduced into the quality control of ADCP data. Robust processing is achieved by introducing a Huber weight function, and an innovative sequence adaptive method is used to dynamically adjust the observation noise covariance matrix, thus effectively controlling outliers and noise in the ADCP data. The robust mechanism automatically identifies and reduces the weight of outliers, making the filter insensitive to outliers. This effectively avoids the problem of large estimation bias in traditional Kalman filtering when outliers are present, and can effectively identify and process outliers in ADCP data, significantly improving the robustness and accuracy of ADCP data processing. This provides a more reliable data foundation for hydrological monitoring and analysis. An innovative sequence adaptive mechanism dynamically adjusts the observation noise covariance matrix, enabling the data processing to automatically and dynamically adapt to changes in noise characteristics under different hydrological environments. It maintains good performance under various complex hydrological conditions without manual intervention, effectively filtering out noise and outliers while preserving the true characteristics of the flow profile, significantly improving the applicability and practicality of the scheme. Furthermore, by comprehensively considering multi-dimensional standards such as velocity differences, signal-to-noise ratio, and correlation coefficients to detect outliers in the raw data from the acoustic Doppler current profiler, the accuracy of outlier detection is improved, the false positive rate is significantly reduced, and the reliability of data quality control is effectively enhanced.

[0051] Figure 8 A structural block diagram of an acoustic Doppler current profiler data quality control device according to an embodiment of this application is shown, as follows: Figure 8 As shown, the device includes: a data acquisition module 810, a filtering module 820, and an outlier detection module 830.

[0052] The data acquisition module 810 is suitable for: acquiring raw data from an acoustic Doppler current profiler, representing the raw data as a superposition of real flow velocity, measurement noise, and outliers; using a state-space model to describe the dynamic characteristics of the raw data from the acoustic Doppler current profiler, and determining the corresponding state vector and observation vector.

[0053] The filtering module 820 is suitable for: performing Kalman filtering prediction processing based on the state vector and observation vector according to the prediction steps of the standard Kalman filtering; modifying the update steps of the standard Kalman filtering by adopting the Huber weight function and the innovative sequence adaptive method to form the update steps of the robust adaptive Kalman filtering; and performing Kalman filtering update processing according to the update steps of the robust adaptive Kalman filtering.

[0054] The outlier detection module 830 is suitable for: detecting outliers in the raw data of the acoustic Doppler current profiler, replacing the outliers with the predicted values ​​at the corresponding time after robust adaptive Kalman filtering, and outputting the acoustic Doppler current profiler data after quality control.

[0055] Optionally, the filtering module 820 is further adapted to: calculate the observation innovation sequence, dynamically adjust the observation noise covariance matrix using an innovation sequence adaptive method; perform robust processing using the Huber weight function, calculate equivalent weights, and use the equivalent weights to form a diagonal weight matrix; and modify the update steps of the standard Kalman filter based on the observation noise covariance matrix, the diagonal weight matrix, and the robust factor to form the update steps of the robust adaptive Kalman filter.

[0056] Optionally, the formula for calculating the observation noise covariance matrix is: (9) in, Let represent the observation noise covariance matrix at time k. Let represent the observation noise covariance matrix at time k-1; Indicates the forgetting factor; Indicates the adaptive factor; This represents the innovation covariance matrix at time k. This represents the observation vector at time k. This represents the observation matrix at time k. Let T represent the prior state estimate at time k; T represents the transpose of the matrix.

[0057] Optionally, the formula for calculating the equivalent weight is: (7) in, This represents the i-th equivalent weight; Represents the i-th standardized residual; This represents the parameters of the Huber weight function.

[0058] Optionally, the update step of the robust adaptive Kalman filter can be expressed as follows: (8) in, This represents the Kalman gain at time k; Let represent the prior estimation error covariance at time k; Let represent the observation matrix at time k; T represents the transpose of the matrix; m represents the robustness factor, whose value is updated iteratively. Let represent the observation noise covariance matrix at time k. This represents the posterior state estimate at time k; This represents the prior state estimate at time k; Represents the diagonal weight matrix; This represents the observation vector at time k. Let represent the posterior estimation error covariance at time k; Represents the identity matrix.

[0059] Optionally, the outlier detection module 830 is further adapted to: for the observed flow velocity at any moment in the acoustic Doppler current profiler raw data, compare the flow velocity difference, signal-to-noise ratio and correlation coefficient at that moment with the corresponding thresholds respectively, and detect whether the observed flow velocity at that moment is an outlier.

[0060] The descriptions of the above modules refer to the corresponding descriptions in the method embodiments, and will not be repeated here.

[0061] According to the acoustic Doppler current profiler data quality control device provided in this application embodiment, a robust adaptive Kalman filtering algorithm is introduced into the quality control of ADCP data. Robust processing is achieved by introducing a Huber weight function, and an innovative sequence adaptive method is used to dynamically adjust the observation noise covariance matrix, thus realizing effective control of outliers and noise in ADCP data. The robust mechanism automatically identifies and reduces the weight of outliers, making the filter insensitive to outliers. This effectively avoids the problem of large estimation bias in traditional Kalman filtering when outliers are present, and can effectively identify and process outliers in ADCP data, significantly improving the robustness and accuracy of ADCP data processing. This provides a more reliable data foundation for hydrological monitoring and analysis. An innovative sequence adaptive mechanism dynamically adjusts the observation noise covariance matrix, enabling the data processing to automatically and dynamically adapt to changes in noise characteristics under different hydrological environments. It maintains good performance under various complex hydrological conditions without manual intervention, effectively filtering out noise and outliers while preserving the true characteristics of the flow profile, significantly improving the applicability and practicality of the scheme. Furthermore, by comprehensively considering multi-dimensional standards such as velocity differences, signal-to-noise ratio, and correlation coefficients to detect outliers in the raw data from the acoustic Doppler current profiler, the accuracy of outlier detection is improved, the false positive rate is significantly reduced, and the reliability of data quality control is effectively enhanced.

[0062] This application provides a non-volatile computer storage medium storing at least one executable instruction or computer program that enables a processor to perform the operation corresponding to the acoustic Doppler current profiler data quality control method in any of the above method embodiments.

[0063] This application provides a computer program product, which includes at least one executable instruction or computer program that enables a processor to perform the operation corresponding to the acoustic Doppler current profiler data quality control method in any of the above method embodiments.

[0064] Figure 9 The diagram shows a structural schematic of a computing device according to one embodiment of the present application. The specific embodiments of the present application do not limit the specific implementation of the computing device.

[0065] like Figure 9 As shown, the computing device may include: a processor 902, a communications interface 904, a memory 906, and a communications bus 908.

[0066] The processor 902, communication interface 904, and memory 906 communicate with each other via communication bus 908. Communication interface 904 is used to communicate with other network elements such as clients or other servers. Processor 902 executes program 910, specifically performing the relevant steps in the above-described embodiment of the acoustic Doppler current profiler data quality control method for computing devices.

[0067] Specifically, program 910 may include program code that includes computer operation instructions.

[0068] The processor 902 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The computing device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0069] Memory 906 is used to store program 910. Memory 906 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0070] Specifically, program 910 can be used to cause processor 902 to execute the acoustic Doppler current profiler data quality control method in any of the above method embodiments. The specific implementation of each step in program 910 can be found in the corresponding descriptions of the steps and units in the above-described acoustic Doppler current profiler data quality control embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0071] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of this application are not directed to any particular programming language. It should be understood that the contents of the embodiments of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best implementation of the embodiments of this application.

[0072] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0073] Similarly, it should be understood that, in order to simplify this disclosure and aid in understanding one or more of the various inventive aspects, in the foregoing description of exemplary embodiments of the present application, various features of the present application embodiments are sometimes grouped together into a single embodiment, figure, or description thereof. However, this approach to disclosure should not be construed as reflecting an intention that the claimed embodiments of the present application require more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the present application.

[0074] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0075] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are meant to be within the scope of the embodiments of this application and form different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination.

[0076] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of this application. The embodiments of this application can also be implemented as device or apparatus programs (e.g., computer programs and computer program products) for performing part or all of the methods described herein. Such programs implementing the embodiments of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0077] It should be noted that the above embodiments are illustrative of the embodiments of this application and not limiting of the embodiments of this application, and those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Embodiments of this application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

Claims

1. A method for controlling the data quality of an acoustic Doppler current profiler, characterized in that, include: Acquire raw data from an acoustic Doppler current profiler and represent the raw data as a superposition of actual flow velocity, measurement noise, and outliers; The dynamic characteristics of the raw data from the acoustic Doppler current profiler are described using a state-space model to determine the corresponding state vector and observation vector. Following the standard Kalman filtering prediction steps, Kalman filtering prediction processing is performed based on the state vector and the observation vector. By employing Huber weighting functions and an innovative sequence adaptive method, the update steps of the standard Kalman filter are modified to form the update steps of the robust adaptive Kalman filter. The Kalman filter is updated according to the update steps of the robust adaptive Kalman filter. Outliers in the raw data of the acoustic Doppler current profiler are detected, and the outliers are replaced with the predicted values ​​at the corresponding time after robust adaptive Kalman filtering, and the quality-controlled acoustic Doppler current profiler data is output.

2. The method according to claim 1, characterized in that, The modification of the update steps for the standard Kalman filter using the Huber weight function and an innovative sequence adaptive method to form a robust adaptive Kalman filter update step further includes: Calculate the innovative observation sequence and dynamically adjust the observation noise covariance matrix using an innovative sequence adaptive method. The Huber weight function is used for robust processing, equivalent weights are calculated, and a diagonal weight matrix is ​​formed using the equivalent weights. Based on the observed noise covariance matrix, the diagonal weight matrix, and the robustness factor, the update steps of the standard Kalman filter are modified to form the update steps of the robust adaptive Kalman filter.

3. The method according to claim 2, characterized in that, The formula for calculating the observation noise covariance matrix is ​​as follows: （9） in, Let represent the observation noise covariance matrix at time k. Let represent the observation noise covariance matrix at time k-1; Indicates the forgetting factor; Indicates the adaptive factor; This represents the innovation covariance matrix at time k. This represents the observation vector at time k. This represents the observation matrix at time k. Let T represent the prior state estimate at time k; T represents the transpose of the matrix.

4. The method according to claim 2, characterized in that, The formula for calculating the equivalent weight is: （7） in, This represents the i-th equivalent weight; Represents the i-th standardized residual; This represents the parameters of the Huber weight function.

5. The method according to any one of claims 1-4, characterized in that, The formula for the update step of the robust adaptive Kalman filter is as follows: （8） in, This represents the Kalman gain at time k; Let represent the prior estimation error covariance at time k; Let represent the observation matrix at time k; T represents the transpose of the matrix; m represents the robustness factor, whose value is updated iteratively. Let represent the observation noise covariance matrix at time k. This represents the posterior state estimate at time k; This represents the prior state estimate at time k; Represents the diagonal weight matrix; This represents the observation vector at time k. Let represent the posterior estimation error covariance at time k; Represents the identity matrix.

6. The method according to any one of claims 1-4, characterized in that, The detection of outliers in the raw data from the acoustic Doppler current profiler further includes: For the observed flow velocity at any moment in the raw data of the acoustic Doppler current profiler, the flow velocity difference, signal-to-noise ratio and correlation coefficient at that moment are compared with the corresponding thresholds to detect whether the observed flow velocity at that moment is an outlier.

7. A data quality control device for an acoustic Doppler current profiler, characterized in that, include: The data acquisition module is adapted to acquire raw data from the acoustic Doppler current profiler and represent the raw data from the acoustic Doppler current profiler as a superposition of the actual flow velocity, measurement noise, and outliers. The dynamic characteristics of the raw data from the acoustic Doppler current profiler are described using a state-space model to determine the corresponding state vector and observation vector. The filtering module is adapted to perform Kalman filtering prediction processing based on the state vector and the observation vector according to the prediction steps of standard Kalman filtering; it modifies the update steps of standard Kalman filtering by using Huber weight function and innovative sequence adaptive method to form a robust adaptive Kalman filtering update step; and performs Kalman filtering update processing according to the robust adaptive Kalman filtering update step. The outlier detection module is adapted to detect outliers in the raw data of the acoustic Doppler current profiler, and replace the outliers with the predicted values ​​at the corresponding time after robust adaptive Kalman filtering, and output the acoustic Doppler current profiler data after quality control.

8. A computing device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the acoustic Doppler current profiler data quality control method as described in any one of claims 1-6.

9. A computer storage medium storing at least one executable instruction that causes a processor to perform an operation corresponding to the acoustic Doppler current profiler data quality control method as described in any one of claims 1-6.

10. A computer program product comprising at least one executable instruction that causes a processor to perform an operation corresponding to the acoustic Doppler current profiler data quality control method as described in any one of claims 1-6.

Images