Real-time processing method for denoising and time lag correction of marine dissolved gas sensor data

Abstract

The present application belongs to the field of marine environment monitoring and signal processing technology, and relates to a real-time processing method for data denoising and time lag correction of a marine dissolved gas sensor, which comprises: identifying a steady-state interval of historical observation data of the sensor, fitting a steady-state jump with a Sigmoid function to generate a virtual true value of gas concentration; constructing a neural network model with observation data as input and a preliminary correction value of gas concentration as output, training the model with historical observation data as input and the virtual true value of gas concentration as a training target; using a sliding window mechanism to intercept observation data segments of the current and historical time of the sensor, inputting the data segments into the model to output a preliminary correction value of gas concentration at the current time; and using a logic state machine for post-processing to obtain a final correction value of gas concentration at the current time. The present application can provide the sensor with real-time compensation functions that take into account high dynamic response and zero steady-state drift, thereby realizing in-situ acquisition and real-time return of high-precision observation data.

Description

Technical Field

[0001] This invention belongs to the field of marine environmental monitoring and signal processing technology, specifically relating to a real-time processing method for denoising and time delay correction of marine dissolved gas sensor data. Background Technology

[0002] Existing marine dissolved gas sensors, such as membrane-based dissolved oxygen or carbon dioxide sensors, primarily rely on gas molecules permeating through a breathable membrane into an internal electrolyte for electrochemical or optical detection, thereby obtaining the concentration of the gas to be measured. Limited by the physical diffusion rate of the breathable membrane, these sensors generally exhibit severe physical response lag when facing environments with drastic concentration changes, such as deep-sea strata. This causes what should be a steep concentration change (step-like) to be measured as a slowly changing curve (bent-like), i.e., a severe "tailing" of the measurement curve, resulting in observational data that cannot accurately reflect the environmental gradient.

[0003] Currently, the mainstream hysteresis correction methods applied to marine dissolved gas sensors are generally of the following two types, each with its own shortcomings:

[0004] 1) Mathematical inversion method based on deconvolution; this method is extremely sensitive to sensor noise and is prone to amplifying high-frequency electronic noise during the calibration process, thus reducing the signal-to-noise ratio;

[0005] 2) Data-driven methods based on traditional recurrent neural networks (such as unidirectional LSTM); although this method can nonlinearly fit lag features, in the steady-state range of constant environmental concentration, the model is prone to non-physical random numerical drift (also known as illusion), which leads to a decrease in the scientific credibility of the data.

[0006] In view of this, there is an urgent need to develop a real-time processing method for marine dissolved gas sensors that can both compensate for dynamic hysteresis and ensure zero steady-state drift. Summary of the Invention

[0007] To address the shortcomings of related technologies, this invention provides a real-time processing method for denoising and time-delay correction of marine dissolved gas sensor data. The aim is to provide the sensor with a real-time compensation function that balances high dynamic response and zero steady-state drift, thereby enabling in-situ acquisition and real-time transmission of high-precision observation data.

[0008] This invention provides a real-time processing method for denoising and time-delay correction of marine dissolved gas sensor data, comprising the following steps:

[0009] S1. Obtain historical observation data from the sensor, including the raw values ​​of water temperature, water pressure, and gas concentration observed simultaneously; identify the steady-state intervals of the historical observation data from the sensor, and use the Sigmoid function to fit steady-state transitions to generate virtual true value labels for gas concentration.

[0010] S2. Construct a neural network model that takes the observed data as input parameters and the preliminary correction value of the gas concentration as the output result; input historical observation data into the neural network model and train the neural network model with the virtual true value label of the gas concentration as the training target;

[0011] S3. A sliding window mechanism is used to extract observation data segments from the sensor at current and historical moments, and these segments are input into the trained neural network model to output the current... Preliminary correction value of gas concentration at time ;

[0012] S4. Utilize a logic state machine to... Post-processing includes calculating the uncertainty of the neural network model, the instantaneous slope of the sensor's original output signal, and the current prediction deviation in real time within a pre-defined first-in-first-out sliding buffer, in order to determine and update the binary state variables of the logic state machine. , ,in, Indicates a locked state. Indicates the following state; calculate the final correction value of the gas concentration at the current moment according to equation (1). ,in, To lock the baseline value;

[0013] (1).

[0014] In some embodiments, step S1 specifically includes the following steps:

[0015] S11. Using Henry's law as a physical constraint, and combined with synchronously observed water temperature and water pressure, multi-dimensional physical parameter compensation is performed on the original value of gas concentration to obtain the gas concentration value after physical compensation.

[0016] S12. Based on the prior physical knowledge of the layered distribution of marine environmental parameters in the vertical profile and the assumption of total variation minimization, the steady-state intervals and transition points between adjacent steady states are identified. A local mean smoothing strategy based on a sliding window is adopted, and the transition process of the mean of adjacent steady states is fitted using the Sigmoid function to construct an ideal hysteresis-free step curve, thereby obtaining a virtual true value label for gas concentration. .

[0017] In some embodiments, in step S2, the neural network model includes a BiLSTM layer, an attention layer, and a fully connected layer; the training of the neural network model includes the following steps:

[0018] S21. Construct a time series matrix using historical observation data. This matrix includes the original values ​​of water temperature, water pressure, and gas concentration. Normalize the data in the time series matrix.

[0019] S22. The normalized data is input into a BiLSTM layer for temporal feature extraction, and the context feature vector is dynamically calculated through an attention layer. Then, the preliminary correction value of the gas concentration is predicted through a fully connected layer. ;

[0020] S23. Using Huber Loss as the loss function, calculate... and The error between the two is minimized to train the neural network model.

[0021] In some embodiments, during step S22, when the BiLSTM layer performs temporal feature extraction, it calculates the time step for each time step. The forward and backward hidden states are concatenated to obtain a hidden state vector that fuses bidirectional information. When the attention layer dynamically calculates the context feature vector, it calculates the first feature vector through a learnable non-linear transformation layer. Energy fraction of time step Using the Softmax function to Convert to a probability distribution to obtain the first Attention weights at time steps By analyzing the hidden state vectors at all time steps Weighted summation to obtain context feature vectors The sample is then fed into a fully connected layer for concentration regression prediction, yielding a preliminary corrected value for the predicted gas concentration. .

[0022] In some embodiments, in step S3, a sliding window mechanism is used for streaming processing. The system continuously extracts observation data segments from the sensor at preset time steps, both current and historical, and constructs a sliding window matrix using these segments. This matrix includes the raw values ​​of water temperature, water pressure, and gas concentration. The data in the sliding window matrix is ​​then normalized, and the normalized data is directly input into a trained neural network model to output a preliminary corrected value for the gas concentration at the current moment. .

[0023] In some embodiments, in step S4, the sliding buffer is used to store the current and historical gas concentration preliminary correction value sequence and the gas concentration original value sequence of the sensor; the uncertainty of the neural network model is obtained by calculating the standard deviation of the gas concentration preliminary correction values ​​in the sliding buffer. The instantaneous slope of the sensor's original output signal is obtained by calculating the first-order difference of the original gas concentration within the sliding buffer. By calculating the output value of the neural network model With locked reference value The absolute difference between them yields the current prediction bias. .

[0024] In some embodiments, in step S4, the binary state variables of the logic state machine are determined and updated according to equation (19). ,in, The preset locking variance threshold, The preset unlock slope threshold, This is a preset threshold for the unlocking range;

[0025] (19).

[0026] In some embodiments, step S5 is further included, which is the final correction value of the gas concentration at the current moment. The data is input into a Kalman filter for real-time processing. A parameter configuration strategy dominated by observation confidence is adopted, and the recursive nature of the Kalman filter is used to smooth the tiny steps during the transition between the locked and follower states, thereby obtaining the final, finely calibrated value of the gas concentration at the current moment. .

[0027] Based on the above technical solutions, the real-time processing method for denoising and time-delay correction of marine dissolved gas sensor data in this embodiment of the invention solves the problem of lack of true training labels in deep-sea in-situ observations by adopting a virtual true label generation strategy based on physical constraints and total variation minimization. This provides training targets for the training of neural network models, thereby achieving low-cost adaptive training of neural network models without the need for expensive calibration equipment. By cascading a logic state machine at the back end of the neural network model, numerical drift is completely eliminated logically, achieving zero steady-state error. This eliminates non-physical random jitter generated by the model in the steady-state range and ensures the model's high sensitivity and fast response to dynamic changes, obtaining the final corrected value of the gas concentration at the current moment. By cascading a Kalman filter at the output of the logic state machine, the small step of the measurement curve at the moment of switching between the locked and follow states is smoothed, obtaining a final trajectory that balances high dynamics and high smoothness, further improving the accuracy of in-situ acquisition of observation data. Therefore, this invention can provide sensors with a real-time compensation function that balances high dynamic response and zero steady-state drift, thereby achieving in-situ acquisition and real-time transmission of high-precision observation data. Attached Figure Description

[0028] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0029] Figure 1 This is an architecture diagram of the real-time processing method for denoising and time delay correction of marine dissolved gas sensor data according to the present invention;

[0030] Figure 2 This is a schematic diagram illustrating the generation of virtual truth labels based on total variation minimization in this invention;

[0031] Figure 3 This is a flowchart illustrating the logic state machine in this invention.

[0032] Figure 4 This is a comparison chart showing the effects of processing raw observation data from a marine dissolved gas sensor using the methods of this invention and conventional methods. Detailed Implementation

[0033] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0034] In the description of this invention, it should be understood that the terms "center", "lateral", "longitudinal", "upper", "lower", "top", "bottom", "inner", "outer", "left", "right", "front", "rear", "vertical", "horizontal", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0035] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0036] refer to Figure 1As shown, this invention provides a real-time processing method for denoising and time-delay correction of marine dissolved gas sensor data. This method is used to process raw observation data from marine dissolved gas sensors performing marine environmental monitoring tasks, thereby achieving in-situ acquisition of high-precision observation data. It is understood that marine dissolved gas sensors are typically mounted on mobile marine observation platforms, which are equipped with marine dissolved gas sensor data processing systems to perform the real-time denoising and time-delay correction method for the marine dissolved gas sensor data.

[0037] The real-time processing method for denoising and time delay correction of marine dissolved gas sensor data includes the following steps S1 to S4.

[0038] Step S1: Obtain historical observation data from the sensor. This data includes the raw values ​​of water temperature, water pressure, and gas concentration observed simultaneously. It can be understood that historical observation data is a dataset containing observation data from different time points. The steady-state intervals of the sensor's historical observation data are identified, and the Sigmoid function is used to fit steady-state transitions to generate virtual true value labels for gas concentration.

[0039] For further explanation, please refer to Figure 2 As shown, step S1 specifically includes the following steps S11 to S12.

[0040] Step S11: Using Henry's Law (a known physical law) as a physical constraint, and combined with the synchronously observed water temperature and pressure, perform multi-dimensional physical parameter compensation on the original value of gas concentration to obtain the physically compensated gas concentration value, so as to eliminate the basic influence of environmental factors on the sensor membrane permeability and gas partial pressure; that is, obtain the physically compensated value according to equation (2). Gas concentration value at time ,in, , , They are respectively The original values ​​of water temperature, water pressure, and gas concentration at any given time. The standard atmospheric pressure constant is taken as 101325 Pa. The standard temperature constant is taken as 25℃. The temperature sensitivity coefficient of the sensor (calibrated value, such as 0.004) is used to eliminate non-hysteresis environmental interference errors caused by deep-sea pressure changes and temperature fluctuations, ensuring that the data processed in step S1 reflects pure gas concentration changes rather than environmental noise, thus providing a clean physical reference for subsequent hysteresis correction.

[0041] (2).

[0042] Step S12: After obtaining the physically compensated gas concentration value sequence using Step S11, based on the prior physical knowledge that marine environmental parameters are distributed in a stratified manner on the vertical profile (i.e., the ideal real gas concentration change should exhibit "step-like" or "piecewise smooth" characteristics rather than the "tailing" shape in the original measurement) and the assumption of Total Variation Minimization, the steady-state interval and the transition point between adjacent steady states are identified; then, using the local mean smoothing strategy based on the sliding window shown in Equation (3), the concentration benchmark values ​​of the steady-state region before the transition and the steady-state region after the transition are calculated respectively. ,in, The preset width of the sliding window determines the smoothing force. The relative index offset within the sliding window is an integer variable whose value ranges from the leftmost edge of the window ( (i.e., past time) to the far right of the window ( That is, a future moment; It is a rounding function; After physical compensation The gas concentration value at a given time; this formula achieves centering smoothing and noise reduction by taking half the width forward and half backward; then, within the interval where a concentration jump is detected, the jump process of adjacent steady-state mean values ​​is fitted using the Sigmoid function, that is, the concentration at the jump point is obtained by fitting the jump edge using the Sigmoid function. As shown in equation (4), where, and The concentration reference values ​​for the steady-state region before and after the jump, respectively, are obtained according to the sliding window local mean smoothing strategy shown in equation (3). For the identified transition center moment, To control the step coefficient along the steepness of the jump, by adjusting... The value can simulate response curves with different steepness. A large positive number is set to approximate the ideal hysteresis-free step curve. By smoothing the vertical step signal into an S-shaped curve with extremely steep rising / falling edges, the steep step characteristics are preserved while being mathematically smooth and differentiable, achieving the effect of time lag removal. This allows the construction of an ideal hysteresis-free step curve, thereby obtaining a virtual truth label for the gas concentration over the entire time period. That is, quasi-ideal signal features constructed based on prior physical knowledge, thereby providing an ideal training target that conforms to physical continuity for subsequent neural network models. ;

[0043] (3);

[0044] (4).

[0045] Step S2: Construct a neural network model that takes the observed data as input parameters and the preliminary correction value of the gas concentration as the output result. This neural network model includes a BiLSTM layer (i.e., a bidirectional long short-term memory network layer), an attention layer, and a fully connected layer. This neural network model needs to be trained offline in advance. The historical sensor observation data from step S1 is input into the neural network model, and the virtual ground truth label of the gas concentration is used as the training target to train the neural network model.

[0046] To further explain, the training of the neural network model includes the following steps S21 to S23.

[0047] Step S21: Construct a time series matrix using historical sensor observation data. , ,in, For time step, The feature dimension includes historical observation data with noise and hysteresis features, such as the original values ​​of water temperature, water pressure and gas concentration, as shown in Equation (5);

[0048] (5).

[0049] To eliminate numerical differences in data with different dimensions and accelerate model convergence, The data in the dataset undergoes MinMax normalization, i.e., data homogenization preprocessing, as shown in Equation (6), to map each feature to... The interval, where, For the first Time step The original values ​​of the dimensional features. The value is the normalized value;

[0050] (6).

[0051] Step S22: Input the normalized data into the BiLSTM layer for temporal feature extraction, and dynamically calculate the context feature vector through the attention layer. Then, predict the preliminary correction value of the gas concentration through the fully connected layer. .

[0052] Furthermore, when the BiLSTM layer performs temporal feature extraction, it calculates the time step for each time step. The following forward hidden state and backward hidden state , expressed as equation (7), where, The data is normalized and then input into the neural network model; then, through... and By concatenating the hidden state vectors, we can obtain a hidden state vector that incorporates bidirectional information. , expressed as equation (8);

[0053] (7);

[0054] (8).

[0055] It's important to note that, unlike unidirectional LSTM which only utilizes historical information, BiLSTM contains two independent hidden layers: a forward layer that processes sequences from the past to the present, capturing the historical cumulative effect and hysteresis of sensor responses; and a backward layer that processes reverse sequences during offline training, capturing recovery trends at future moments. Therefore, BiLSTM layers have bidirectional weights, i.e., the forward LSTM weight matrix... , , and backward LSTM weight matrix , , It can more comprehensively analyze the contextual dependencies of signals on the time axis.

[0056] Furthermore, to address the issue of traditional RNNs' insufficient ability to capture key mutation information in long sequences, this embodiment introduces an attention mechanism after the BiLSTM layer to dynamically calculate the context feature vector. When the attention layer dynamically calculates the context feature vector, it first calculates the first... according to equation (9) through a learnable nonlinear transformation layer. Energy fraction of time step ,in, The hyperbolic tangent activation function is used. The output of the BiLSTM layer The hidden state vector at each time step. Here is the weight matrix of the attention layer. This is the bias vector for the attention layer. This is a weight vector used to map high-dimensional features to scalar scores. In linear algebra, matrix transpose transforms column vectors into row vectors for dot product operations. Then, according to equation (10), the Softmax function is used to convert the energy fraction... Transform into a probability distribution to obtain the first... Attention weights at time steps , Indicates the neural network model for the first... The level of attention given to time step information, to meet As will be understood by those skilled in the art, dummy variables are introduced in equation (10). The exponent summation is used in the denominator to separate the summation index of the numerator and denominator, avoiding interference with the current time step in the numerator. Confusion; then, according to equation (11), by applying it to all time steps Hidden state vector Weighted summation to calculate context feature vector ;

[0057] (9);

[0058] (10);

[0059] (11).

[0060] Furthermore, this context feature vector Information on key mutation points in the sequence was aggregated and fed into a fully connected layer for concentration regression prediction. The preliminary corrected value of the gas concentration was predicted according to Equation (12). ;in, This is the regression weight matrix of the fully connected layer. For bias terms;

[0061] (12).

[0062] It should be noted that by introducing an attention mechanism to weight the feature sequence within the sliding window, the model can automatically focus on moments when the signal gradient changes drastically, thereby capturing the transient features of the sensor sensing layer crossing the boundary layer and significantly improving the dynamic response speed of time delay compensation.

[0063] Step S23: During the training of the neural network model, considering that the data from the marine dissolved gas sensor are often affected by abnormal spikes caused by circuit pulses or biofouling, to prevent the calibration curve from oscillating due to spike noise generated by the sensor circuit due to overfitting, the robust Huber Loss is used as the loss function to calculate the output value of the neural network model. With the virtual truth labels generated in step S1 The error between the two is minimized to train the neural network model.

[0064] Specifically, calculate according to equation (13) and Error between ,in, These are preset hyperparameters used to control sensitivity to outliers; they are used to force the neural network to learn to extract "tailed data". Mapped to "step label" Based on the inverse physical law of "", an optimization objective to minimize this error is constructed, expressed as Equation (14), to achieve adaptive correction and training, where, This refers to the set of parameters to be trained in the neural network model. Includes: the forward LSTM weight matrix in the BiLSTM layer , , and backward LSTM weight matrix , , In the attention layer , , In fully connected layers , Therefore, this is calculated using the Huber Loss function. and The greater the difference between them, the greater the penalty. During the offline training phase, the neural network model forces the model to adjust its internal weights through the backpropagation algorithm, so that the output value of the neural network model is as close as possible to the virtual true value label. This forces the model to learn the inverse physical mapping law from "tail" to "step" (i.e., straightening the tail data), thereby eliminating lag and completing model training.

[0065] (13);

[0066] (14).

[0067] The significance of using Huber Loss as the loss function is that, for small error cases (normal data, i.e.) It uses mean squared error (MSE), and the gradient decreases as the error decreases, which allows the model to converge very accurately; for large error cases (outliers, i.e.) It uses linear error (MAE), and the gradient is constant, so it will not produce huge gradients due to large errors, which will cause the model to "go astray" and improve robustness.

[0068] To further explain, after introducing the attention mechanism, the hidden state vector of BiLSTM is mapped to an energy fraction through the hyperbolic tangent activation function. Then, the attention weights are calculated using the Softmax function. The final context feature vector The weighted summation of the hidden state vectors at all time steps allows the model to automatically focus on gradient abrupt changes in concentration, while assigning lower weights to redundant information in stable regions. This significantly improves the accuracy of reconstructing the boundary of the strata and achieves deep feature reconstruction. Furthermore, considering that marine sensor data is often affected by abnormal spikes caused by circuit pulses or biological attachments during model training, this embodiment abandons the mean squared error, which is sensitive to outliers, and selects the more robust Huber Loss as the loss function. This effectively prevents the calibration curve from oscillating due to overfitting to noise.

[0069] In summary, by employing step S1, the problem of severe response lag and inability to obtain true concentration values ​​as training labels in deep-sea in-situ observations is solved. By adopting a virtual truth label generation strategy based on physical constraints and total variation minimization, the physical prior knowledge that marine environmental parameters are typically distributed in a layered manner is utilized to identify the steady-state intervals of historical sensor observation data and the transition points between adjacent steady states. The transition process between the means of adjacent steady states is then fitted using the Sigmoid function to construct a mathematically ideal lag-free step curve, thereby obtaining a virtual truth label for gas concentration. Furthermore, by employing step S2, the virtual truth label is used as a training target. Through self-supervised learning, the neural network model is forced to master the physical law of "straightening" the tailed data, that is, to master how to map input parameters with lag characteristics into a lag-free ideal step signal. This eliminates the need to deploy expensive standard gas cylinders or reference devices for calibration in the deep sea. Low-cost adaptive training of the model can be achieved using historical observation data, greatly reducing the engineering implementation cost of marine observation.

[0070] Step S3: When the sensor is mounted on a mobile marine observation platform for marine environmental monitoring, a sliding window mechanism is used to extract observation data segments from the current and historical moments of the sensor, and input them into the trained neural network model. The neural network model directly loads the model parameters trained in step S2, and processes the observation data segments with noise and hysteresis features in real time. It can complete the parsing of the signal context dependency at the current moment within milliseconds, thereby outputting the current moment in real time. Preliminary correction value of gas concentration at time. .

[0071] Specifically, in step S3, a sliding window mechanism is used for streaming data processing. The sensor continuously extracts observation data segments at preset time steps, both current and historical, and uses these segments to construct a sliding window matrix. ,in, For time step, The feature dimensions include multi-dimensional features such as the original values ​​of water temperature, water pressure, and gas concentration, which are subject to noise and hysteresis; the sliding window matrix... The data in the dataset undergoes MinMax normalization, a data homogenization preprocessing method that is theoretically the same as step S21. The normalized data is then directly input into the trained neural network model to output the current... Preliminary correction value of gas concentration at time. .

[0072] Understandably, when sensors are actually used for monitoring, in the online real-time processing of the raw sensor observation data, since the neural network model has been trained to master the rule of mapping "tailed data" to "step data", there is no need for physical compensation, total variation processing or fitting operations during the training phase. The raw sensor observation data only needs to be processed by simple sliding window truncation and normalization, and directly input into the neural network model to reconstruct and obtain the preliminary corrected value of the gas concentration at the current moment with noise reduction and no time delay.

[0073] Step S4, Reference Figure 3 As shown, in order to completely eliminate the non-physical, small random fluctuations (also known as "drift") in the output of the neural network model in the steady-state region, this embodiment cascades a logic state machine (StepLocker) based on the Schmitt trigger principle in control theory at the output of the neural network model. It is a discrete-time finite state machine that maintains a binary state variable in real time. Using a logic state machine Real-time post-processing is performed, including: maintaining a preset length for statistical analysis. A first-in-first-out (FIFO) sliding buffer is used to store the current and historical gas concentration preliminary correction values ​​sequence of the sensor. Gas concentration raw value sequence Within the sliding buffer, the uncertainty of the neural network model, the instantaneous slope of the sensor's original output signal, and the current prediction deviation are calculated in real time to determine and update the binary state variables of the logic state machine. , ,when This indicates a locked state (meaning the environment is in a steady state). This indicates a following state (meaning the environment is in a changing state); the final correction value of the gas concentration at the current moment is calculated in real time according to equation (1). And it is transmitted back to the ocean mobile observation platform in real time, among which, To lock the baseline value;

[0074] (1).

[0075] Specifically, according to equation (15), the uncertainty of the neural network model is obtained by calculating the standard deviation of the preliminary correction value of the gas concentration in the sliding buffer. It is used to quantify the standard deviation of the model output for the current "hesitation level" of the neural network model, where, The summation index ranges from 0 to... These represent moments from the present to distant historical times, covering a total of [number missing]. One data point; Stored in the sliding buffer Preliminary corrected values ​​for gas concentration at any given time; The arithmetic mean of the predicted values ​​within the window is calculated according to equation (16);

[0076] (15);

[0077] (16);

[0078] According to equation (17), the instantaneous slope of the sensor's original output signal is obtained by calculating the first-order difference of the original gas concentration value within the sliding buffer. It is used to determine whether there are potential changes in the physical environment;

[0079] (17);

[0080] According to equation (18), the output value of the neural network model is calculated. With locked reference value The absolute difference between them yields the current prediction bias. ;

[0081] (18).

[0082] Real-time acquisition of three key statistical features , , Based on this, the binary state variables of the logic state machine are determined and updated according to equation (19). ,in, The preset locking variance threshold, The preset unlock slope threshold, The preset unlocking threshold should be noted that the threshold was determined by pre-testing the sensor's response characteristics under standard laboratory conditions and combining it with the statistical distribution of background noise in the target sea area.

[0083] (19).

[0084] As can be seen from equation (19), the state update of the logic state machine follows the double threshold logic. The AND operation in its logic reflects the noise resistance of the Schmitt trigger, preventing a single noise spike from triggering a false lock.

[0085] To further explain, the status The updates follow asymmetric logic rules with hysteresis characteristics to prevent frequent state jumps; specifically, when the system is in a follower state, the uncertainty of the neural network model is continuously monitored. If detected Less than the preset locking variance threshold (can be set) This indicates that the neural network model believes the current data is trending towards stability, at which point it determines that the system has entered a steady state and sets the state... Flip to the locked state and record the current mean as the locking baseline value. Conversely, when the system is in a locked state, to prevent false locking due to a single noise spike, the system uses strict AND logic as the unlocking condition. Only when both of the following conditions are met simultaneously is a genuine abrupt change determined and the lock released: one condition is a drastic change in the original signal, i.e., the instantaneous slope of the original output signal... Exceeding the preset unlock slope threshold (can be set) Another condition is the current prediction bias. Exceeding the preset unlock threshold (can be set) Only when both physical trends and model predictions indicate a sudden change, is the state... Only then does it flip back to the follower state.

[0086] Combining equation (1), it can be seen that in the locked state ( (Under) current Final correction value of gas concentration at time 1 That is, to force the output to lock the reference value constant. At this point, the output curve appears as an absolutely horizontal straight line, achieving zero steady-state error from a mathematical perspective; in the follower state... Below, currently Final correction value of gas concentration at time 1 That is, directly outputting the output value of the neural network model. This preserves the model's high sensitivity to dynamic changes and its rapid response capability.

[0087] In summary, by employing step S3, a preliminary correction value for the current gas concentration is obtained in real time. By employing step S4, non-physical random fluctuations generated by the neural network model in the steady-state region are eliminated, achieving zero steady-state error. This allows for the real-time acquisition of the final corrected value of the gas concentration at the current moment with high observational accuracy. This enables the in-situ acquisition and real-time transmission of high-precision observation data.

[0088] Furthermore, although the StepLocker mechanism effectively eliminates steady-state drift, it still experiences instantaneous (millisecond-level) switching between the locked and follower states (i.e., (The moment when it flips from 0 to 1 or from 1 to 0), step S4 outputs the curve. Microscopically, tiny steps may occur (i.e., derivative discontinuities, manifested as sharp bends). To obtain a smooth trajectory that conforms to physical continuity, in some embodiments, the real-time processing method for denoising and time-delay correction of ocean dissolved gas sensor data further includes step S5, which cascades a real-time Kalman filter (RTKF) at the output of the logic state machine to convert the final corrected gas concentration value output in step S4 at the current moment into a fixed value. The observed values ​​are input into a Kalman filter for real-time processing. An observation-trusted parameter configuration strategy is adopted, and the recursive nature of the Kalman filter is used to smooth the tiny steps during the transition between the locked and follower states, thereby obtaining the final, finely calibrated value of the gas concentration at the current moment. And transmit the data back to the marine mobile observation platform in real time.

[0089] Specifically, the final correction value of the gas concentration at the current moment output in step S4 is obtained using a Kalman filter. Real-time processing is performed, including establishing the first-order kinematic state equation and observation equation, and the resulting state-space model is constructed as shown in equation (20).

[0090] (20);

[0091] The first equation in equation (20) is the state equation; where, For state vectors, Defined as , This is the final, precisely calibrated value for the gas concentration at the current moment. This represents the rate of change of the gas concentration at the current moment. This is the state transition matrix; Process noise represents the uncertainty within the system. Follows Gaussian distribution , The process noise covariance matrix is... The larger the value, the more drastic the change in gas concentration is allowed; the second equation in equation (20) is the observation equation; where, The observation matrix; To measure noise, the input observation value is represented. Credibility, obey , To measure the noise covariance matrix, The smaller the value, the greater the trust. Simply put, The magnitude of the value determines whether a drastic change in gas concentration is permissible. The magnitude of the value determines whether the input observations are trusted. It should be noted that the initial value of the Kalman filter is set to the first stable value output by the logic state machine to ensure rapid convergence and stability of the system in the early stages of real-time processing.

[0092] Furthermore, since the final corrected value of the gas concentration at the current moment is output by the logic state machine in step S4... This data has already undergone denoising, deep learning, and logic locking processing, making it highly reliable. The output values ​​are based on a logic state machine. Given the prior assumption of a high signal-to-noise ratio, this embodiment employs a parameter configuration strategy dominated by observation confidence, unlike traditional Kalman filtering which typically sets a large measurement noise covariance. Instead of the conventional approach to suppressing sensor noise, this embodiment employs a special parameter configuration strategy that is not typical and is dominated by observation confidence, namely, configuring a higher process noise covariance. and extremely low measurement noise covariance ;high The settings allow for rapid changes in gas concentration over a short period, enabling the Kalman filter to follow these abrupt step changes and further preventing tailing at abrupt transitions and signal lag caused by excessive smoothing; extremely low... Settings ( This means that the Kalman filter is forcibly configured to accept highly trusted observation inputs. This ensures that the Kalman filter can closely follow the millisecond-level response of the logic state machine when dynamic changes occur, without losing the advantage of early correction due to filtering lag.

[0093] In summary, step S5 involves cascading a Kalman filter at the output of the logic state machine and employing a parameter configuration strategy dominated by observation confidence (high... Low The Kalman filter, by maximizing the trust in the dynamic changes of the input signal (i.e., rapidly following step changes), utilizes the high dynamic characteristics of the initial correction data to accelerate filter convergence. Simultaneously, it leverages the recursive nature of the Kalman filter to smooth out minor discontinuities that may occur during state transitions, thus obtaining a final trajectory that balances high dynamics and high smoothness. The Kalman filter acts as a follower with inertial smoothing capabilities; it does not alter the overall shape of the logic state machine output (i.e., it does not cause lag due to filtering). Instead, it utilizes the integral characteristics of the Kalman filter to transform the hard shear angles generated during state transitions into physically continuous and differentiable smooth curves. This balances the stability of logic locking with the continuity of physical changes, ultimately obtaining a final, precise calibration value of the gas concentration at the current moment with higher observation accuracy. .

[0094] This invention also provides a marine dissolved gas sensor data processing system, deployed within the chip of a marine mobile observation platform, i.e., supporting edge deployment within the embedded chip of the marine mobile observation platform. This marine dissolved gas sensor data processing system employs the aforementioned real-time processing method for denoising and time-delay correction of marine dissolved gas sensor data. The system includes a data acquisition module, a model building module, a virtual truth label generation module, a model training module, a model prediction module, a logic state machine, and a data output module.

[0095] The data acquisition module acquires observation data from the sensor. The model building module constructs a neural network model using the observation data as input parameters and the preliminary correction value of the gas concentration as the output. The virtual truth label generation module performs multi-dimensional physical parameter compensation on the raw gas concentration values ​​in the sensor's historical observation data, using Henry's Law as a physical constraint; then, based on the total variation minimization assumption, it uses the Sigmoid function to fit the steady-state jump to construct an ideal hysteresis-free step curve, thereby generating a virtual truth label for the gas concentration. The model training module trains the constructed neural network model using historical observation data as input and the virtual truth labels of the gas concentration as the training target. The model prediction module inputs current and historical observation data segments from the sensor into the trained neural network model and outputs the preliminary correction value of the gas concentration at the current moment. A logic state machine cascaded at the output of the model prediction module calculates in real time the standard deviation of the neural network model output value, the instantaneous slope of the sensor's original output signal, and the prediction deviation between the neural network model output value and the locked benchmark value. Based on this, it judges and updates the binary state variables of the logic state machine, and then calculates the final correction value of the gas concentration at the current moment. The data output module is used to output the final corrected value of the gas concentration at the current moment after processing by the logic state machine, and to transmit it back to the ocean mobile observation platform in real time.

[0096] Furthermore, the ocean dissolved gas sensor data processing system also includes a Kalman filter; the Kalman filter is cascaded at the output of the logic state machine and is used to process the final calibration value of the gas concentration in real time; the Kalman filter adopts a parameter configuration strategy dominated by observation confidence and uses the recursive nature of the Kalman filter to smooth the tiny step transitions at the moment of switching between the locked state and the follower state, so as to obtain the final fine calibration value of the gas concentration at the current moment; the data output module is used to output the final fine calibration value of the gas concentration at the current moment after processing by the Kalman filter and transmit it back to the ocean mobile observation platform in real time.

[0097] In summary, by refining the key module settings of the ocean dissolved gas sensor data processing system, it is made capable of edge computing with simultaneous detection and correction, thereby enabling the in-situ acquisition and real-time transmission of high-precision observation data.

[0098] To illustrate the technical effects of this invention, the raw observation data from the marine dissolved gas sensor were processed using both the methods of this invention and conventional methods. A comparison of the processed data is shown in the figure below. Figure 4 As shown. By Figure 4 It can be seen that when using traditional deep learning correction methods for data processing, there are continuous fluctuations in the steady-state region, and the correction response lag is quite obvious, failing to achieve synchronization with the true value. However, when using the present invention for data processing, an absolute locking effect can be achieved in the steady-state region, which is basically consistent with the true value, and the correction response can achieve zero lag, achieving complete synchronization with the true value. This verifies that the present invention can achieve real-time compensation function that balances high dynamic response and zero steady-state drift, thereby realizing the in-situ acquisition of high-precision observation data from marine dissolved gas sensors.

[0099] Through the description of several embodiments of the real-time processing method for denoising and time-delay correction of marine dissolved gas sensor data of the present invention, it can be seen that the present invention has at least one or more of the following advantages:

[0100] 1) This invention solves the problem of severe response lag and inability to obtain real concentration values ​​as training labels in deep-sea in-situ observations by adopting a virtual truth label generation strategy based on physical constraints and total variation minimization, thus providing training targets for neural network models. By using a self-supervised learning approach, the invention forces the neural network model to master the rules of how to map input parameters with lag characteristics into ideal step signals without lag, thereby eliminating the need to deploy expensive standard gas cylinders or reference devices in the deep sea for calibration. Low-cost adaptive training of the model can be achieved using historical observation data, greatly reducing the engineering implementation cost of ocean observation.

[0101] 2) This invention constructs a hybrid processing architecture including deep feature reconstruction based on an attention mechanism and logic locking control based on a state machine. In the real-time processing stage, a sliding window mechanism is first used to extract observation data segments of a preset time step from the current and historical moments as input parameters, which are then input into a shared-weight neural network model for deep feature reconstruction. By introducing an attention layer after the BiLSTM layer, the attention mechanism is used to focus on gradient abrupt change points and extract lag features, thereby outputting a preliminary corrected value of the gas concentration at the current moment. Subsequently, post-processing is performed by a logic state machine based on the Schmitt trigger principle, which monitors the statistical characteristics of the model output in real time. When the uncertainty of the neural network model within the sliding window is detected to be lower than the preset locking threshold, the system is determined to have entered a steady state, and the model output update is immediately cut off and the current mean is forcibly maintained, thereby logically eliminating numerical drift and achieving zero steady-state error. This invention eliminates non-physical random jitter generated by the neural network model in the steady-state range, thereby improving the scientific credibility of the data. It also ensures millisecond-level response to sudden signals through a dual-threshold mechanism. The state machine only unlocks and switches to follow mode when the instantaneous slope of the original sensor signal and its deviation from the current prediction simultaneously exceed a preset physical threshold. This effectively prevents mis-locking caused by environmental noise while ensuring the model's high sensitivity and rapid response to dynamic changes. The invention abandons complex global deconvolution calculations, using sliding windows and logic gates to complete the operation and obtain the final corrected value of the gas concentration at the current moment in real time, achieving in-situ acquisition of high-precision observation data. The entire algorithm has low complexity and can be directly deployed in low-power ARM / STM32 chips of marine mobile observation platforms such as underwater gliders and Argo buoys, enabling simultaneous detection, correction, and real-time transmission of high-precision observation data.

[0102] 3) This invention smooths the tiny step change in the measurement curve at the moment of switching between locked state and follower state by cascading a Kalman filter at the output of the logic state machine, thereby obtaining a final trajectory that balances high dynamics and high smoothness, and then obtaining the final fine-calibrated value of the gas concentration at the current moment, further improving the accuracy of in-situ acquisition of observation data.

[0103] 4) This invention achieves real-time denoising and time-delay correction of raw observation data from ocean dissolved gas sensors by constructing virtual truth labels based on physical constraints and training a neural network model, reconstructing deep features based on an attention mechanism, performing logic locking post-processing based on a state machine, and performing Kalman filtering post-processing based on a dominant parameter configuration strategy for observation confidence. Especially when ocean mobile observation platforms traverse complex concentration gradient environments, this invention can provide sensors with real-time compensation functions that balance high dynamic response and zero steady-state drift, thereby achieving in-situ acquisition and real-time transmission of high-precision observation data. In addition, because the neural network model of this invention adopts a lightweight BiLSTM structure and the logic state machine only involves simple arithmetic comparisons, the single inference time of this method on embedded microcontrollers (such as STM32, FPGA, or ARM) is much less than the second-level sampling interval of the sensor, exhibiting excellent real-time processing performance.

[0104] It should also be noted that although the specific embodiments of the present invention are based on marine dissolved gas sensors, the algorithm logic is also universally applicable or referential to other sensors that need to deal with physical hysteresis problems. Its application scenarios can be extended to various time-series sensors with physical hysteresis characteristics, such as temperature sensors, electrochemical electrodes, and industrial process control sensors.

[0105] Finally, it should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0106] The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them; although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can still be made to the specific implementation of the present invention or equivalent substitutions can be made to some technical features without departing from the spirit of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the scope of the technical solutions claimed in the present invention.

Claims

1. A real-time processing method for denoising and time-delay correction of marine dissolved gas sensor data, characterized in that, Includes the following steps: S1. Obtain historical observation data from the sensor, including the raw values ​​of water temperature, water pressure, and gas concentration observed simultaneously; identify the steady-state interval of the historical observation data from the sensor, and use the Sigmoid function to fit the steady-state transition to generate a virtual true value label for the gas concentration. S2. Construct a neural network model that takes the observed data as input parameters and the preliminary correction value of the gas concentration as the output result; input historical observation data into the neural network model and train the neural network model with the virtual true value label of the gas concentration as the training target; S3. A sliding window mechanism is used to extract observation data segments from the sensor at current and historical moments, and these segments are input into the trained neural network model to output the current... Preliminary correction value of gas concentration at time ; S4. Utilize a logic state machine to... Post-processing includes calculating the uncertainty of the neural network model, the instantaneous slope of the sensor's original output signal, and the current prediction deviation in real time within a pre-defined first-in-first-out sliding buffer, in order to determine and update the binary state variables of the logic state machine. , ,in, Indicates a locked state. Indicates the following state; calculate the final correction value of the gas concentration at the current moment according to equation (1). ,in, To lock the baseline value; (1)。 2. The real-time processing method for denoising and time-delay correction of marine dissolved gas sensor data according to claim 1, characterized in that, Step S1 specifically includes the following steps: S11. Using Henry's law as a physical constraint, and combined with synchronously observed water temperature and water pressure, multi-dimensional physical parameter compensation is performed on the original value of gas concentration to obtain the gas concentration value after physical compensation. S12. Based on the prior physical knowledge of the layered distribution of marine environmental parameters in the vertical profile and the assumption of total variation minimization, the steady-state intervals and transition points between adjacent steady states are identified. A local mean smoothing strategy based on a sliding window is adopted, and the transition process of the mean of adjacent steady states is fitted using the Sigmoid function to construct an ideal hysteresis-free step curve, thereby obtaining a virtual true value label for gas concentration. .

3. The real-time processing method for denoising and time-delay correction of marine dissolved gas sensor data according to claim 2, characterized in that, In step S2, the neural network model includes a BiLSTM layer, an attention layer, and a fully connected layer; the training of the neural network model includes the following steps: S21. Construct a time series matrix using historical observation data. This matrix includes the original values ​​of water temperature, water pressure, and gas concentration. Normalize the data in the time series matrix. S22. The normalized data is input into a BiLSTM layer for temporal feature extraction, and the context feature vector is dynamically calculated through an attention layer. Then, the preliminary correction value of the gas concentration is predicted through a fully connected layer. ; S23. Using Huber Loss as the loss function, calculate... and The error between the two is minimized to train the neural network model.

4. The real-time processing method for denoising and time-delay correction of marine dissolved gas sensor data according to claim 3, characterized in that, In step S22, when the BiLSTM layer extracts temporal features, it calculates the time step for each time step. The forward and backward hidden states are concatenated to obtain a hidden state vector that fuses bidirectional information. ; When the attention layer dynamically computes the context feature vector, it computes the first feature vector through a learnable non-linear transformation layer. Energy fraction of time step Using the Softmax function to Convert to a probability distribution to obtain the first Attention weights at time steps By analyzing the hidden state vectors at all time steps Weighted summation to obtain context feature vectors The sample is then fed into a fully connected layer for concentration regression prediction, yielding a preliminary corrected value for the predicted gas concentration. .

5. The real-time processing method for denoising and time-delay correction of marine dissolved gas sensor data according to claim 4, characterized in that, In step S3, a sliding window mechanism is used for streaming processing. The system continuously extracts observation data segments of the current and historical moments of the sensor at preset time steps, and uses these segments to construct a sliding window matrix, which includes the original values ​​of water temperature, water pressure, and gas concentration. The data in the sliding window matrix is ​​normalized, and the normalized data is directly input into the trained neural network model to output a preliminary corrected value of the gas concentration at the current time. .

6. The real-time processing method for denoising and time-delay correction of marine dissolved gas sensor data according to claim 5, characterized in that, In step S4, the sliding buffer is used to store the current and historical gas concentration preliminary correction value sequence and the original gas concentration value sequence of the sensor; by calculating the standard deviation of the gas concentration preliminary correction values ​​in the sliding buffer, the uncertainty of the neural network model is obtained. The instantaneous slope of the sensor's original output signal is obtained by calculating the first-order difference of the original gas concentration within the sliding buffer. By calculating the output value of the neural network model With locked reference value The absolute difference between them yields the current prediction bias. .

7. The real-time processing method for denoising and time-delay correction of marine dissolved gas sensor data according to claim 6, characterized in that, In step S4, the binary state variables of the logic state machine are determined and updated according to equation (19). ,in, The preset locking variance threshold, The preset unlock slope threshold, This is a preset threshold for the unlocking range; (19)。 8. The real-time processing method for denoising and time-delay correction of marine dissolved gas sensor data according to any one of claims 1 to 7, characterized in that, It also includes step S5, which calculates the final corrected value of the gas concentration at the current moment. The data is input into a Kalman filter for real-time processing. A parameter configuration strategy dominated by observation confidence is adopted, and the recursive nature of the Kalman filter is used to smooth the tiny steps during the transition between the locked and follower states, thereby obtaining the final, finely calibrated value of the gas concentration at the current moment. .

Images