Intelligent water station calibration cup liquid level control method and system

Abstract

The application provides a smart water station calibration cup liquid level control method and system, and relates to the technical field of automatic control. The method comprises the following steps: first, acquiring first time series data representing control action information and second time series data representing liquid level change information; after time alignment and feature extraction, inputting a hierarchical hidden Markov model, and outputting the state probability distribution of the current liquid level and the predicted time to reach the target liquid level; based on the state probability distribution and the predicted time, constructing a decision state vector, and generating a control instruction sequence through a hierarchical reinforcement learning model; finally, obtaining the current liquid level estimate value through an adaptive particle filtering algorithm, calculating the corresponding confidence score, and generating a control instruction; the application can also be extended to a group intelligence system composed of multiple smart water stations, realizing multi-station collaborative optimization. Through multi-model fusion and decision, the application realizes accurate control of the calibration cup liquid level under complex working conditions, significantly improving the automation level and reliability of water station calibration.

Description

Technical Field

[0001] This invention relates to the field of automation control technology, and more specifically, to a method and system for controlling the liquid level of a calibration cup in a smart water station. Background Technology

[0002] Calibration cup level control is a core component of the smart water station calibration process. Its accuracy, stability, and adaptability directly affect whether the sensor can complete calibration under the correct liquid level conditions, thus impacting the overall measurement accuracy of the water station. Traditional level control methods often employ logic control based on switching signals, such as directly controlling the start and stop of the inlet valve and drain pump using signals from high and low level switches. This method is simple in structure and low in cost, but its control accuracy is limited by the physical installation location of the switches, making it prone to oscillations and overshoot, and unsuitable for calibration scenarios requiring precise liquid level positioning. Another common method is based on proportional-integral-derivative continuous control, using analog feedback from the level gauge for closed-loop regulation. However, the water station environment is complex, with level signals often accompanied by severe fluctuations and noise. Furthermore, the physical model of the calibration cup changes due to water quality, temperature, and device aging. Fixed-parameter PID controllers struggle to maintain optimal performance under all operating conditions, making parameter tuning difficult and resulting in poor adaptability.

[0003] To overcome the shortcomings of the aforementioned methods, some studies in recent years have begun to introduce data-driven approaches for liquid level control. For example, some schemes use a single Hidden Markov Model to model the liquid level change process and combine it with reinforcement learning to generate control strategies. However, these methods often treat the action sequence and liquid level change as a whole, ignoring the coupling relationship between control actions and liquid level changes at different levels of abstraction. This makes it difficult for the model to accurately infer the true process state when faced with similar liquid level changes caused by combined pump and valve actions, resulting in low reliability of the state probability distribution. Furthermore, existing methods generally lack a step for quantitatively assessing uncertainties in the control process. When sensors are disturbed or actuators malfunction, the system cannot perceive the decrease in the reliability of its own state estimation and may still make control decisions based on incorrect estimates, leading to control failure or even accidents such as overflow or idling.

[0004] Therefore, how to overcome the shortcomings of existing technologies, such as poor ability to identify state aliasing, low learning efficiency of multi-stage tasks, and lack of quantitative evaluation and adaptive processing of the reliability of estimation results, and provide a smart water station calibration cup level control method that can be applied independently to single-station control, provide basic support for multi-station group intelligent collaboration, and can accurately predict, make efficient decisions and have high reliability, is an urgent technical problem to be solved. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for controlling the liquid level of a calibration cup in a smart water station.

[0006] In a first aspect, embodiments of the present invention provide a method for controlling the liquid level of a calibration cup in a smart water station, comprising:

[0007] First time-series data and second time-series data are acquired, and time alignment and feature extraction are performed on the first time-series data and the second time-series data to obtain a multi-dimensional feature sequence. The first time-series data represents control action information, and the second time-series data represents liquid level change information.

[0008] The multidimensional feature sequence is input into a hierarchical hidden Markov model that includes at least two levels to output the current liquid level state probability distribution and the predicted time to reach the target liquid level.

[0009] A decision state vector is constructed based on the state probability distribution and prediction time, and a control command sequence is generated through a preset hierarchical reinforcement learning model.

[0010] Based on the control command sequence, the current liquid level estimate is calculated using an adaptive particle filter algorithm. The confidence score corresponding to the liquid level estimate is calculated based on the dispersion of the particle set. The confidence score is then compared with a preset confidence threshold to generate control commands.

[0011] Furthermore, acquiring the first time-series data and the second time-series data includes:

[0012] The water quality parameter type and calibration procedure type are determined based on the current task. The water quality parameter type includes one of pH value, conductivity, turbidity, and dissolved oxygen. The calibration procedure type includes one of blind sample measurement procedure, standard solution calibration procedure, and air calibration procedure.

[0013] The first timing data includes valve and pump switching signals, and the second timing data includes liquid level dry contact switching signals and current analog signals.

[0014] The first time series data and the second time series data are acquired using the same time reference.

[0015] Furthermore, the step of performing time alignment and feature extraction on the first time-series data and the second time-series data includes:

[0016] The second timing data is preprocessed to debounce. When the transition edge of the liquid level dry contact switch signal is detected, a current waveform segment is extracted based on a sliding window of a preset time length.

[0017] Extract the time-domain and frequency-domain features of the current waveform segment as the first feature vector;

[0018] The first feature vector is input into a pre-trained lightweight gradient booster classification model, which outputs the discrimination result of the current current waveform segment; the lightweight gradient booster classification model determines whether the current waveform belongs to the effective load or interference / no load category.

[0019] Based on the discrimination result, the switch signal determined to be valid load is taken as a valid event. Based on the dynamic time warping algorithm, the first time series data and the second time series data corresponding to the valid event are time-aligned.

[0020] Furthermore, the step of inputting the multidimensional feature sequence into a hierarchical hidden Markov model comprising at least two levels includes:

[0021] The first and second time series data, which are marked as valid events after time alignment, are used as inputs to a hierarchical hidden Markov model, and two levels are constructed, including a first level representing the action sequence and a second level representing the liquid level change process.

[0022] The first level of hidden state set includes different combinations of working states. Each hidden state corresponds to a set of pump valve open or closed states. The transition between hidden states is driven by control commands.

[0023] The second level of hidden state set includes at least three states: liquid level rising, holding, and falling.

[0024] The input data is decoded, and the posterior probability of belonging to each second-level state at each time step is recursively calculated using a forward-backward algorithm. The probability distribution of the liquid level state at the current time step is then output.

[0025] Based on the second-level state obtained from decoding and its average duration, combined with the liquid level change rate per unit time under the corresponding state, the predicted time from the current liquid level to the target liquid level is calculated using the state duration model.

[0026] Furthermore, the hierarchical hidden Markov model also includes parameter acquisition:

[0027] The set of hidden states at the first level is encoded to obtain the first level of hidden states, where each hidden state is represented by a 4-bit binary number to indicate the opening or closing of the inlet valve and the drain pump, respectively.

[0028] The Baum-Welch algorithm is used to iteratively train historical data, calculate the probability of the observed sequence appearing under the current parameters, and adjust the transition probability values ​​between the states of the second level based on the probability until convergence to obtain the state transition probability matrix of the second level.

[0029] The duration of each second-level state in the historical data is statistically analyzed, and the gamma distribution shape parameters and scale parameters corresponding to each state are obtained by fitting the maximum likelihood estimation method.

[0030] Calculate the mean and variance of the second-level liquid level observations, and use them as the Gaussian distribution parameters corresponding to each second-level hidden state.

[0031] Furthermore, the generation of the control instruction sequence through the preset hierarchical reinforcement learning model includes:

[0032] The state probability distribution, prediction time, and current calibration process type are concatenated to construct a decision state vector;

[0033] The hierarchical reinforcement learning model includes a high-level policy network and a low-level execution network. The decision state vector is input into the high-level policy network, and a deep Q-network structure is used to output the Q value of each sub-task. The sub-task with the largest Q value is selected as the target sub-task of the current stage.

[0034] Based on the target subtask, activate the corresponding low-level execution network and select the atomic action with the largest Q value as the action to be executed at the current moment;

[0035] Arrange all actions to be executed in sequence to generate a control instruction sequence.

[0036] Furthermore, the hierarchical reinforcement learning model includes a high-level policy network and a low-level execution network, comprising:

[0037] Both the high-level policy network and the low-level execution network adopt a dual-network structure, including a current Q network and a target Q network. The current Q network selects actions, the target Q network calculates the temporal difference target value, and copies the parameters of the current Q network to the target Q network at preset fixed intervals.

[0038] The reward function of the lower-level execution network is obtained based on the control objective of the current subtask, and the reward value is positively correlated with the degree of completion of the current subtask.

[0039] After the current calibration process is completed, the liquid level error, total time, and stabilization time of multi-parameter measurements are obtained, and the weighted sum is taken as the negative value, which is used as the reward value of the high-level policy network.

[0040] Furthermore, the calculation of the current liquid level estimate using the adaptive particle filter algorithm includes:

[0041] Using the current liquid level state probability distribution as prior information, initialize the particle set. Each particle contains three sets of state variables: liquid level height, liquid level change rate, and disturbance coefficient.

[0042] The particle state transition equation is constructed based on the control command sequence, and the particle set is predicted and iterated to obtain the predicted particle set.

[0043] Using the analog current signal and the liquid level dry contact switch signal in the second time series data as observation values, an observation equation is constructed to calculate the deviation between the predicted value and the observed value for each particle.

[0044] The likelihood probability of each particle is calculated based on the aforementioned deviation. The likelihood probability is used as the update weight of each particle, and the update weight of all particles is normalized.

[0045] Calculate the reciprocal of the normalized weights sum of squares of the current particle set to obtain the effective particle count. If the effective particle count is less than a preset threshold, resample the particle set to obtain the filtered particle set.

[0046] The filtered particle set is weighted and summed to obtain the current liquid level estimate.

[0047] The variance and entropy of the liquid level state of the filtered particle set are calculated, the variance and entropy are mapped to a confidence score, and the confidence score is compared with a preset confidence threshold to generate a control command.

[0048] Further, comparing the confidence score with a preset confidence threshold includes:

[0049] If the confidence score is greater than or equal to the preset confidence threshold, the current liquid level estimate is used as the control basis, and valve pump control commands consistent with the control command sequence are output.

[0050] If the confidence score is less than the preset confidence threshold, the particle set is reinitialized.

[0051] Secondly, embodiments of the present invention also provide a smart water station calibration cup liquid level control system, including: a data acquisition module, a prediction module, a decision-making module, and a control module;

[0052] Data acquisition module: used to acquire first time series data and second time series data, and perform time alignment and feature extraction on the first time series data and the second time series data to obtain a multi-dimensional feature sequence. The first time series data represents control action information, and the second time series data represents liquid level change information.

[0053] Prediction module: used to input the multidimensional feature sequence into a hierarchical hidden Markov model including at least two levels, so as to output the state probability distribution of the current liquid level and the predicted time to reach the target liquid level;

[0054] Decision module: used to construct a decision state vector based on the state probability distribution and prediction time, and generate a sequence of control commands through a preset hierarchical reinforcement learning model;

[0055] Control module: Based on the control command sequence, it calculates the current liquid level estimate using an adaptive particle filter algorithm, calculates the confidence score corresponding to the liquid level estimate according to the dispersion of the particle set, compares the confidence score with a preset confidence threshold, and generates control commands.

[0056] Compared with existing technologies, this invention constructs a hierarchical Hidden Markov Model (HMM) with at least two levels, separating the first level representing the action sequence from the second level representing the liquid level change process. This effectively decouples the complex relationship between control actions and liquid level changes, thereby more accurately outputting the current liquid level state probability distribution and the predicted time to reach the target liquid level. A high-level policy network selects the sub-task of the current stage based on the global state, and a low-level execution network generates specific atomic actions for that sub-task. This decomposes the complex end-to-end learning task into multiple more easily learned sub-problems, effectively reducing liquid level overshoot during calibration and shortening the total time. Furthermore, an adaptive particle filter-based confidence assessment is introduced into the control loop. This not only calculates an accurate estimate of the current liquid level by fusing digital and analog signals, but also obtains a confidence score for the estimate by calculating the dispersion of the particle set. This prevents control failures due to misjudgment when facing sudden situations such as instantaneous sensor interference or actuator malfunctions, greatly improving the robustness and operational safety of liquid level control. Attached Figure Description

[0057] Figure 1 A flowchart of a smart water station calibration cup level control method provided in an embodiment of the present invention;

[0058] Figure 2 A flowchart for predicting the liquid level of a calibration cup in a smart water station is provided as an embodiment of the present invention;

[0059] Figure 3 A flowchart illustrating a liquid level estimation method based on particle filtering, provided for an embodiment of the present invention;

[0060] Figure 4 This is a schematic diagram of a smart water station calibration cup level control system provided in an embodiment of the present invention. Detailed Implementation

[0061] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0062] It should be noted that the method provided by this invention can be applied independently to the level control of the calibration cup in a single smart water station, or it can be extended to a swarm intelligence system consisting of multiple smart water stations. In the swarm intelligence scenario, each water station achieves information interaction and joint optimization through a cloud-based collaborative center; the specific implementation method will be detailed later.

[0063] The following describes an example of single-station control.

[0064] See Figure 1 This is a flowchart of a smart water station calibration cup level control method provided in this embodiment. The method includes steps S101 to S104, wherein:

[0065] S101: Acquire first time series data and second time series data, and perform time alignment and feature extraction on the first time series data and second time series data to obtain a multi-dimensional feature sequence. The first time series data represents control action information, and the second time series data represents liquid level change information.

[0066] S102: Input the multidimensional feature sequence into a hierarchical hidden Markov model including at least two levels to output the current liquid level state probability distribution and the predicted time to reach the target liquid level.

[0067] S103: Construct a decision state vector based on the state probability distribution and prediction time, and generate a control command sequence through a preset hierarchical reinforcement learning model;

[0068] S104: Based on the control command sequence, the current liquid level estimate is calculated using an adaptive particle filter algorithm, and the confidence score corresponding to the liquid level estimate is calculated according to the dispersion of the particle set. The confidence score is compared with a preset confidence threshold to generate control commands.

[0069] Regarding step S101:

[0070] The water quality parameter type and calibration procedure type are determined based on the current task. The water quality parameter type includes one of pH value, conductivity, turbidity, and dissolved oxygen. The calibration procedure type includes one of blind sample measurement procedure, standard solution calibration procedure, and air calibration procedure.

[0071] The first timing data includes valve and pump switching signals, and the second timing data includes high and low liquid level dry contact switching signals, as well as current analog signals.

[0072] The first time series data and the second time series data are acquired using the same time reference.

[0073] In practice, the first time-series data represents control action information, including valve and pump switching signals.

[0074] For example, in the standard solution calibration process for pH parameters, the first time series data involves five valves, including a cleaning valve, standard solution 1 valve, standard solution 2 valve, standard solution 3 valve, and blind sample valve, as well as three pumps, including a sample pump, a drain pump, and the on / off state changes of the stirring motor.

[0075] As an optional implementation, in the pH parameter calibration procedure, the standard solution can typically be a standard buffer solution, including low-concentration standard solution, medium-concentration standard solution and high-concentration standard solution.

[0076] For example, in pH measurement, standard solutions pH 4.00, pH 6.86, and pH 9.18 are used, corresponding to acidic, neutral, and alkaline calibration points, respectively; standard solution valve 1, standard solution valve 2, and standard solution valve 3 are used to introduce these three standard buffer solutions of different concentrations, respectively.

[0077] In practice, a two-point or three-point calibration method is usually adopted. By introducing two or three standard buffer solutions with known pH values, the response value of the electrode in each standard solution is measured, the actual working curve of the electrode is fitted, and the slope and zero point are corrected.

[0078] The air calibration procedure for dissolved oxygen parameters includes only three valves: a cleaning valve, a blind sample valve, a standard solution valve, and a switch signal for the same pump set.

[0079] Each of the aforementioned switch signals is recorded in the form of a high level or a low level, and is accompanied by a timestamp.

[0080] In practice, the dry contact switching signal corresponds to the low liquid level position and the high liquid level position. When the liquid level rises or falls to this position, the switch contact closes or opens, generating a switching signal.

[0081] As an alternative implementation, an analog current signal is acquired using an immersion level gauge or pressure sensor.

[0082] Taking the pH blind sample measurement process as an example, when a pH blind sample measurement task is initiated, the water quality parameter type is determined to be pH value, and the calibration process type is blind sample measurement process, based on the task identifier. During the process execution, both types of data are collected in real time.

[0083] The first time-series data represents control action information. For example, when the process reaches the evacuation stage, the discharge pump is started, and the time of this action is recorded as t. O The switch state of the discharge pump changes from 0 to 1, and this state change, along with a timestamp, is recorded as a data point.

[0084] When the process enters the water intake stage, the blind sample valve and the injection pump are opened simultaneously. The blind sample valve state changes from 0 to 1, and the injection pump state changes from 0 to 1. Time t1 and the corresponding state are recorded. All such control actions are arranged in chronological order to form the first time-series data sequence.

[0085] The second time-series data characterizes the liquid level change information. When the liquid level rises to the point of touching the high liquid level switch, the high liquid level dry contact switch signal changes from open (0) to closed (1), generating a rising edge.

[0086] When the liquid level drops to the point where it touches the low liquid level switch, the low liquid level dry contact switch signal changes from closed (1) to open (0), generating a falling edge.

[0087] The analog current signal comes from a continuous liquid level sensor, whose output current value is 4-20mA and has a linear relationship with the liquid level. This signal is continuously acquired and recorded at a fixed sampling rate of 10Hz, and each sampling point is accompanied by a timestamp.

[0088] The second timing data is preprocessed to debounce. When the transition edge of the liquid level dry contact switch signal is detected, a current waveform segment is extracted based on a sliding window of a preset time length.

[0089] As an optional implementation, when a level change is detected in any dry contact switch signal, a timer is started, and the debounce time window length is set to 50 milliseconds;

[0090] Within the anti-shake time window, the state of the dry contact switch signal is sampled five times at 10-millisecond intervals. If the five sampling results are consistent with the state after the transition, the transition is confirmed as a valid transition at the end of the window, and the moment is recorded.

[0091] If any sampling result bounces back, it is determined to be contact bounce, the timer is reset and the transition is abandoned, and subsequent signals are monitored.

[0092] In practice, after confirming a valid transition edge, a sliding window with a total length of 2 seconds is formed by taking the moment when the transition edge occurs as the center and capturing one second forward and one second backward.

[0093] As an optional implementation, all sampling points within the sliding window are extracted from the current analog quantity sequence of the second time-series data. The sampling frequency is set to 10Hz, and each sliding window contains 20 current sampling points to form a current waveform segment.

[0094] Extract the time-domain and frequency-domain features of the current waveform segment as the first feature vector;

[0095] In specific implementations, the time-domain features include mean, variance, peak value, and waveform rate of change.

[0096] The mean is the arithmetic mean of all sampling points within the segment, the variance is the average of the sum of squares of the differences between each sampling point and the mean; the peak value is the difference between the maximum and minimum values ​​within the segment; and the waveform change rate is the sum of the absolute values ​​of the differences between adjacent sampling points.

[0097] The frequency domain features are obtained by performing a fast Fourier transform on the waveform segment. A fast Fourier transform is performed on 20 sampling points to obtain the spectrum. The main frequency component, spectral energy concentration, and high-frequency noise energy are extracted from the spectrum.

[0098] As an optional implementation, the spectral energy concentration is obtained by summing the energy of all frequency components within a 0.5Hz bandwidth near the dominant frequency component and dividing by the total energy of the entire spectrum.

[0099] The high-frequency noise energy is the sum of the energy of all frequency components with frequencies higher than 5Hz.

[0100] The seven eigenvalues ​​mentioned above are arranged in a fixed order, such as mean, variance, peak value, waveform change rate, dominant frequency component, spectral energy concentration, and high-frequency noise energy combination, to form a seven-dimensional first eigenvector.

[0101] The first feature vector is input into a pre-trained lightweight gradient booster classification model, which outputs the discrimination result of the current current waveform segment; the lightweight gradient booster classification model determines whether the current waveform belongs to the effective load or interference / no load category.

[0102] The lightweight gradient boosting classification model uses a gradient boosting decision tree, which is composed of additive combinations. Each tree fits the negative gradient direction of the predicted residual of the previous tree.

[0103] The first feature vector passes through each decision tree in sequence. Each tree reaches a leaf node along the branch according to its feature value and outputs a predicted value.

[0104] The predicted values ​​of all trees are multiplied by the learning rate and summed to obtain the total score. This score is then converted into a probability value using the sigmoid function, which is the discriminant probability output by the model.

[0105] Based on the discrimination result, the switch signal determined to be valid load is taken as a valid event. Based on the dynamic time warping algorithm, the first time series data and the second time series data corresponding to the valid event are time-aligned.

[0106] In practical implementation, the current waveform segments and their corresponding dry contact signal transition events collected during the historical operation of the smart water station are used as historical datasets. Data collection scenarios include:

[0107] Effective load scenarios: During normal calibration processes, such as water inlet of pH standard solution valve 1, water inlet of pH standard solution valve 2, water inlet of blind sample, and water drainage, when the liquid level actually rises or falls, the liquid level switch triggers the transition edge, and the corresponding current waveform segment is captured.

[0108] Interference / no-load scenarios: These include situations such as pump running without liquid, bubble disturbance such as liquid level fluctuation causing sensor output fluctuation, and instantaneous sensor interference such as electromagnetic interference, poor contact, and pipeline blockage, which cause the liquid level to remain unchanged but the pump to continue running. The corresponding current waveform segments are extracted.

[0109] As an optional implementation, each acquired current waveform segment is categorized by experts based on actual operating conditions. The categorization rules include:

[0110] If the segment corresponds to a real process of liquid level rise or fall, that is, the liquid level does change continuously after the pump valve is activated, and eventually triggers or disconnects the liquid level switch, then it is marked as a valid load.

[0111] If the segment corresponds to non-realistic liquid level changes such as pump idling, bubble disturbance, sensor interference, or pipeline blockage, it is marked as interference / idle.

[0112] After labeling, each waveform segment corresponds to a binary label, with an effective load of 1 and an interference / no load of 0.

[0113] For each labeled current waveform segment, the corresponding first feature vector is extracted and combined to form a training sample.

[0114] In practice, the historical dataset is randomly divided into a training set and a validation set, such as 80% for the training set and 20% for the validation set.

[0115] In specific implementation, the maximum number of leaf nodes in each tree of the lightweight gradient booster classification model is set to 31, the maximum depth of the tree is not limited, the learning rate is set to 0.1, the number of iteration rounds is set to 100 rounds, the subsampling ratio is set to 0.8, the feature sampling ratio is set to 0.8, and both the L1 regularization coefficient and the L2 regularization coefficient are set to 0.1.

[0116] Taking the process of standard solution 1 valve reaching a high level during pH standard solution calibration as an example, during one pH standard solution 1 valve inlet process, the liquid level gradually rises from a low level to a high level. When the liquid level reaches the high level switch installation position, the high level dry contact switch signal changes from open to closed, triggering a rising edge transition. After the system detects this transition, it extracts a current waveform segment to obtain 20 current sampling values: [8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7, 8.8, 8.9, 9.0, 9.1, 9.2, 9.3, 9.4, 9.3, 9.2, 9.1, 9.0, 8.9, 8.8].

[0117] In practice, the frequency domain characteristics are obtained by performing a Fast Fourier Transform on 20 sampling points. The resulting spectrum has the following amplitude values ​​for each frequency component: 0.5 Hz component amplitude is 0.05; 1.0 Hz component amplitude is 0.23; 1.5 Hz component amplitude is 0.12; 2.0 Hz component amplitude is 0.04; 2.5 Hz component amplitude is 0.02; and the amplitude values ​​for components at 3.0 Hz and above are all less than 0.01.

[0118] The main frequency component is 1.0Hz, and its amplitude is 0.23.

[0119] The sum of squares of the amplitudes of the main frequency 1.0Hz and its adjacent 0.5Hz and 1.5Hz components is calculated to be 0.0698, and the sum of squares of the amplitudes of all components in the entire spectrum is 0.072. Therefore, the spectral energy concentration is 0.97.

[0120] Since the sampling frequency is 10Hz and the Nyquist frequency is 5Hz, there are no frequency components higher than 5Hz, so the high-frequency noise energy is 0.

[0121] Extracting time-domain and frequency-domain features yields the first feature vector as [8.85, 0.14, 1.3, 2.6, 0.23, 0.97, 0].

[0122] The first feature vector is input into the pre-trained lightweight gradient booster classification model for forward computation.

[0123] Taking the first tree as an example, the splitting feature set at the root node is the first feature, and the splitting threshold is 8.0. If the mean of the first feature vector (8.85) is greater than 8.0, then the tree enters the right branch.

[0124] The next node in the right branch uses the third feature as the splitting feature, with a splitting threshold of 1.0. If the peak value of 1.3 in the first feature vector is greater than 1.0, the node enters the right branch and reaches a leaf node with an output value of 0.15. The output of the first tree is recorded as 0.15.

[0125] The second tree splits at the root node using the fifth feature as the splitting feature, with a splitting threshold of 0.2. The dominant frequency component of the first feature vector, 0.23, is greater than 0.2, so it enters the right branch.

[0126] The next node of the right branch takes the seventh feature as the splitting feature, the splitting threshold is 0.01, and the high-frequency noise energy in the first feature vector is less than 0.01. It enters the left branch, reaches a leaf node, and the output value is -0.08. The output of the second tree is recorded as -0.08.

[0127] Multiply the output values ​​of all trees by the learning rate of 0.1 and sum them up to get the total score of 1.25.

[0128] The total score is input into the sigmoid function, and the resulting probability value is 0.777, which is less than the preset probability threshold of 0.8. Therefore, the current waveform segment is determined to belong to the interference / no-load category, and the corresponding dry contact transition edge is not treated as a valid event.

[0129] In specific implementation, if the current waveform segment feature vector captured during the water inlet process of standard liquid valve 1 is input into the model and the calculated probability value is 0.95, which is greater than 0.8, it is determined to be a valid load. The transition edge of the dry contact is marked as a valid event, and the first timing data and the second timing data are time-aligned.

[0130] As an optional implementation, taking the time of occurrence of the effective event as the center, a window of the same time length is extracted from the first time series data and the second time series data to obtain the control action coding sequence and the current analog quantity sequence, respectively.

[0131] Construct a cumulative distance matrix, calculate the minimum cumulative distance between two sequences using dynamic programming, and backtrack to obtain the optimal matching path.

[0132] The current analog quantity sequence is resampled according to the optimal matching path to obtain a new current sequence aligned with the time axis of the control action encoding sequence, so that each control action moment has a corresponding liquid level current value.

[0133] Regarding step S102:

[0134] See Figure 2 The flowchart for predicting the liquid level of a calibration cup in a smart water station, provided by an embodiment of the present invention, includes:

[0135] S201: Construct a hierarchical hidden Markov model, define the sets of hidden states of the first and second levels, and establish the conditional dependencies between the two levels.

[0136] S202: Iterative training is performed using the Baum-Welch algorithm. The forward-backward algorithm is used to calculate the posterior probability of the state and the transition probability, and the initial state probability, the state transition probability matrix and the Gaussian observation parameters are updated until convergence is obtained to obtain the optimized model parameters.

[0137] S203: The target's second-level state sequence is obtained by decoding the historical observation sequence using the Viterbi algorithm;

[0138] S204: Based on the second-level state sequence of the target, the duration samples of continuous occurrence of each state are statistically analyzed, and the shape parameters and scale parameters of each state are obtained by fitting the gamma distribution through maximum likelihood estimation.

[0139] S205: The posterior probability distribution of each second-level state at the current time is obtained by recursively calculating the real-time input observation sequence through the forward algorithm.

[0140] S206: Calculate the predicted time required to reach the target liquid level based on the posterior probability of the current state, the rate of change of liquid level, the duration of the change, and the gamma distribution parameters.

[0141] Regarding S201:

[0142] The first and second time series data, which are marked as valid events after time alignment, are used as inputs to a hierarchical hidden Markov model, and two levels are constructed, including a first level representing the action sequence and a second level representing the liquid level change process.

[0143] In practice, the two layers are connected by a conditional dependency relationship. The hidden state of the first layer affects the transition probability between the states of the second layer, and the hidden state of the second layer determines the distribution of the liquid level observation value at the current moment.

[0144] The hierarchical hidden Markov model takes the action code and current value at each time step as input and outputs the posterior probability distribution of the second-level state at each time step and prediction information based on the current state.

[0145] The first level of hidden state set includes different combinations of working states. Each hidden state corresponds to a set of pump valve open or closed states. The transition between hidden states is driven by control commands.

[0146] Taking the pH parameter calibration process as an example, the calibration cup control involves five valves and three pumps. The on / off status of all actuators is encoded into a binary number, with each bit representing the on / off status of an actuator, such as 1 indicating on and 0 indicating off.

[0147] For example, the following components are arranged in order: cleaning valve, standard solution 1 valve, standard solution 2 valve, standard solution 3 valve, blind sample valve, sample pump, drain pump, and stirring motor, resulting in an eight-bit binary number with a value range of 0 to 255. Each value corresponds to a pump and valve combination.

[0148] The second level of hidden state set includes at least three states: liquid level rising, maintaining, and falling.

[0149] In practice, the liquid level rise is determined by observing that the continuous sampling points of the analog current signal show an increasing trend and the rate of change is greater than zero.

[0150] The liquid level drop is determined by observing that the continuous sampling points of the analog current signal show a decreasing trend and the rate of change is less than zero.

[0151] The liquid level holding state is determined by observing that the current analog signal fluctuates within a small range and the rate of change is close to zero.

[0152] In practice, states can transition between each other.

[0153] For example, a state can be transitioned from rising to holding (stopping water intake) or falling (immediately draining water), and a transition from rising to rising indicates continuous rising.

[0154] In practice, the initial probability distribution of the second-level hidden state is a vector of length 3, where each element represents the probability of being in the corresponding state at the initial moment.

[0155] The state transition probability matrix is ​​a 3×3 matrix, where each element represents the probability of transitioning from the current state to the next state;

[0156] The observation probability distribution follows a Gaussian distribution, with each state corresponding to a mean and a variance.

[0157] The duration of continuous residence in each state in the duration distribution follows a gamma distribution, which is described by shape and scale parameters.

[0158] As an optional implementation, the input data is decoded, and the posterior probability of belonging to each second-level state at each time step is recursively calculated using a forward-backward algorithm, and the probability distribution of the liquid level state at the current time step is output.

[0159] By forward recursion of the observation sequence (current value at each time step), model parameters (initial state probability, state transition probability matrix), and Gaussian observation parameters of each state, the forward probability of each state at the initial time step is obtained.

[0160] For each subsequent time step, the forward probability of each state at the current time step is recursively calculated based on the forward probability of each state at the previous time step, the state transition probability matrix, and the observation probability at the current time step.

[0161] The backward recursion initializes the backward probabilities of each state at the last moment to 1. For the previous moment, the backward probabilities of each state at the current moment are calculated in reverse recursion based on the backward probabilities of each state at the current moment, the state transition probability matrix, and the observation probability at the current moment.

[0162] The posterior probability is obtained by multiplying the forward probability and the backward probability at each time step and then dividing by the sum of all states, thus obtaining the posterior probability distribution of each state at that time step.

[0163] The output yields the second-level state posterior probability distribution for each time step.

[0164] Based on the second-level state obtained from decoding and its average duration, combined with the liquid level change rate per unit time under this state, the predicted time required to reach the target liquid level from the current liquid level is calculated using the state duration model.

[0165] The set of hidden states at the first level is encoded to obtain the first level of hidden states, where each hidden state is represented by a 4-bit binary number to indicate whether the inlet valve and the drain pump are open or closed.

[0166] In practice, a set of hidden states is defined to characterize the change pattern of the liquid level in the calibration cup.

[0167] The set of hidden states is consistent with the state space in the hierarchical hidden Markov model, including: rapid liquid level drop, slow liquid level drop, stable liquid level, and rising liquid level. Each state represents a process stage with a specific liquid level change pattern.

[0168] Each hidden state is configured with an independent duration probability distribution model to represent the statistical regularity of the time length experienced by the state from start to end; the distribution adopts a Gaussian distribution form, and each state records two parameters: average duration and duration variance.

[0169] In practical implementation, a state transition probability matrix is ​​constructed, where each element represents the probability of transitioning to another state after the current state ends. This matrix shares the same set of parameters as the state transition probability matrix of the hierarchical Hidden Markov Model, ensuring the consistency of the state evolution logic.

[0170] All parameters of the state duration model were estimated offline using historical data during the model training phase. The training data consisted of complete time-series data recorded during multiple calibration cup operations, including the liquid level status label, state duration, and state transition sequence at each moment.

[0171] For example, parameter estimation can be performed using the following methods:

[0172] Duration parameter estimation: For each state, extract all continuous segments of that state from the training data, calculate the duration of each segment, and then calculate its mean and variance as the duration distribution parameter of that state.

[0173] Transition probability matrix estimation: Statistically analyze the frequency of state transitions in the training data, calculate the probability of transitioning from each state to other states, and obtain the transition probability matrix.

[0174] Liquid level change rate estimation: For each state, extract the sequence of liquid level change over time under that state, and obtain the change rate by linear regression fitting.

[0175] Regarding S202:

[0176] As an optional implementation, the Baum-Welch algorithm is used to iteratively train on historical data, with each iteration consisting of an expectation step and a maximization step.

[0177] In practice, during the expectation step, a forward-backward algorithm is executed for each historical observation sequence to calculate the posterior probability of being in each second-level state at each time step, as well as the joint probability of transitioning from one state to another state at the next time step.

[0178] In the maximization step, the model parameters are re-estimated based on the probability values ​​obtained in the expectation step.

[0179] As an optional implementation, when updating the initial state probability, the posterior probability of each state at the first time step of all sequences is averaged to obtain the new initial state probability.

[0180] When updating the state transition probability matrix, for each initial state and target state, calculate the expected number of times the state transitions from the initial state to the target state at all times, divide it by the expected number of times the state is in the initial state at all times, and the ratio of the two is the new transition probability.

[0181] When updating the Gaussian observation parameters, for each state, the weighted mean and weighted variance are calculated using the current observations at all times in that state and the corresponding posterior probabilities, and used as the new Gaussian distribution mean and variance.

[0182] After completing one expectation step and one maximization step, a new set of model parameters is obtained. The new parameters are compared with the parameters of the previous round. If the change of all parameters is less than the preset first convergence threshold, the iteration stops; otherwise, the new parameters are used as the current parameters, and the expectation step and maximization step are repeated until convergence or the preset maximum number of iterations is reached.

[0183] Regarding S203:

[0184] As an optional implementation, the Viterbi algorithm is used to decode each historical observation sequence to obtain the target's second-level state sequence.

[0185] For example, the Viterbi algorithm uses dynamic programming to recursively calculate the maximum probability path to each state starting from the first time step, and selects the state with the highest probability at the last time step, and then backtracks to obtain the hidden states of the entire sequence.

[0186] Regarding S204:

[0187] Based on the decoded state sequence, the duration of each consecutive occurrence of a second-level state is counted. Each state sequence is traversed, and whenever a state changes, the duration of the previous state is recorded. The duration segments of all ascending states are collected into a sample set; similarly, sample sets are collected for maintaining states and descending states.

[0188] Regarding S205:

[0189] The maximum likelihood estimation method is used to fit the gamma distribution to the duration sample set for each state. By calculating the statistical properties of the sample set, the shape and scale parameters of the gamma distribution corresponding to each state are obtained. These parameters will be used in the online prediction stage to describe the duration distribution of each state, thereby more accurately calculating the predicted time required to reach the target liquid level.

[0190] The duration of each second-level state in the historical data is statistically analyzed, and the gamma distribution shape parameter and scale parameter corresponding to each state are obtained by fitting the maximum likelihood estimation method.

[0191] Calculate the mean and variance of the second-level liquid level observations, and use them as the Gaussian distribution parameters corresponding to each second-level hidden state.

[0192] Regarding S206:

[0193] As an optional implementation, the current value corresponding to the high liquid level is set according to the typical current value when the high liquid level switch is triggered, and the same applies to the low liquid level; the state with the highest posterior probability at the current moment is taken as the current state.

[0194] By combining the state duration model, the remaining time of the state is calculated and compared with the time required for the liquid level difference, and the smaller one is taken as the predicted time.

[0195] Taking the pH standard solution calibration process as an example, in the low standard calibration, the following steps are performed in sequence: draining, introducing standard solution 1, reading, draining, introducing standard solution 2, reading, etc. Taking the introduction of standard solution 1 as an example, the inlet valve is open and the drain pump is closed. The first level state is "inlet valve open, drain pump closed", and the corresponding code is binary 10.

[0196] In this scenario, the liquid level rises from a low level to a high level, and the second level of the actual state is an upward movement.

[0197] Alignment data is extracted from the historical dataset to form a training set. The training set contains 200 observation sequences, each sequence having 100 time points, and each time point has an action code and a current value.

[0198] In specific implementation, the initial parameters are set as follows: the initial state probability distribution is uniform, that is, the probability of rising, maintaining, and falling is one-third each, which is 0.33, 0.34, and 0.33 respectively.

[0199] The state transition probability matrix is ​​initialized as a 3×3 matrix with rows in ascending, holding, and descending order, and columns in the same order. The diagonal is set to 0.8, and the off-diagonal is evenly divided into 0.2. The matrix is ​​as follows: ascending to ascending 0.8, ascending to holding 0.1, ascending to descending 0.1; holding to ascending 0.1, holding to holding 0.8, holding to descending 0.1; descending to ascending 0.1, descending to holding 0.1, descending to descending 0.8.

[0200] As an optional implementation, all current values ​​in the training set are clustered into three classes using the K-means clustering method, and the mean and variance of each class are calculated as the initial observation distribution for rising, holding, and falling.

[0201] For example, the clustering results are: the first cluster has a mean of 8.5 mA and a variance of 0.12; the second cluster has a mean of 9.0 mA and a variance of 0.05; and the third cluster has a mean of 7.5 mA and a variance of 0.15, which are used as the initial observation distribution parameters.

[0202] In practice, the Baum-Welch iteration is performed using 100 sequences as the training set, for a total of 50 iterations. In each iteration, the transition probability and observation parameters are updated by calculating the forward and backward probabilities of all sequences.

[0203] In the expectation step, the forward probability, backward probability, state probability, and transition probability of the sequence are calculated using the current parameters.

[0204] For example, at the first moment, the observed current is 7.6 mA. For the rising state, its forward probability is equal to the initial probability of the rising state (0.33) multiplied by the probability density value of the observation occurring in the rising state. This probability density is calculated according to a Gaussian distribution and is approximately 0.02. Multiplying the two together gives the forward probability of the rising state as 0.0066. Similarly, the forward probability of the hold state is calculated to be 0.0068, and the forward probability of the fall state is 0.0065.

[0205] Entering the second time step, the observed current value is 7.7 mA. Taking the rising state as an example, multiply the forward probability of the rising state in the previous time step (0.0066) by its transition probability of maintaining the rising state (0.8), multiply the forward probability of maintaining the state in the previous time step (0.0068) by its transition probability of transitioning to the rising state (0.1), and multiply the forward probability of the falling state in the previous time step (0.0065) by its transition probability of transitioning to the rising state (0.1). Add these three products together to get 0.00661.

[0206] Multiplying the sum of these products by the probability density of 0.03 for the current observed value of 7.7 mA in the rising state, we obtain the forward probability of the rising state at the current time as 0.0001983. Similarly, we continue to calculate the forward probabilities of the holding and falling states, and recursively calculate them time by time until we reach time 100.

[0207] The backward probability is calculated recursively from the last moment. At moment 100, the backward probability of the three states of rising, holding, and falling is set to 1.

[0208] To calculate the backward probability of the rising state at time 99, multiply the transition probability of maintaining the rising state (0.8) by the probability density of an observed value of 8.5 mA at time 100 while in the rising state (0.2), and then multiply by the backward probability of the rising state at time 100 (1). Add the transition probability of the rising state transitioning to the hold state (0.1) multiplied by the probability density of 8.5 mA in the hold state (0.01), and then multiply by the backward probability of the hold state at time 100 (1). Finally, add the transition probability of the rising state transitioning to the falling state (0.1) multiplied by the probability density of 8.5 mA in the falling state (0.001), and then multiply by the backward probability of the falling state at time 100 (1). The sum of these three terms is 0.1611, which is the backward probability of the rising state at time 99.

[0209] Calculate the backward probabilities of the state remaining in place and the state falling at time 99 in the same way, and continue to recursively advance to time 1.

[0210] After obtaining the forward and backward probabilities at each time step, for time step 1, the forward probability of the rising state is multiplied by the backward probability, the forward probability of the holding state is multiplied by the backward probability, and the forward probability of the falling state is multiplied by the backward probability to obtain three product values ​​respectively.

[0211] Add these three product values ​​together to get a sum, then divide the product value of each state by this sum to obtain the posterior probability of each state at that time. Suppose that the calculated posterior probability of the rising state at time 1 is 0.4, the holding state is 0.3, and the falling state is 0.3.

[0212] After calculating for all time points, the state probability and transition probability at each time point are obtained.

[0213] In the maximization step, the model parameters are re-estimated using the state probabilities and transition probabilities at all time points calculated by the forward-backward algorithm.

[0214] In practice, the state probabilities of all sequences in the training set at the first time step are averaged to obtain the new initial state probabilities. The probabilities of each sequence being in an ascending state at the first time step are summed and divided by 100 to obtain a new initial probability of an ascending state of 0.35; similarly, the initial probability of a maintaining state is calculated to be 0.33, and the initial probability of a descending state is calculated to be 0.32.

[0215] In this embodiment, taking the transition probability from an ascending state to an ascending state as an example, the numerator is the sum of the transition probabilities of all sequences at all times from an ascending state, and the denominator is the sum of the state probabilities of all sequences at all times in an ascending state. The new transition probability is obtained by dividing the two. Assuming the numerator is 1200 and the denominator is 1500, the new transition probability is 0.8.

[0216] For each pair of initial and target states, calculate the complete state transition probability matrix. For example, the probability of rising to hold is 0.15, rising to falling is 0.05, rising to rising is 0.8; holding to rising is 0.1, holding to holding is 0.8, holding to falling is 0.1; falling to rising is 0.1, falling to holding is 0.15, falling to falling is 0.75.

[0217] Taking the rising state as an example, the new mean is equal to the probability of being in the rising state at all times multiplied by the weighted sum of the current values ​​at the corresponding times, divided by the sum of the probabilities of being in the rising state at all times. Assuming the sum of the numerators is 127500 and the sum of the denominators is 15000, then the new mean is 8.5.

[0218] The new variance is equal to the weighted sum of the probabilities of being in an upward state at all times multiplied by the squares of the differences between the current value at that time and the new mean, divided by the sum of the probabilities of being in an upward state at all times. Let's assume the result is 0.12. In the same way, the mean and variance for the holding state are calculated to be 9.0 and 0.05, respectively, and the mean and variance for the falling state are calculated to be 7.5 and 0.15, respectively.

[0219] After completing one maximization step, a new set of model parameters is obtained. The new parameters are compared with the parameters from the previous iteration. If the change in all parameters is less than the preset first convergence threshold, the iteration stops; otherwise, the new parameters are used as the current parameters, and the expectation step and maximization step are repeated.

[0220] For example, assume that after 20 iterations, the parameter change is less than the first convergence threshold, and the model converges. The final state transition probability matrix is ​​as follows: the probability of maintaining an ascending state is 0.75, the probability of transitioning from an ascending state to a holding state is 0.15, and the probability of transitioning from an ascending state to a descending state is 0.10; the probability of transitioning from a holding state to an ascending state is 0.08, the probability of maintaining a holding state is 0.84, and the probability of transitioning from a holding state to a descending state is 0.08; the probability of transitioning from a descending state to an ascending state is 0.05, the probability of transitioning from a descending state to a holding state is 0.10, and the probability of maintaining a descending state is 0.85. The Gaussian observation parameters for each state are: mean 8.5 and variance 0.12 for the ascending state, mean 9.0 and variance 0.05 for the holding state, and mean 7.5 and variance 0.15 for the descending state.

[0221] After the model parameters converge, the Viterbi algorithm is used to decode each observation sequence in the training set to obtain the target second-level state sequence.

[0222] In practical implementation, taking the current analog signal of 8.3 mA and the current value corresponding to the target high liquid level of 9.0 mA as an example, the liquid level prediction time is calculated.

[0223] For example, in the rising state, the average current increase per 0.1 seconds in the historical data is 0.02 mA, which translates to a rise rate of 0.2 mA per second.

[0224] The time required to reach the target liquid level is calculated based on the difference between the current and target current values. Subtracting the current value of 8.3 mA from the target current value of 9.0 mA yields a current difference of 0.7 mA. Dividing this difference by the rising rate of 0.2 mA per second, the time required to reach the target liquid level using only the current rising rate is calculated to be 3.5 seconds.

[0225] Based on the state probability distribution output by the hierarchical hidden Markov model, the most likely state at the current moment is the rising state. From the state sequence decoded by the Viterbi algorithm, the rising state started at time 21, and the current time is time 50, lasting for 30 time steps. Calculated at a sampling interval of 0.1 seconds, the rising state has lasted for 3.0 seconds.

[0226] Assuming the average duration of the rising state obtained from fitting historical data is 5.5 seconds, subtracting the current duration from the average duration gives an expected remaining duration of 2.5 seconds in the current state.

[0227] Comparing the two times mentioned above, the time required to reach the target solely by the rate of ascent is 3.5 seconds, while the remaining time in the current ascent state may only be 2.5 seconds.

[0228] Since the remaining state time is less than the time required for the level difference, it is anticipated that the target level cannot be reached before the current rising state ends, and it may be necessary to wait for a state transition or enter the next control phase. The smaller of the two, 2.5 seconds, is taken as the final predicted time output.

[0229] Regarding S103:

[0230] The state probability distribution, prediction time, and current calibration process type are concatenated to construct the decision state vector.

[0231] The hierarchical reinforcement learning model includes a high-level policy network and a low-level execution network. The decision state vector is input into the high-level policy network, and a deep Q-network structure is used to output the Q value of each sub-task. The sub-task with the largest Q value is selected as the target sub-task of the current stage.

[0232] Based on the target subtask, activate the corresponding low-level execution network and select the atomic action with the largest Q value as the action to be executed at the current moment;

[0233] Arrange all actions to be executed in sequence to generate a control instruction sequence.

[0234] As an optional implementation, a hierarchical reinforcement learning model based on the Option-Critic architecture is adopted, which includes a high-level policy network and four low-level execution networks.

[0235] In its implementation, the high-level policy network adopts a deep Q-network structure. The input layer receives the decision state vector and is connected to the first fully connected layer, which contains 128 neurons and uses the ReLU activation function. The second fully connected layer contains 64 neurons and uses the ReLU activation function. The output layer contains 4 neurons, corresponding to the four sub-tasks of rapid water injection, precise fluid replenishment, liquid level maintenance, and static waiting. The output layer uses a linear activation function to output the Q value of each sub-task.

[0236] The four low-level execution networks correspond to the four sub-tasks mentioned above, and each low-level execution network adopts a deep Q-network structure.

[0237] In practice, the input layer of each low-level execution network receives a concatenated vector, which is composed of the decision state vector and the currently activated subtask encoding. The first fully connected layer of the low-level execution network contains 256 neurons and uses the ReLU activation function; the second fully connected layer contains 128 neurons and uses the ReLU activation function. The output layer contains 7 neurons, corresponding to seven atomic actions: opening water inlet valve A, opening water inlet valve B, closing water inlet valve A, closing water inlet valve B, starting the drain pump, closing the drain pump, and no action. The output layer uses a linear activation function to output the Q-value of each atomic action.

[0238] In specific implementation, both the high-level policy network and the low-level execution network adopt a dual-network structure, including a current Q network and a target Q network. The current Q network selects actions, the target Q network calculates the temporal difference target value, and copies the parameters of the current Q network to the target Q network at preset fixed intervals.

[0239] The reward function of the lower-level execution network is obtained based on the control objective of the current subtask, and the reward value is positively correlated with the degree of completion of the current subtask.

[0240] As an optional implementation, for the rapid water injection subtask, the liquid level rise is measured after each atomic action is performed, and the liquid level rise is used as an instant reward value. The reward value is positively correlated with the liquid level rise per unit time.

[0241] For the precise liquid replenishment sub-task, after each atomic action is performed, the difference between the current liquid level and the target liquid level is measured. The negative value of the difference is used as the instant reward value. The reward value is positively correlated with the closeness between the actual liquid level and the target liquid level.

[0242] For the liquid level maintenance subtask, after each atomic action is executed, the absolute value of the change in liquid level within the two control cycles before and after is measured. The negative value of the absolute value of the change is used as the instantaneous reward value. The reward value is negatively correlated with the liquid level fluctuation amplitude.

[0243] For idle subtasks, a fixed positive value will be used as an immediate reward value after each control cycle until the subtask ends.

[0244] After the current calibration process is completed, the liquid level error, total time, and stabilization time of multi-parameter measurements are obtained, and the weighted sum is taken as the negative value, which is used as the reward value of the high-level policy network.

[0245] As an optional implementation method, the difference between the final liquid level and the target liquid level is measured to obtain the final liquid level error, and the total time consumed by the entire process from start to finish is recorded.

[0246] The time required for each water quality parameter reading to stabilize during subsequent multi-parameter measurements is measured, and the maximum value is taken as the multi-parameter measurement stabilization time.

[0247] The final liquid level error, total time, and multi-parameter measurement stabilization time are all negative. These three negative values ​​are then weighted and summed to obtain the delay reward value of the high-level policy network.

[0248] In this embodiment, taking the pH low standard calibration process as an example, the current time is in the standard solution 1 water inlet stage, the liquid level is gradually rising, and the output state probability is rising state probability 0.91, maintaining state probability 0.06, and falling state probability 0.03; the prediction time to reach the high liquid level is 2.5 seconds.

[0249] The current calibration process type is low-standard calibration process, which is represented by one-hot encoding as [1,0,0], and the decision state vector is obtained by splicing them together.

[0250] In practice, the high-level policy network has an input layer dimension of 7, and two fully connected hidden layers: the first layer contains 128 neurons, and the second layer contains 64 neurons. Both layers use the ReLU activation function. The output layer dimension is equal to the preset number of target subtasks.

[0251] For example, for the low-standard calibration process, the subtasks are divided into five subtasks: evacuation subtask, rapid water intake subtask, precise fine-tuning subtask, static reading subtask, and cleaning subtask. The output layer consists of five units, each corresponding to the Q value of the subtask.

[0252] In practice, the low-level execution network sets up a deep Q-network independently for each subtask. The input layer dimension of each low-level network is 7, the hidden layer structure is the same as that of the high-level policy network, and the output layer dimension is equal to the number of atomic actions that can be executed under that subtask.

[0253] Taking the fine-tuning subtask as an example, the executable atomic actions include: opening standard solution valve 1, closing standard solution valve 1, opening the drain pump, closing the drain pump, and no action, totaling 5 atomic actions. Other subtasks set corresponding sets of atomic actions according to their control requirements.

[0254] In practice, the constructed seven-dimensional decision state vector is input into the current Q-network of the high-level policy network. After forward propagation, the Q values ​​of five sub-tasks are output, assuming they are -1.2 for emptying, 0.5 for rapid water intake, 2.8 for precise fine-tuning, 0.1 for static reading, and -0.8 for cleaning.

[0255] As an optional implementation, a greedy strategy (ε=0.1) is used to select the action, and the subtask with the largest Q value is selected, that is, the fine-tuning subtask with a Q value of 2.8. Then the target subtask of the current stage is determined to be fine-tuning.

[0256] Activate the low-level execution network corresponding to the precise fine-tuning subtask, input the same decision state vector into the current Q-network of this low-level network, and output the Q-values ​​of five atomic actions; for example, open standard solution valve 1 (3.5), close standard solution valve 1 (-1.0), open drainage pump (-2.0), close drainage pump (0.2), and no action (1.5). Similarly, using a greedy strategy, select the atomic action with the largest Q-value, i.e., open standard solution valve 1.

[0257] This atomic action is converted into a specific control command: the relay of standard liquid valve 1 is closed to maintain the water inlet state.

[0258] The current action to be executed, "Open Standard Liquid Valve 1," is recorded in the control command sequence. In the next control cycle, such as every 0.1 seconds, a new current value and action code are acquired, and the above steps are repeated until the process ends. All actions to be executed at all times are arranged in chronological order to generate a complete control command sequence.

[0259] Taking the precise fine-tuning subtask as an example, the reward function is to give a positive reward of +1 if the error between the current liquid level and the target liquid level is less than the preset first error threshold.

[0260] If the error exceeds the preset first error threshold, a negative reward of -1 is given; otherwise, the reward value is 0.

[0261] Assuming that after this low-level calibration, the recorded liquid level error is 0.05 mA, the total time is 25 seconds, and the stabilization time is 3 seconds, multiply these three indicators by their respective weights, for example, liquid level error weight 5, total time weight 1, and stabilization time weight 2, sum them up, and take the negative value to obtain the high-level reward value of -31.25.

[0262] Based on the current liquid level and process type, subtasks and atomic actions are adaptively selected to generate an optimized sequence of control instructions.

[0263] Regarding S104:

[0264] As an optional implementation, the sequence importance resampling particle filter algorithm is used to approximate the posterior probability distribution of the liquid level state using a set of weighted random samples, i.e., particles.

[0265] See Figure 3 The flowchart of a smart water station calibration cup level estimation method based on particle filtering provided in an embodiment of the present invention includes:

[0266] Using the current liquid level state probability distribution as prior information, initialize the particle set. Each particle contains three sets of state variables: liquid level height, liquid level change rate, and disturbance coefficient.

[0267] In practice, each particle is a three-dimensional state vector, including: liquid level height, whose value range corresponds to the lower to upper limit of the current analog signal range; liquid level change rate, whose value range is set according to the extreme values ​​of rise and fall rates statistically analyzed from historical data; and disturbance coefficient, which is used to characterize random disturbance factors not captured by the model, and whose value range is set according to the actual measured noise level on site.

[0268] The particle state transition equation is constructed based on the control command sequence, and the particle set is predicted and iterated to obtain the predicted particle set.

[0269] In specific implementation, the state transition equation includes a deterministic part and a stochastic part. The deterministic part determines the change in liquid level height based on the type of atomic action currently being executed: if the current action is to open the inlet valve, the liquid level height increases by an amount equal to the rated flow rate of the inlet valve multiplied by the execution time.

[0270] If the current action is to start the drain pump, the liquid level will decrease by an amount equal to the rated flow rate of the drain pump multiplied by the execution time.

[0271] If the current action is to close the valve or water pump, the liquid level will remain unchanged.

[0272] In practice, the update rules for the liquid level change rate include: when the inlet valve is opened, the change rate is adjusted in the positive direction, and the adjustment range is positively correlated with the inlet flow rate; when the drain pump is turned on, the change rate is adjusted in the negative direction, and the adjustment range is positively correlated with the drain flow rate; when the actuator is turned off, the change rate returns to zero, and the return speed is related to the damping characteristics.

[0273] The randomness component is used to describe the uncertainty of the system, including factors such as fluctuations in influent flow, fluctuations in drainage flow, and oscillations in liquid level.

[0274] As an alternative implementation, the random perturbation is modeled using a zero-mean Gaussian distribution, with the variance proportional to the perturbation coefficient.

[0275] The state transition equation is applied to each particle individually to obtain the predicted liquid level height, predicted rate of change, and predicted disturbance coefficient for each particle at the next time step. The prediction results for all particles constitute the predicted particle set.

[0276] In practice, the analog current signal and the liquid level dry contact switch signal in the second time series data are used as observation values ​​to construct an observation equation and calculate the deviation between the predicted value and the observed value of each particle.

[0277] As an optional implementation, an adaptive weighted fusion algorithm is used to construct the observation equation.

[0278] The least squares fitting algorithm is used to calculate the deviation of the current analog quantity. The predicted liquid level height of each particle is input, and the corresponding predicted current value is output.

[0279] In practice, a sliding window filtering algorithm is used to denoise the actual current observation values. Then, the mean square error algorithm is used to calculate the difference between the denoised current observation values ​​and the predicted current values, which is used as the deviation of the current analog quantity.

[0280] As an optional implementation, the predicted liquid level height of each particle is input into an adaptive adjustment algorithm to obtain a dynamic threshold determination result, thereby determining the liquid level switch state corresponding to the particle.

[0281] In specific implementation, the liquid level switch threshold is initialized, and the liquid level switch threshold is set according to the hardware parameters of the liquid level switch and the actual application scenario;

[0282] The liquid level switch threshold is dynamically adjusted by an adaptive adjustment algorithm, and the predicted liquid level height of the particle is compared with the dynamically adjusted threshold. If the predicted liquid level height is higher than the dynamic threshold, the liquid level switch state corresponding to the particle is determined to be "closed"; if it is lower than the dynamic threshold, it is determined to be "open". Thus, the liquid level switch state determination result corresponding to each particle is obtained.

[0283] As an optional implementation method, a Bayesian estimation algorithm is used, combined with the false alarm rate and false alarm rate of the switching signal, to calculate the matching degree between the determined switching state and the actual collected liquid level dry contact switching state, and to quantify the switching deviation.

[0284] In practice, the weights of analog current signals and dry contact switching signals are determined using the analytic hierarchy process (AHP).

[0285] For example, a hierarchical model is established, with "determining the weights of two signals" as the target layer, "analog current signal" and "dry contact switching signal" as the criterion layer, and "observation accuracy" and "signal stability" as the indicator layer.

[0286] Construct a judgment matrix to compare the importance of the two signals in the criterion layer pairwise and generate the judgment matrix;

[0287] Calculate the maximum eigenvalue and consistency index of the judgment matrix. If the consistency index is less than the preset threshold, the judgment matrix is ​​valid; otherwise, adjust the judgment matrix until the consistency requirement is met.

[0288] The weight vector is calculated, and the eigenvectors of the judgment matrix are solved by eigenvalue decomposition. The eigenvectors are then normalized to obtain the initial weights of the two signals.

[0289] The likelihood probability of each particle is calculated based on the switching deviation, and the likelihood probability is used as the update weight of each particle. The update weight of all particles is then normalized.

[0290] Calculate the reciprocal of the normalized weights sum of squares of the current particle set to obtain the effective particle count. If the effective particle count is less than a preset threshold, resample the particle set to obtain the filtered particle set.

[0291] The filtered particle set is weighted and summed to obtain the current liquid level estimate.

[0292] The variance and entropy of the liquid level state of the filtered particle set are calculated, the variance and entropy are mapped to a confidence score, and the confidence score is compared with a preset confidence threshold to generate a control command.

[0293] If the confidence score is greater than or equal to the preset confidence threshold, the current liquid level estimate is used as the control basis, and valve pump control commands consistent with the control command sequence are output.

[0294] If the confidence score is less than the preset confidence threshold, the particle set is reinitialized.

[0295] In this embodiment, the total number of particles is set to 500, and the probability of the liquid level output at the current moment is 0.91 for rising, 0.06 for maintaining, and 0.03 for falling.

[0296] As an optional implementation, during initialization, the state type of each particle is randomly sampled according to the probability distribution, and a random number between 0 and 1 is generated. If the random number is less than 0.91, the state is rising; if the random number is greater than or equal to 0.91 and less than 0.97, the state is remaining unchanged; if the random number is greater than or equal to 0.97, the state is falling.

[0297] The liquid level height and liquid level change rate are sampled according to the state type. If the state is rising, the liquid level height is sampled from a normal distribution with a variance of 0.1 centered on the current actual observed current value, and the liquid level change rate is sampled from a normal distribution with a variance of 0.01 centered on the rated rise rate of 0.2 mA / s.

[0298] If the state remains unchanged, the center of the liquid level height sampling is still 8.3 mA, and the rate of change is sampled from a normal distribution with a mean of 0 and a variance of 0.01.

[0299] If the state is decreasing, the rate of change is sampled from a normal distribution centered at the rated decreasing rate of -0.3 mA / s with a variance of 0.01. The perturbation coefficients of all particles are initialized to a fixed value of 0.001 mA / s², and the initial weights of all particles are equal, each being one-five-hundredth.

[0300] For example, the liquid level of particle 1 is set to 8.28 mA, with a change rate of 0.19 mA / s and a perturbation coefficient of 0.001; the liquid level of particle 2 is set to 8.32 mA, with a change rate of 0.21 mA / s and a perturbation coefficient of 0.001; the liquid level of particle 3 is set to 8.30 mA, with a change rate of 0.20 mA / s and a perturbation coefficient of 0.001; the liquid level of particle 4 is set to 8.25 mA, with a change rate of 0.18 mA / s and a perturbation coefficient of 0.001; and the liquid level of particle 5 is set to 8.35 mA, with a change rate of 0.22 mA / s and a perturbation coefficient of 0.001. Only the initialization states of five representative particles are listed here; actual calculations require calculations for all 500 particles.

[0301] The change in liquid level is obtained by multiplying the rise rate corresponding to the rated flow of the inlet valve by the time step. The rated rise rate is set to 0.2 mA per second based on historical calibration, and the time step is set to 0.1 seconds. Therefore, the height increases by 0.02 mA.

[0302] The update rule for the rate of change is that the new rate equals the old rate multiplied by the regression coefficient 0.9, plus the rated rate multiplied by 0.1, so that the rate is slowly adjusted toward the rated value.

[0303] For the randomness component: a zero-mean Gaussian random perturbation is added to the liquid level height, with the variance proportional to the perturbation coefficient. The perturbation coefficient of 0.001 multiplied by the time step of 0.1 seconds yields a process noise variance of 0.0001 mA squared and a standard deviation of 0.01 mA. A random perturbation is also added to the rate of change, with a variance of 0.0001. The perturbation coefficient remains unchanged in this example.

[0304] Calculate the predicted values ​​for each of the five particles:

[0305] The new liquid level height for particle 1 is 8.305, and the new rate of change is 0.191; the new liquid level height for particle 2 is 8.337, and the new rate of change is 0.209; the new liquid level height for particle 3 is 8.321, and the new rate of change is 0.20; the new liquid level height for particle 4 is 8.268, and the new rate of change is 0.182; the new liquid level height for particle 5 is 8.377, and the new rate of change is 0.218; the weights of all particles remain at the initial value of one-five-hundredth, and the perturbation coefficients remain unchanged.

[0306] In practice, the analog current signal after sliding window filtering is 8.3 mA; the high-level dry contact switch status is 0 (open), and the low-level dry contact switch status is 0 (open). The high-level switch threshold is set to 9.0 mA according to hardware parameters, and the low-level switch threshold is set to 7.0 mA.

[0307] The predicted current value for each particle is the predicted liquid level height. The absolute values ​​of the current deviations of particles 1, 2, 3, 4 and 5 are 0.005 mA, 0.037 mA, 0.021 mA, 0.032 mA and 0.077 mA, respectively.

[0308] In practice, the current likelihood factor adopts a Gaussian function, and the current observation noise variance is set to 0.01 mA squared based on the sensor accuracy. The current likelihood factor for each particle is then calculated.

[0309] Particle 1: The squared deviation is 0.000025, divided by twice the variance of 0.02, we get 0.00125, and taking the negative exponent, we get 0.99875; similarly, the current likelihood factors of particles 2, 3, 4, and 5 are 0.9338, 0.9782, 0.9501, and 0.7436, respectively.

[0310] A Bayesian estimation algorithm was used, with a false alarm rate of 0.01 and a false negative rate of 0.01. The predicted switch state of each particle was obtained by comparing its predicted liquid level height with the liquid level switch threshold. For example, the high liquid level switch threshold could be set to 9.0, and the low liquid level switch threshold to 7.0. When the predicted liquid level is higher than the high liquid level switch threshold, the high liquid level switch is 1, and the low liquid level switch is 0; similarly, when the predicted liquid level is lower than the low liquid level switch threshold, the low liquid level switch is 1, and the high liquid level switch is 0. When all particles predict liquid levels lower than the high liquid level switch threshold and higher than the low liquid level switch threshold, the predicted high liquid level switch is 0, and the predicted low liquid level switch is 0, which is a normal liquid level state consistent with the actual observation. For a single switch, when the prediction is 0 and the observation is 0, the observation likelihood is 0.99; for two independent switches, the joint switch likelihood factor is 0.9801, which is the same for all particles.

[0311] The joint likelihood probability of each particle is the current likelihood factor multiplied by the switch likelihood factor, resulting in 0.9788, 0.9153, 0.9588, 0.9311, and 0.7287 for the five particles.

[0312] The weights of all particles were normalized to obtain 0.002175, 0.002034, 0.002131, 0.002069, and 0.001619, respectively.

[0313] In practice, the normalized sum of squared weights of the current particle set is calculated, and the reciprocal is taken to obtain the effective number of particles.

[0314] For example, calculating the sum of squares of the normalized weights for all 500 particles, assuming it's 0.0025, results in 400 effective particles. A preset threshold is set to two-thirds of the total number of particles, approximately 333. Since 400 is greater than 333, particle diversity is good, and resampling is unnecessary.

[0315] If the number of effective particles is lower than the threshold, a multinomial resampling method is used to extract 500 new particles with replacement from the current particle set according to the weight distribution, and the weight of all new particles is reset to one-five-hundredth.

[0316] The liquid level height of each particle in the filtered particle set is weighted and summed to obtain the current liquid level estimate.

[0317] Assuming the weighted average of 500 particles yields a liquid level estimate of 8.32 mA, the variance and entropy of the liquid level height of the filtered particle set are calculated to obtain the confidence score.

[0318] Calculate the weighted mean of the liquid level height, then calculate the weighted variance, sum these products to obtain the total variance, assuming the total variance is 0.0012 mA squared.

[0319] If the maximum variance is set to 0.1 mA squared, then the score component based on variance is 0.988. The entropy value can be calculated using the continuous entropy formula. The maximum possible value of entropy corresponds to the case of uniform weight distribution. When the weights of 500 particles are equal, such as 0.002 for each, the entropy is 0.5317.

[0320] In this embodiment, an equal-weighted average is used, with each accounting for 50%, resulting in a final confidence score of 0.76.

[0321] The confidence score is lower than the preset confidence threshold of 0.85, therefore a re-initialization needs to be triggered.

[0322] In practice, the confidence threshold can be determined experimentally.

[0323] For example, comparison data between particle filter estimates and reference values ​​were collected in historical calibration processes, resulting in 1000 sets of samples. For each set of samples, a confidence score and estimation error were calculated.

[0324] Plot a scatter plot of confidence scores versus estimation errors, and count the proportion of estimation errors less than 0.1 mA under different scoring thresholds.

[0325] In practice, when the scoring threshold is 0.85, 96% of the samples have an estimation error of less than 0.1 mA, which meets the control accuracy requirements. If the threshold is increased to 0.9, although the accuracy is higher, it will lead to frequent reinitialization and reduce efficiency. Therefore, 0.85 is chosen as the balance point.

[0326] In this embodiment, the calculated confidence score is 0.76, which is less than the threshold of 0.85. Therefore, the current liquid level estimate is not used as the control basis. Instead, the particle set is reinitialized until the confidence score reaches the confidence threshold or the maximum number of retries, such as 3, is reached. If the target is still not met after 3 retries, the system enters a safety mode, suspends automatic control, and issues an alarm.

[0327] Based on the single-station control method, the present invention can also be extended to a swarm intelligence system consisting of multiple smart water stations.

[0328] In this specific implementation, this embodiment takes three smart water station sites A, B, and C as examples and uses federated learning to share the global model.

[0329] Each smart water station regularly uploads three types of information to the cloud-based collaboration center, including:

[0330] State transition probability matrix of a hierarchical hidden Markov model;

[0331] The current liquid level estimate and confidence score output by the particle filter;

[0332] Current calibration process type and abnormal event identifier.

[0333] In practice, each site completes one round of training of the hierarchical hidden Markov model locally, which can be done using the Baum-Welch algorithm. Each iteration includes an expectation step and a maximization step until the model converges.

[0334] After training, the updated gradients of the model parameters are obtained. The gradients include the gradient of the initial state probability vector, the gradient of the state transition probability matrix, and the gradients of the Gaussian observation parameters for each state.

[0335] Each site will encrypt the calculated gradients and upload them to the cloud-based collaborative center. As an optional implementation method, differential privacy technology is used for encryption, which involves adding random noise following a Laplace distribution to the gradient data. The scale parameter of the noise is set according to the privacy budget, which is set to 1.0.

[0336] The encrypted gradient is weighted and aggregated to calculate the global gradient. The weights can be determined based on the amount of data at each site.

[0337] For example, if site A has 1000 data entries, site B has 800, and site C has 1200, then the weights are 0.33, approximately 0.27, and 0.4, respectively.

[0338] As an alternative implementation, the hierarchical hidden Markov model can be updated using global gradients.

[0339] The initial parameters of the global model are copied from the local model parameters of the first participating site. After each aggregation, the global model parameters are updated according to the gradient descent method: the global model parameters are updated according to the gradient descent direction, and the update step size can be set to 0.1.

[0340] The structure of the global model is consistent with that of the local model, including the initial state probability vector, the state transition probability matrix, and the mean and variance of the Gaussian distribution of each state.

[0341] The updated global model parameters are distributed to all sites, and each site merges the updated global model with its local model.

[0342] For example, the new local model parameter is equal to the global model parameter multiplied by the fusion coefficient plus the original local model parameter multiplied by 1 minus the fusion coefficient; wherein the fusion coefficient can be dynamically adjusted according to the amount of site data.

[0343] In practice, if the particle filter confidence score at any site remains below the preset confidence threshold and cannot be recovered even after local reinitialization, the cloud-based collaborative center will activate the collaborative control mode.

[0344] As an alternative implementation, a graph structure can be constructed based on the geographical location of the water stations, pipeline connection relationships, or historical data similarity, where nodes represent each water station and edges represent the potential for collaboration between stations, such as sharing the same water source or being located in the same process section.

[0345] For the target water station, extract the real-time status information of its neighboring stations at the current moment, including the hidden state probability distribution, current control command, liquid level estimate and confidence score.

[0346] After normalizing this information, it is input into a graph attention network, which calculates attention weights for each neighboring site. The weighted aggregated neighborhood features are then concatenated with the local state vector of the target site and fed into a lightweight policy network to output auxiliary control suggestions.

[0347] Taking the sudden drop in pressure in the main inlet pipeline as an example, the inlet flow of the three stations decreased simultaneously, and the particle filter confidence scores of stations A, B, and C all decreased.

[0348] Suppose that the confidence level of site C drops sharply from 0.92 to 0.61, below the threshold of 0.7. Site C initiates local reinitialization, but the confidence level still does not recover.

[0349] In practice, information such as the abnormal event identifier of site C, the current confidence level of 0.61, the estimated liquid level of 8.1 mA, and the ongoing sub-task "rapid water inflow" are uploaded to the cloud collaboration center.

[0350] The confidence level of site C under normal operating conditions fluctuates between 0.85 and 0.95, with a maximum fluctuation of 0.15. The current decrease of 0.31 indicates a genuine anomaly.

[0351] The collaboration center extracts real-time status information for sites A and B:

[0352] Site A: Hidden state probability distribution [0.88, 0.08, 0.04], current control command "open inlet valve", code [1, 0, 0, 0, 0, 0, 0], liquid level estimate 8.2 mA, confidence level 0.78, calibration process type pH standard solution calibration, code [1, 0, 0].

[0353] Site B: Hidden state probability distribution [0.10, 0.85, 0.05], current control command "no action", code [0, 0, 0, 0, 0, 1], liquid level estimate 8.5 mA, confidence level 0.81, calibration process type pH standard solution calibration, code [1, 0, 0].

[0354] The information from stations A and B is concatenated in sequence to form a 15-dimensional feature vector, including 3-dimensional hidden state, 7-dimensional instruction, 1-dimensional liquid level, 1-dimensional confidence, and 3-dimensional process type, forming a neighborhood feature matrix with two rows and fifteen columns.

[0355] In a specific implementation, the neighborhood feature matrix is ​​input into a graph attention network, which includes a 64-dimensional fully connected layer. Feature transformation is performed through the fully connected layer, and the attention weight of each neighboring station to station C is calculated through a multi-head attention layer with four heads.

[0356] The results showed that the attention weight of site B was 0.7, and the attention weight of site A was 0.3.

[0357] The feature vectors of stations A and B are multiplied by their respective weights and then summed to obtain a weighted aggregated neighborhood feature vector. This vector is then concatenated with the local state vector of station C.

[0358] The local state of station C includes: liquid level estimate of 8.1 mA, confidence level of 0.61, and the current subtask "rapid water inflow" encoding [0,1,0,0], totaling 6 dimensions, which are concatenated to obtain a 21-dimensional vector.

[0359] In practice, the concatenated vector is input into the policy network, which is a two-layer fully connected network. The first layer has 128 neurons with the ReLU activation function, and the second layer has 64 neurons with the ReLU activation function.

[0360] The output layer is divided into two branches. The action type branch outputs 4 neurons, corresponding to four suggestion types: adjusting water inlet time, adjusting target liquid level, switching subtasks, and no action, using the softmax activation function. The parameter branch outputs the parameter values ​​for the corresponding type, using the linear activation function.

[0361] The policy network outputs an action type probability distribution [0.82, 0.08, 0.06, 0.04], with a parameter branch output value of 3.2.

[0362] The collaboration center selects the suggestion type with the highest probability, "adjust water inlet time," rounds the parameter to 3 seconds, and generates the auxiliary control suggestion, "pause water inlet for 3 seconds, referencing station B to maintain the current liquid level." This suggestion is then sent to station C.

[0363] For example, the low-level execution network of site C is a deep Q network corresponding to the fine-tuning subtask. The Q value of each atomic action of the deep Q network at the current moment is originally: open water valve 3.2, close water valve -1.0, open drain pump -2.0, close drain pump 0.3, no action 1.8;

[0364] Based on the aforementioned auxiliary control suggestion, the deep Q-network temporarily increases the Q-value of "no action" to 4.5, making it the action with the maximum Q-value. The network selects no action, generates a specific control command "close the inlet valve and hold for 3 seconds," and executes it.

[0365] Based on the same inventive concept, this embodiment also provides a smart water station calibration cup liquid level control system for the smart water station calibration cup liquid level control method. Since the principle of the control method in this embodiment is similar to the smart water station calibration cup liquid level control method described above, the implementation of the control system can refer to the implementation of the method, and the repeated parts will not be described again.

[0366] See Figure 4The diagram shown is a schematic of a smart water station calibration cup level control system provided in this embodiment. The system includes: a data acquisition module 10, a prediction module 20, a decision-making module 30, and a control module 40, wherein:

[0367] Data acquisition module 10: used to acquire first time series data and second time series data, and perform time alignment and feature extraction on the first time series data and the second time series data to obtain a multi-dimensional feature sequence. The first time series data represents control action information, and the second time series data represents liquid level change information.

[0368] Prediction module 20: used to input the multidimensional feature sequence into a hierarchical hidden Markov model including at least two levels, so as to output the state probability distribution of the current liquid level and the predicted time to reach the target liquid level;

[0369] Decision module 30: used to construct a decision state vector based on the state probability distribution and prediction time, and generate a control command sequence through a preset hierarchical reinforcement learning model;

[0370] Control module 40: Based on the control command sequence, it calculates the current liquid level estimate using an adaptive particle filter algorithm, calculates the confidence score corresponding to the liquid level estimate according to the dispersion of the particle set, compares the confidence score with a preset confidence threshold, and generates control commands.

[0371] Those skilled in the art will understand that, in the methods described above in the specific embodiments, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic. It should be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0372] In the description of this specification, the terms "exemplary," "for example," "specifically," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

Claims

1. A method for controlling the liquid level of a calibration cup in a smart water station, characterized in that, The method includes: First time-series data and second time-series data are acquired, and time alignment and feature extraction are performed on the first time-series data and the second time-series data to obtain a multi-dimensional feature sequence. The first time-series data represents control action information, and the second time-series data represents liquid level change information. The multidimensional feature sequence is input into a hierarchical hidden Markov model that includes at least two levels to output the current liquid level state probability distribution and the predicted time to reach the target liquid level. A decision state vector is constructed based on the state probability distribution and prediction time, and a control command sequence is generated through a preset hierarchical reinforcement learning model. Based on the control command sequence, the current liquid level estimate is calculated using an adaptive particle filter algorithm. The confidence score corresponding to the liquid level estimate is calculated based on the dispersion of the particle set. The confidence score is then compared with a preset confidence threshold to generate control commands.

2. The method for controlling the liquid level of a smart water station calibration cup according to claim 1, characterized in that, The acquisition of the first time series data and the second time series data includes: The water quality parameter type and calibration procedure type are determined based on the current task. The water quality parameter type includes one of pH value, conductivity, turbidity, and dissolved oxygen. The calibration procedure type includes one of blind sample measurement procedure, standard solution calibration procedure, and air calibration procedure. The first timing data includes valve and pump switching signals, and the second timing data includes liquid level dry contact switching signals and current analog signals. The first time series data and the second time series data are acquired using the same time reference.

3. The method for controlling the liquid level of a smart water station calibration cup according to claim 2, characterized in that, The step of time alignment and feature extraction of the first time series data and the second time series data includes: The second timing data is preprocessed to debounce. When the transition edge of the liquid level dry contact switch signal is detected, a current waveform segment is extracted based on a sliding window of a preset time length. Extract the time-domain and frequency-domain features of the current waveform segment as the first feature vector; The first feature vector is input into a pre-trained lightweight gradient booster classification model, which outputs the discrimination result of the current current waveform segment; the lightweight gradient booster classification model determines whether the current waveform belongs to the effective load or interference / no load category. Based on the discrimination result, the switch signal determined to be valid load is taken as a valid event. Based on the dynamic time warping algorithm, the first time series data and the second time series data corresponding to the valid event are time-aligned.

4. The method for controlling the liquid level of a smart water station calibration cup according to claim 1, characterized in that, The step of inputting the multidimensional feature sequence into a hierarchical hidden Markov model comprising at least two levels includes: The first and second time series data, which are marked as valid events after time alignment, are used as inputs to a hierarchical hidden Markov model, and two levels are constructed, including a first level representing the action sequence and a second level representing the liquid level change process. The first level of hidden state set includes different combinations of working states. Each hidden state corresponds to a set of pump valve open or closed states. The transition between hidden states is driven by control commands. The second level of hidden state set includes at least three states: liquid level rising, holding, and falling. The input data is decoded, and the posterior probability of belonging to each second-level state at each time step is recursively calculated using a forward-backward algorithm. The probability distribution of the liquid level state at the current time step is then output. Based on the second-level state obtained from decoding and its average duration, combined with the liquid level change rate per unit time under the corresponding state, the predicted time from the current liquid level to the target liquid level is calculated using the state duration model.

5. The method for controlling the liquid level of a smart water station calibration cup according to claim 4, characterized in that, The hierarchical hidden Markov model also includes parameter acquisition: The set of hidden states at the first level is encoded to obtain the first level of hidden states, where each hidden state is represented by a 4-bit binary number to indicate the opening or closing of the inlet valve and the drain pump, respectively. The Baum-Welch algorithm is used to iteratively train historical data, calculate the probability of the observed sequence appearing under the current parameters, and adjust the transition probability values ​​between the states of the second level based on the probability until convergence to obtain the state transition probability matrix of the second level. The duration of each second-level state in the historical data is statistically analyzed, and the gamma distribution shape parameters and scale parameters corresponding to each state are obtained by fitting the maximum likelihood estimation method. Calculate the mean and variance of the second-level liquid level observations, and use them as the Gaussian distribution parameters corresponding to each second-level hidden state.

6. The method for controlling the liquid level of a smart water station calibration cup according to claim 1, characterized in that, The generation of control command sequences through a preset hierarchical reinforcement learning model includes: The state probability distribution, prediction time, and current calibration process type are concatenated to construct a decision state vector; The hierarchical reinforcement learning model includes a high-level policy network and a low-level execution network. The decision state vector is input into the high-level policy network, and a deep Q-network structure is used to output the Q value of each sub-task. The sub-task with the largest Q value is selected as the target sub-task of the current stage. Based on the target subtask, activate the corresponding low-level execution network and select the atomic action with the largest Q value as the action to be executed at the current moment; Arrange all actions to be executed in sequence to generate a control instruction sequence.

7. The method for controlling the liquid level of a smart water station calibration cup according to claim 6, characterized in that, The hierarchical reinforcement learning model includes a high-level policy network and a low-level execution network, including: Both the high-level policy network and the low-level execution network adopt a dual-network structure, including a current Q network and a target Q network. The current Q network selects actions, the target Q network calculates the temporal difference target value, and copies the parameters of the current Q network to the target Q network at preset fixed intervals. The reward function of the lower-level execution network is obtained based on the control objective of the current subtask, and the reward value is positively correlated with the degree of completion of the current subtask. After the current calibration process is completed, the liquid level error, total time, and stabilization time of multi-parameter measurements are obtained, and the weighted sum is taken as the negative value, which is used as the reward value of the high-level policy network.

8. The method for controlling the liquid level of a calibration cup in a smart water station according to claim 1, characterized in that, The current liquid level estimate calculated using the adaptive particle filter algorithm includes: Using the current liquid level state probability distribution as prior information, initialize the particle set. Each particle contains three sets of state variables: liquid level height, liquid level change rate, and disturbance coefficient. The particle state transition equation is constructed based on the control command sequence, and the particle set is predicted and iterated to obtain the predicted particle set. Using the analog current signal and the liquid level dry contact switch signal in the second time series data as observation values, an observation equation is constructed to calculate the deviation between the predicted value and the observed value for each particle. The likelihood probability of each particle is calculated based on the aforementioned deviation. The likelihood probability is used as the update weight of each particle, and the update weight of all particles is normalized. Calculate the reciprocal of the normalized weights sum of squares of the current particle set to obtain the effective particle count. If the effective particle count is less than a preset threshold, resample the particle set to obtain the filtered particle set. The filtered particle set is weighted and summed to obtain the current liquid level estimate. The variance and entropy of the liquid level state of the filtered particle set are calculated, the variance and entropy are mapped to a confidence score, and the confidence score is compared with a preset confidence threshold to generate a control command.

9. The method for controlling the liquid level of a smart water station calibration cup according to claim 8, characterized in that, The step of comparing the confidence score with a preset confidence threshold includes: If the confidence score is greater than or equal to the preset confidence threshold, the current liquid level estimate is used as the control basis, and valve pump control commands consistent with the control command sequence are output. If the confidence score is less than the preset confidence threshold, the particle set is reinitialized.

10. A smart water station calibration cup level control system, used to implement the smart water station calibration cup level control method according to any one of claims 1-9, characterized in that, The system includes: Data acquisition module: used to acquire first time series data and second time series data, and perform time alignment and feature extraction on the first time series data and second time series data to obtain a multi-dimensional feature sequence. The first time series data represents control action information, and the second time series data represents liquid level change information. Prediction module: used to input the multidimensional feature sequence into a hierarchical hidden Markov model including at least two levels, so as to output the state probability distribution of the current liquid level and the predicted time to reach the target liquid level; Decision module: used to construct a decision state vector based on the state probability distribution and prediction time, and generate a sequence of control commands through a preset hierarchical reinforcement learning model; Control module: Based on the control command sequence, it calculates the current liquid level estimate using an adaptive particle filter algorithm, calculates the confidence score corresponding to the liquid level estimate according to the dispersion of the particle set, compares the confidence score with a preset confidence threshold, and generates control commands.

Images