Water quality monitoring data dynamic optimization method and device based on cloud end and side end cooperation

Abstract

The invention relates to the technical field of water quality monitoring, and provides a water quality monitoring data dynamic optimization method and device based on cloud and side end collaboration, and the method comprises the steps: receiving real-time water quality monitoring data sent by a plurality of side end devices; associating the real-time water quality monitoring data with historical sampling data in a pre-established water quality sample library; fitting training is carried out on the associated real-time water quality monitoring data and historical sampling data in the water quality sample library to obtain a water quality prediction model, and fitting training is carried out based on feature correlation and label consistency of the real-time data and the historical data; calculating the difference between the output result of the water quality prediction model and the corresponding real-time water quality monitoring data, and determining the parameter adjustment amount of the data processing model corresponding to the edge equipment according to the difference; and sending the parameter adjustment amount to the corresponding edge end equipment, so that the edge end equipment updates the parameters of the corresponding data processing model according to the parameter adjustment amount, and dynamic optimization of the edge end data processing model is realized.

Description

Technical Field

[0001] This application relates to the field of water quality monitoring technology, and in particular to a method and apparatus for dynamic optimization of water quality monitoring data based on cloud and edge collaboration. Background Technology

[0002] Existing water quality monitoring systems typically employ a simple "edge-end acquisition + cloud storage" architecture. Edge devices only perform preliminary filtering of the raw sensor data (such as fixed-parameter Kalman filtering), while the cloud serves merely as a historical data storage center, lacking the ability to dynamically optimize the edge data processing model. This traditional approach suffers from the following technical shortcomings: 1. Static data optimization: The parameters of the edge data processing model (such as filter gain and machine learning algorithm weights) need to be preset manually and cannot be dynamically adjusted according to the real-time monitoring environment. This results in incomplete noise suppression and fixed feature extraction dimensions, and the reliability of the data decreases significantly, especially when there are sudden changes in water quality (such as sudden pollution).

[0003] 2. Inefficient edge-cloud collaboration: The interaction between the cloud and the edge is limited to data transmission, and a closed-loop collaboration mechanism of "model training-parameter feedback" has not been formed. It is impossible to use the spatiotemporal correlation of historical data to improve the quality of real-time data. The model prediction accuracy depends on the initial parameter settings and has insufficient adaptive capability.

[0004] 3. Inefficient allocation of computing power: The computing tasks of edge devices and the load of the cloud lack dynamic coordination, often resulting in idle or overloaded computing power on the edge, while the training frequency of cloud models is fixed, making it difficult to balance real-time performance and computing resource utilization.

[0005] Therefore, a method is urgently needed to solve at least one of the above problems. Summary of the Invention

[0006] This application provides a method and apparatus for dynamic optimization of water quality monitoring data based on cloud and edge collaboration. It aims to solve the problem that although some solutions in the current technology attempt to integrate lightweight algorithms at the edge or perform batch model training in the cloud, none of them have built an edge-cloud collaborative optimization link based on real-time data differences, nor have they involved a mechanism for dynamically adjusting the processing strategy according to the computing power status, making it difficult to achieve dynamic adaptive optimization of water quality monitoring data.

[0007] In a first aspect, this application provides a method for dynamic optimization of water quality monitoring data based on cloud and edge collaboration, applied to a cloud server, wherein the cloud server is communicatively connected to multiple edge devices, and each edge device is equipped with multiple water quality sensors and a data processing model. The method includes: The system receives real-time water quality monitoring data from multiple edge devices. The real-time water quality monitoring data is obtained by the edge devices after feature extraction and real-time optimization of the raw data collected by multiple water quality sensors. The system associates the real-time water quality monitoring data with historical sampling data in a pre-established water quality sample library. The water quality sample library contains historical water quality data from different times and different monitoring points, as well as corresponding water quality category or pollution level label information. A water quality prediction model is obtained by fitting and training the correlated real-time water quality monitoring data with historical sampling data in the water quality sample library. The fitting and training is based on the feature correlation and label consistency between the real-time data and the historical data. The difference between the output of the water quality prediction model and the corresponding real-time water quality monitoring data is calculated. Based on the difference, the parameter adjustment amount of the data processing model corresponding to the edge device is determined. The parameter adjustment amount includes the filter gain parameter of Kalman filter and the model weight parameter of lightweight machine learning algorithm. The parameter adjustment amount is sent to the corresponding edge device, so that the edge device updates the parameters of the corresponding data processing model according to the parameter adjustment amount, thereby realizing the dynamic optimization of the edge data processing model.

[0008] In some embodiments, the method further includes: during the process of receiving real-time data, associating historical data, fitting and training models, calculating parameter adjustments, and sending parameter adjustments, allocating computing tasks between the cloud server and the edge device through a preset computing power balancing and coordination strategy. The computing power balancing and coordination strategy dynamically adjusts the complexity of data processing on the edge device and the frequency of model training on the cloud server based on the real-time computing power status of the edge device and the load status of the cloud server. The real-time computing power status includes CPU utilization and memory usage, the load status includes the number of currently processed tasks and network bandwidth usage, and the complexity includes the feature extraction dimension of the lightweight machine learning algorithm and the number of Kalman filter iterations.

[0009] In some embodiments, the real-time water quality monitoring data is obtained by edge device after feature extraction and real-time optimization of raw data collected by multiple water quality sensors. This includes: the edge device performing time-series segmentation of the raw data using a sliding window algorithm, calculating time-domain statistical features, frequency-domain energy features, and inter-sensor correlation features for each segment, using an adaptive Kalman filter algorithm to suppress noise in the raw data of each sensor, and dynamically adjusting the filter gain parameters by estimating the sensor measurement noise covariance matrix in real time to obtain optimized real-time water quality monitoring data.

[0010] In some embodiments, associating the real-time water quality monitoring data with historical sampling data in a pre-established water quality sample library includes: constructing a spatial distance index based on the geographical coordinates of the monitoring point; performing preliminary screening of the real-time data with historical data from the same monitoring point or within a spatial distance threshold in the water quality sample library; extracting historical data within a preset time window based on the data collection timestamp; and performing multi-dimensional matching through water quality parameter types and data quality identifiers to form spatiotemporal parameter matching associated data pairs; the water quality parameter types include at least pH value, dissolved oxygen, and conductivity.

[0011] In some embodiments, the step of fitting and training the correlated real-time water quality monitoring data with historical sampling data in the water quality sample library to obtain a water quality prediction model includes: using a time-series decomposition algorithm to decompose the correlated data pairs into trend components, periodic components, and random components; constructing a time-series prediction model based on a long short-term memory network, using the decomposed components as input features, and learning the temporal evolution law of water quality parameters through end-to-end training; and using an attention mechanism to automatically weight the influence of historical data at different time steps on the current prediction, thereby optimizing the model's response capability to abnormal data.

[0012] In some embodiments, calculating the difference between the output of the water quality prediction model and the corresponding real-time water quality monitoring data includes: setting differentiated loss function weights for different water quality parameters based on their monitoring accuracy requirements; calculating the weighted root mean square error between the predicted and measured values, wherein the error weights corresponding to parameters with pollution levels exceeding a threshold are automatically amplified; and calculating the dynamic error mean through a sliding time window to eliminate the interference of random noise on the difference calculation.

[0013] In some embodiments, determining the parameter adjustment amount of the data processing model corresponding to the edge device based on the difference includes: calculating the gradient value of the difference with respect to the edge model parameters using the backpropagation algorithm; updating the gain coefficients of the state transition matrix and observation matrix using the gradient descent method for Kalman filter parameters; adjusting only the model weight parameters that contribute significantly to the current error using sparse optimization techniques for lightweight machine learning algorithms; and attenuating the gradient value using the exponential smoothing method to suppress oscillations during parameter adjustment.

[0014] In some embodiments, the edge device updates the parameters of the corresponding data processing model according to the parameter adjustment amount to achieve dynamic optimization of the edge data processing model, including: after receiving the parameter adjustment amount, the edge device first verifies the adjustment amount; if the verification fails, the historical parameters are used; an incremental update strategy is adopted to adjust the iterative parameters of the Kalman filter and the weight matrix of the machine learning model in batches; after the update is completed, sensor data for a period of time is collected for model verification; if the verification fails, a parameter rollback mechanism is triggered.

[0015] In some embodiments, the method further includes: deploying a reinforcement learning agent in a cloud server, wherein the agent uses the reduction rate of prediction error after edge model parameter adjustment as a reward signal to automatically optimize parameter adjustment strategies through a deep reinforcement learning algorithm; the reinforcement learning agent dynamically generates differentiated parameter adjustment rules for different water quality parameters and different edge device computing power states to form an adaptive parameter adjustment strategy library; when a sudden change pattern is detected in the water quality data, the agent automatically switches to an emergency adjustment strategy, increasing the adjustment step size of the model parameters and increasing the adjustment frequency.

[0016] Secondly, this application provides a dynamic optimization device for water quality monitoring data based on cloud and edge collaboration, applied to a cloud server. The cloud server is communicatively connected to multiple edge devices, each edge device being deployed with multiple water quality sensors and a data processing model. The device is used to implement the method provided in any embodiment of this application, and the device includes: The data receiving unit is used to receive real-time water quality monitoring data sent by multiple edge devices. The real-time water quality monitoring data is obtained by the edge devices after feature extraction and real-time optimization of the raw data collected by multiple water quality sensors. The real-time water quality monitoring data is associated with historical sampling data in a pre-established water quality sample library. The water quality sample library contains historical water quality data at different times and different monitoring points, as well as corresponding water quality category or pollution degree label information. The fitting training unit is used to fit and train the correlated real-time water quality monitoring data with historical sampling data in the water quality sample library to obtain a water quality prediction model. The fitting training is based on the feature correlation and label consistency between real-time data and historical data. The unit calculates the difference between the output of the water quality prediction model and the corresponding real-time water quality monitoring data, and determines the parameter adjustment amount of the data processing model corresponding to the edge device based on the difference. The parameter adjustment amount includes the filter gain parameter of Kalman filtering and the model weight parameter of lightweight machine learning algorithm. The adjustment sending unit is used to send the parameter adjustment amount to the corresponding edge device, so that the edge device updates the parameters of the corresponding data processing model according to the parameter adjustment amount, thereby realizing the dynamic optimization of the edge data processing model.

[0017] Thirdly, this application also provides a cloud server, including: Memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, in executing the computer program, implement the steps of the dynamic optimization method for water quality monitoring data based on cloud and edge collaboration as described in the first aspect above.

[0018] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the steps of the dynamic optimization method for water quality monitoring data based on cloud and edge collaboration as described in the first aspect above.

[0019] By performing spatiotemporal correlation analysis and fitting training on real-time and historical data in the cloud, parameter adjustments (such as Kalman filter gain and machine learning weights) are generated for the edge model, achieving closed-loop optimization of the edge data processing model. This significantly reduces sensor noise interference and improves the accuracy of feature extraction. Based on real-time data differences, model parameters are dynamically adjusted, enabling the system to automatically adapt to water quality changes (such as seasonal fluctuations and sudden pollution events), avoiding prediction lags or misjudgments caused by traditional static parameters. A computing power balancing strategy dynamically allocates tasks between the edge and cloud, adjusting data processing complexity (such as feature dimensions and filter iterations) according to edge computing power status. Combined with cloud load optimization, the model training frequency is increased, improving the overall system's real-time performance and resource utilization. Through dynamic modeling in a collaborative manner between the edge and cloud, the time chain from data acquisition to anomaly identification is shortened, providing more reliable real-time data support for water pollution early warning and emergency response.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic flowchart illustrating the steps of a method for dynamically optimizing water quality monitoring data based on cloud and edge collaboration, provided in an embodiment of this application. Figure 2 This is a schematic diagram of the structure of a water quality monitoring data dynamic optimization device based on cloud and edge collaboration provided in an embodiment of this application; Figure 3 This is a schematic block diagram of the structure of a cloud server provided in one embodiment of this application.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0026] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0027] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0028] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0030] With the increasing demand for water environment management, water quality monitoring systems are gradually developing towards a collaborative "cloud + edge" architecture. Edge devices (such as water quality monitoring stations and buoys) deploy multiple sensors to collect raw data and output real-time water quality monitoring data through local data processing models (such as Kalman filtering and lightweight machine learning); cloud servers receive data from multiple edge devices and perform centralized analysis and prediction.

[0031] The existing technology has the following significant shortcomings: (1) Fixed edge data processing model with poor adaptability: The Kalman filter parameters (such as filter gain) and model weights of lightweight machine learning algorithms of edge devices are mostly preset and cannot be dynamically adjusted according to real-time water quality changes (such as sudden discharge of industrial wastewater or sudden changes in turbidity caused by seasonal precipitation). For example, when water quality characteristics (such as dissolved oxygen concentration) fluctuate suddenly and drastically, the fixed Kalman filter gain cannot effectively suppress noise, resulting in increased deviation of real-time monitoring data; the fixed machine learning model weights cannot capture new water quality feature associations (such as a sudden increase in the correlation between chemical oxygen demand and turbidity), resulting in a decrease in the accuracy of feature extraction.

[0032] (2) Lack of closed-loop collaboration between cloud and edge, resulting in low prediction accuracy: Existing cloud servers mostly store or perform simple statistics on real-time data uploaded from the edge in isolation, without associating real-time data with historical sampling data (such as seasonal water quality data of the same monitoring point over the past 3 years, and historical data of similar pollution events). For example, when the current pH value of a monitoring point is 6.5 (close to the acid threshold), the existing cloud model does not combine the pollution event data of the monitoring point when the pH value dropped below 6.0 in history (such as acid pollution caused by industrial wastewater discharge), and cannot explore the trend characteristics of water quality changes. This results in poor generalization ability of water quality prediction models (such as pollution degree prediction) and a large deviation between the output results and the actual water quality status.

[0033] (3) Lack of precise feedback on parameter adjustment, resulting in low optimization efficiency: In existing methods, the cloud does not provide parameter adjustment instructions to the edge based on the difference between the output of the prediction model and the real-time data. For example, when there is a significant difference between the pollution level output by the cloud prediction model (e.g., "moderate pollution") and the actual pollution level corresponding to the real-time monitoring data at the edge (e.g., "heavy pollution"), the existing system cannot locate the root cause of the difference (e.g., the edge Kalman filter parameter is too large, causing the data to be smoothed too much, or the machine learning model weights do not focus on key features), and it is even more impossible to adjust the edge parameters in a targeted manner, resulting in the edge data processing effect not being able to be dynamically optimized according to the cloud prediction needs.

[0034] In summary, existing technologies have not solved the problems of "dynamic adaptation of edge models to real-time water quality changes", "cloud-edge closed-loop collaboration to improve prediction accuracy", and "precise optimization of edge parameters based on prediction differences". There is an urgent need for a method that can achieve deep collaboration between cloud and edge and dynamically optimize edge data processing models.

[0035] To resolve the above issues, please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic flowchart illustrating the steps of a dynamic optimization method for water quality monitoring data based on cloud and edge collaboration, according to an embodiment of this application. The method is applied to a cloud server, which is communicatively connected to multiple edge devices, each edge device being equipped with multiple water quality sensors and a data processing model.

[0036] It should be noted that the acquisition of any information involved in the provided methods is in compliance with relevant regulations and with the consent of the relevant users, and will not infringe on user privacy or violate relevant laws and regulations.

[0037] Specifically, such as Figure 1 As shown, the provided method for dynamic optimization of water quality monitoring data based on cloud and edge collaboration includes steps S101 to S103, which are detailed below: Step S101. Receive real-time water quality monitoring data sent by multiple edge devices. The real-time water quality monitoring data is obtained by the edge devices after feature extraction and real-time optimization of the raw data collected by multiple water quality sensors. Associate the real-time water quality monitoring data with historical sampling data in a pre-established water quality sample library. The water quality sample library contains historical water quality data at different times and different monitoring points, as well as corresponding water quality category or pollution level label information.

[0038] Specifically, this method addresses the problems of poor adaptability of edge models, low prediction accuracy in the cloud, and insufficient efficiency of parameter optimization in existing technologies by constructing a closed-loop collaborative mechanism between the cloud and the edge.

[0039] Step S101 receives real-time water quality monitoring data from multiple edge devices (such as water quality monitoring stations and buoys) via a cloud server. This data is generated by the edge devices after collecting raw data through multiple sensors (such as pH, dissolved oxygen, turbidity, and chemical oxygen demand sensors) and then undergoing preliminary processing by a local data processing model (such as Kalman filtering and lightweight machine learning models).

[0040] Edge data processing models may include: Kalman filtering: used for noise suppression and data smoothing, but requires dynamic adjustment of the filter gain to cope with sudden fluctuations in water quality. Lightweight machine learning models (such as decision trees and linear regression): used for feature extraction (such as the correlation between dissolved oxygen and turbidity), but require dynamic updates of model weights to capture new features.

[0041] The cloud-based system links real-time data with a pre-built water quality sample database, which includes: spatiotemporal data: seasonal water quality data for the same monitoring point over the past three years, and historical records of similar pollution events (such as industrial wastewater discharge). Tagging information: water quality category (such as Class I-V) or pollution level label (such as "lightly polluted / heavily polluted").

[0042] The association method is based on the monitoring point location, timestamp, and similarity of water quality parameters (such as matching pH change patterns with historical acidic pollution events). For example, if the current pH value is 6.5, it can be associated with pollution source data from historical periods when the pH at this monitoring point dropped below 6.0 to help predict trends. Time-series databases (such as InfluxDB) can be used to store real-time data, while relational databases (such as MySQL) can store historical sample databases. Similarity algorithms (such as DTW dynamic time warping) can be used to match the fluctuation patterns of real-time data with those of historical events.

[0043] Step S102. Fit and train the correlated real-time water quality monitoring data with historical sampling data in the water quality sample library to obtain a water quality prediction model. The fitting and training is based on the feature correlation and label consistency between real-time data and historical data. Calculate the difference between the output of the water quality prediction model and the corresponding real-time water quality monitoring data. Determine the parameter adjustment amount of the data processing model corresponding to the edge device based on the difference. The parameter adjustment amount includes the filter gain parameter of Kalman filtering and the model weight parameter of lightweight machine learning algorithm.

[0044] Specifically, the input data for training the water quality prediction model includes: real-time data and historical data (feature matrix + labels) after correlation.

[0045] Training objectives include: Feature correlation: mining the correlation between real-time parameters and historical data (e.g., the similarity between the current surge in turbidity and historical precipitation events). Label consistency: ensuring that the predicted results (e.g., "moderate pollution") are consistent with the labels of similar historical events.

[0046] Model selection utilizes time-series prediction models (such as LSTM) or ensemble learning (such as XGBoost), incorporating attention mechanisms to focus on key features (such as sudden increases in COD). Discrepancy analysis and parameter adjustment are performed by calculating the difference between the predicted results and real-time edge data (e.g., predicted pH=6.8 and actual pH=6.5).

[0047] The parameter adjustment parameters include: Kalman filter gain: if the difference is due to insufficient noise suppression (such as excessive fluctuations in dissolved oxygen data), increase the filter gain; if excessive data smoothing causes delay, decrease the gain.

[0048] Machine learning model weights: If the discrepancy stems from feature extraction bias (such as failing to capture the novel association between COD and turbidity), the weights are updated using gradient descent to strengthen key features. The discrepancy is calculated using mean squared error (MSE) or KL divergence to pinpoint the source of the bias. Parameter adjustments are dynamically generated using backpropagation (for machine learning models) or adaptive Kalman filtering theory (such as Sage-Husa filtering).

[0049] Step S103. Send the parameter adjustment amount to the corresponding edge device, so that the edge device updates the parameters of the corresponding data processing model according to the parameter adjustment amount, thereby realizing the dynamic optimization of the edge data processing model.

[0050] Specifically, parameter distribution and updates are handled via the cloud, sending adjusted parameters (such as filter gain and model weights) to the corresponding edge devices. Upon receiving the parameters, the edge devices immediately update their local models, including: Kalman filtering: replacing the original gain matrix to optimize real-time data smoothing; and machine learning models: loading new weights to improve feature extraction accuracy.

[0051] After the edge device is updated, data is re-collected and uploaded to the cloud. The cloud verifies whether the prediction discrepancy has narrowed, forming a closed loop of "edge optimization → cloud verification → further optimization". Lightweight communication protocols (such as MQTT) can be used to achieve efficient parameter distribution. Edge devices support hot model updates via firmware OTA (Over-The-Air) updates to ensure uninterrupted operation.

[0052] In some embodiments, the method further includes: during the process of receiving real-time data, associating historical data, fitting and training models, calculating parameter adjustments, and sending parameter adjustments, allocating computing tasks between the cloud server and the edge device through a preset computing power balancing and coordination strategy. The computing power balancing and coordination strategy dynamically adjusts the complexity of data processing on the edge device and the frequency of model training on the cloud server based on the real-time computing power status of the edge device and the load status of the cloud server. The real-time computing power status includes CPU utilization and memory usage, the load status includes the number of currently processed tasks and network bandwidth usage, and the complexity includes the feature extraction dimension of the lightweight machine learning algorithm and the number of Kalman filter iterations.

[0053] Specifically, this embodiment aims to address the problem of uneven distribution of computing resources between the cloud and the edge. Through a dynamic task allocation strategy, it avoids system performance degradation due to insufficient edge computing power or excessive cloud load. The core principle is to dynamically adjust the complexity of edge data processing tasks (such as feature extraction dimensions and Kalman filter iterations) and the frequency of cloud model training based on the real-time computing power status of edge devices (CPU utilization, memory usage) and the load status of the cloud server (current number of tasks being processed, network bandwidth usage).

[0054] Monitoring and reporting are achieved by each edge device periodically (e.g., every 5 minutes) reporting its real-time computing power status, such as CPU utilization and memory usage, to the cloud server. The cloud server simultaneously monitors its own task queue length and network bandwidth usage. The cloud server runs a decision engine. This engine dynamically adjusts based on preset rules: if the CPU utilization of an edge device consistently exceeds 80%, the cloud instructs that edge device to reduce data processing complexity, for example, reducing the feature extraction dimension from 10 to 5, or halving the number of Kalman filter iterations. Simultaneously, the cloud server can temporarily reduce the frequency of data analysis and model training for that edge device, instead receiving raw or simplified data.

[0055] If the number of tasks on the cloud server exceeds the threshold or the bandwidth utilization is too high, edge devices with sufficient computing power (e.g., CPU utilization below 30%) will perform more complex local preprocessing (e.g., increasing feature dimensions) to alleviate the data processing pressure on the cloud. The cloud itself may suspend non-urgent batch model training tasks. Adjustment instructions are sent to edge devices via lightweight communication protocols (e.g., CoAP or MQTT). The task scheduler built into the edge device dynamically loads algorithm modules of different complexities according to the instructions, achieving elastic scaling of computing tasks.

[0056] In some embodiments, the real-time water quality monitoring data is obtained by edge device after feature extraction and real-time optimization of raw data collected by multiple water quality sensors. This includes: the edge device performing time-series segmentation of the raw data using a sliding window algorithm, calculating time-domain statistical features, frequency-domain energy features, and inter-sensor correlation features for each segment, using an adaptive Kalman filter algorithm to suppress noise in the raw data of each sensor, and dynamically adjusting the filter gain parameters by estimating the sensor measurement noise covariance matrix in real time to obtain optimized real-time water quality monitoring data.

[0057] Specifically, this embodiment refines the data processing flow of the edge device, improving the quality and feature representativeness of real-time data through sliding window and adaptive filtering techniques. This includes extracting multi-dimensional features using time-series segmentation and dynamically optimizing the data using adaptive Kalman filtering.

[0058] Temporal segmentation and feature extraction are performed by using a fixed-length sliding window (e.g., containing the most recent 100 sampling points) on an edge device to segment the raw sensor data.

[0059] For the data within each window, three types of features are calculated: time-domain statistical features, such as mean, variance, maximum, and minimum values; frequency-domain energy features, calculated using Fast Fourier Transform (FFT) to determine the energy distribution of the main frequency bands; and inter-sensor correlation features, calculating the correlation coefficients between different sensor data (such as pH and dissolved oxygen) within the current window.

[0060] Adaptive Kalman filtering (AKL) runs the algorithm on an edge device. Its core is the real-time estimation of the covariance matrix R of the measurement noise. When sensor data is stable, R is small, resulting in a low filter gain and a filter result that is more reliable than the system model's prediction, leading to good smoothing. When a sudden data change is detected (potentially a useful signal rather than noise), the algorithm automatically increases the estimated value of R, thereby increasing the filter gain and allowing the filter result to track the actual measurement value more quickly, reducing lag. In this way, the filter gain parameter is no longer fixed but dynamically adjusted according to data characteristics, achieving a balance between noise suppression and signal tracking.

[0061] In some embodiments, associating the real-time water quality monitoring data with historical sampling data in a pre-established water quality sample library includes: constructing a spatial distance index based on the geographical coordinates of the monitoring point; performing preliminary screening of the real-time data with historical data from the same monitoring point or within a spatial distance threshold in the water quality sample library; extracting historical data within a preset time window based on the data collection timestamp; and performing multi-dimensional matching through water quality parameter types and data quality identifiers to form spatiotemporal parameter matching associated data pairs; the water quality parameter types include at least pH value, dissolved oxygen, and conductivity.

[0062] This embodiment details how to accurately and efficiently correlate real-time data with a historical sample database. By constructing multi-dimensional matching rules based on space, time, parameter type, and quality identifiers, it ensures that the correlated data pairs have high comparability and reference value.

[0063] Spatial filtering is based on the latitude and longitude coordinates of monitoring points, constructing a spatial index (such as GeoHash or R-tree). Real-time data is first correlated with historical data of the same monitoring point in the sample library. Secondly, historical data from other monitoring points within a spatial distance of a threshold (e.g., within 1 kilometer) can be correlated to account for regional pollution spread.

[0064] The time window filtering method uses the timestamp of real-time data to further extract historical data from the spatially filtered data, where the difference between the collection time and the current timestamp is within a preset time window (e.g., ±15 days). This helps to capture seasonal and periodic water quality change patterns.

[0065] Finally, a fine-grained matching process is performed. This ensures that the associated historical and real-time data share the same water quality parameter types (e.g., both are pH values). Simultaneously, historical data marked as "high-quality" or "verified" is prioritized, while obviously abnormal or unreliable historical data is excluded. This results in individual "real-time data-historical data" pairs for subsequent model training.

[0066] In some embodiments, the step of fitting and training the correlated real-time water quality monitoring data with historical sampling data in the water quality sample library to obtain a water quality prediction model includes: using a time-series decomposition algorithm to decompose the correlated data pairs into trend components, periodic components, and random components; constructing a time-series prediction model based on a long short-term memory network, using the decomposed components as input features, and learning the temporal evolution law of water quality parameters through end-to-end training; and using an attention mechanism to automatically weight the influence of historical data at different time steps on the current prediction, thereby optimizing the model's response capability to abnormal data.

[0067] Specifically, this embodiment improves the accuracy and robustness of the water quality prediction model. By decomposing the data into different components and utilizing an attention mechanism, the model can better learn complex water quality time-series patterns and respond sensitively to abnormal events.

[0068] The time-series decomposition employs algorithms such as STL (Seasonal-Trend decomposition using Loess) to decompose the correlated time-series data into three parts: Trend component: reflecting the long-term direction of water quality changes (e.g., a trend of deterioration year by year); Periodic component: reflecting seasonal and diurnal periodic changes (e.g., decreased dissolved oxygen in summer); Random component: containing noise and sudden events.

[0069] Model construction and training utilize a Long Short-Term Memory (LSTM) network as the core prediction model. The decomposed trend, periodic, and random components are fed as independent input feature sequences into the LSTM model. This decomposition makes it easier for the model to learn patterns at different time scales.

[0070] An attention mechanism is introduced on top of LSTM. This mechanism automatically calculates the importance weight of each time point in the historical sequence for predicting the current water quality. For example, when a sudden pollution event occurs, the attention mechanism assigns higher weight to historical data closer to the time of the pollution event, while reducing the weight of distant historical data, thus making the model prediction more closely reflect recent changes and responding quickly to anomalies.

[0071] In some embodiments, calculating the difference between the output of the water quality prediction model and the corresponding real-time water quality monitoring data includes: setting differentiated loss function weights for different water quality parameters based on their monitoring accuracy requirements; calculating the weighted root mean square error between the predicted and measured values, wherein the error weights corresponding to parameters with pollution levels exceeding a threshold are automatically amplified; and calculating the dynamic error mean through a sliding time window to eliminate the interference of random noise on the difference calculation.

[0072] Specifically, this embodiment optimizes the calculation method for predicted discrepancies to better align with actual monitoring needs. By introducing differentiated weights and moving averages, the discrepancy calculation better reflects the deviations of key parameters and reduces interference from random fluctuations.

[0073] By setting differentiated weights, different loss function weights are assigned to different water quality parameters before calculating the difference between predicted and measured values. For example, higher weights are given to critical pollutant indicators (such as total nitrogen and total phosphorus) or parameters that have exceeded safe thresholds (such as pH values ​​below 6.0). In this way, the prediction deviation of critical parameters will have a greater impact on the overall difference, driving the system to prioritize the optimization of predictions for these parameters.

[0074] Calculating the weighted root mean square error: The weighted root mean square error is used as the difference measure. The formula can be simplified to: Difference = sqrt(Σ(weight_i * (predicted value_i - measured value_i)^2)). This ensures that the error of high-weight parameters is amplified. To avoid the random noise of a single measurement from having an excessive impact on the difference calculation, a sliding time window (e.g., the most recent 10 calculations) is used. The mean of the dynamic error within the window is calculated, and this smoothed mean is used as the final difference evaluation result, improving the stability of the system.

[0075] In some embodiments, determining the parameter adjustment amount of the data processing model corresponding to the edge device based on the difference includes: calculating the gradient value of the difference with respect to the edge model parameters using the backpropagation algorithm; updating the gain coefficients of the state transition matrix and observation matrix using the gradient descent method for Kalman filter parameters; adjusting only the model weight parameters that contribute significantly to the current error using sparse optimization techniques for lightweight machine learning algorithms; and attenuating the gradient value using the exponential smoothing method to suppress oscillations during parameter adjustment.

[0076] This embodiment provides a precise and efficient method for generating parameter adjustment values. Through gradient calculation and sparse optimization, it ensures that parameter adjustments are targeted and stable, avoiding system oscillations caused by over-adjustment.

[0077] Gradient calculation treats the cloud-based prediction model and the edge data processing model as a differentiable, integrated computational graph. Using the backpropagation algorithm, the gradient of the final prediction error with respect to edge model parameters (such as the Kalman filter gain coefficients and machine learning model weights) is calculated. The gradient value indicates the direction and magnitude of parameter adjustments.

[0078] Parameter updates include: Kalman filter parameters: based on the calculated gradient, the gain coefficients in the state transition matrix or observation matrix are fine-tuned using gradient descent.

[0079] The weights of the machine learning model employ sparse optimization techniques (such as L1 regularization). During backpropagation updates, only those weights that contribute significantly to the current prediction error are significantly adjusted, while weights that contribute little are slightly adjusted or remain unchanged. This helps maintain the model's generalization ability and prevents overfitting.

[0080] Smoothing involves exponentially smoothing the calculated gradient values. This means the parameter adjustment amount depends not only on the currently calculated gradient but also on the smoothed values ​​of historical adjustments. This effectively suppresses drastic fluctuations (oscillations) during parameter adjustment, making the optimization process smoother.

[0081] In some embodiments, the edge device updates the parameters of the corresponding data processing model according to the parameter adjustment amount to achieve dynamic optimization of the edge data processing model, including: after receiving the parameter adjustment amount, the edge device first verifies the adjustment amount; if the verification fails, the historical parameters are used; an incremental update strategy is adopted to adjust the iterative parameters of the Kalman filter and the weight matrix of the machine learning model in batches; after the update is completed, sensor data for a period of time is collected for model verification; if the verification fails, a parameter rollback mechanism is triggered.

[0082] This embodiment focuses on the security and reliability of edge parameter updates. By introducing verification, incremental updates, and rollback mechanisms, it ensures that parameter adjustments will not cause malfunctions in edge devices, thus guaranteeing the continuity of monitoring work.

[0083] After receiving the parameter adjustment amount from the cloud via the edge device, the parameter validation process first performs a validity check, such as checking whether the parameter value is within the preset safety range (e.g., whether the filter gain is positive and less than 1) and whether the data format is correct. If the validation fails, an error log is recorded and the update is abandoned, continuing to use the original parameters.

[0084] Incremental updates employ a "small steps, quick iterations" strategy, breaking down large parameter adjustments into multiple small batches of incremental adjustments. For example, the weight matrix of a machine learning model is adjusted in five separate steps, with each adjustment involving only 20%. This reduces the impact of a single update on the system.

[0085] After the parameter update is completed, the edge device does not immediately use it for formal monitoring. Instead, it enters a short verification period (e.g., 5 minutes) to process the newly acquired sensor data using the updated model and calculate the confidence level of the output results or the difference from simple statistical values. If the verification fails (e.g., the confidence level is too low or obvious anomalies occur), the rollback mechanism is automatically triggered, restoring the model parameters to the state before the update and reporting an alarm to the cloud.

[0086] In some embodiments, the method further includes: deploying a reinforcement learning agent in a cloud server, wherein the agent uses the reduction rate of prediction error after edge model parameter adjustment as a reward signal to automatically optimize parameter adjustment strategies through a deep reinforcement learning algorithm; the reinforcement learning agent dynamically generates differentiated parameter adjustment rules for different water quality parameters and different edge device computing power states to form an adaptive parameter adjustment strategy library; when a sudden change pattern is detected in the water quality data, the agent automatically switches to an emergency adjustment strategy, increasing the adjustment step size of the model parameters and increasing the adjustment frequency.

[0087] This embodiment introduces reinforcement learning, enabling the entire system's parameter adjustment strategy to have the ability to learn and optimize itself, thus adapting to various complex water conditions and equipment states and achieving intelligent operation and maintenance.

[0088] Agent deployment involves deploying a reinforcement learning agent (such as using the DDPG or PPO algorithm) on a cloud server.

[0089] The learning process includes: State: The state observed by the agent includes edge device computing power, real-time water quality data characteristics, and historical prediction errors. Action: The action generated by the agent is the parameter adjustment command sent to the edge device (such as adjusting step size or frequency). Reward: The reward signal is based on the rate of reduction in prediction error after parameter adjustment. If the error decreases significantly after adjustment, a positive reward is given; otherwise, a negative reward is given.

[0090] The strategy library is formed by the agent learning the optimal parameter adjustment rules for different scenarios (such as normal periods, flood season, and when a certain type of pollution event occurs) and different edge device states through continuous interaction with the environment (i.e., the actual water quality monitoring system), thus forming an adaptive strategy library.

[0091] The emergency response system is trained by an agent to identify sudden patterns in water quality. Once such a pattern is detected (e.g., multiple indicators fluctuating drastically at the same time), it automatically calls upon or generates an "emergency adjustment strategy" from the strategy library. This strategy is characterized by increasing the step size of parameter adjustments and increasing the frequency of issuance to quickly respond to emergencies and shorten the system's convergence time.

[0092] Please see Figure 2 As shown, Figure 2This is a schematic diagram of the structure of a cloud- and edge-based collaborative water quality monitoring data dynamic optimization device 200 provided in this application embodiment. The cloud- and edge-based collaborative water quality monitoring data dynamic optimization device 200 is used to execute the steps of the cloud- and edge-based collaborative water quality monitoring data dynamic optimization method shown in the above embodiments. The cloud- and edge-based collaborative water quality monitoring data dynamic optimization device 200 can be a single server or a server cluster, or it can be a terminal, such as a handheld terminal, laptop computer, wearable device, or robot.

[0093] like Figure 2 As shown, the water quality monitoring data dynamic optimization device 200 based on cloud and edge collaboration includes: The data receiving unit 201 is used to receive real-time water quality monitoring data sent by multiple edge devices. The real-time water quality monitoring data is obtained by the edge devices after feature extraction and real-time optimization of the raw data collected by multiple water quality sensors. The real-time water quality monitoring data is associated with historical sampling data in a pre-established water quality sample library. The water quality sample library contains historical water quality data at different times and different monitoring points, as well as corresponding water quality category or pollution degree label information. The fitting training unit 202 is used to perform fitting training on the correlated real-time water quality monitoring data and historical sampling data in the water quality sample library to obtain a water quality prediction model. The fitting training is based on the feature correlation and label consistency between real-time data and historical data. The unit calculates the difference between the output of the water quality prediction model and the corresponding real-time water quality monitoring data, and determines the parameter adjustment amount of the data processing model corresponding to the edge device based on the difference. The parameter adjustment amount includes the filter gain parameter of Kalman filtering and the model weight parameter of lightweight machine learning algorithm. The adjustment sending unit 203 is used to send the parameter adjustment amount to the corresponding edge device, so that the edge device updates the parameters of the corresponding data processing model according to the parameter adjustment amount, thereby realizing the dynamic optimization of the edge data processing model.

[0094] In some embodiments, the method further includes: during the process of receiving real-time data, associating historical data, fitting and training models, calculating parameter adjustments, and sending parameter adjustments, allocating computing tasks between the cloud server and the edge device through a preset computing power balancing and coordination strategy. The computing power balancing and coordination strategy dynamically adjusts the complexity of data processing on the edge device and the frequency of model training on the cloud server based on the real-time computing power status of the edge device and the load status of the cloud server. The real-time computing power status includes CPU utilization and memory usage, the load status includes the number of currently processed tasks and network bandwidth usage, and the complexity includes the feature extraction dimension of the lightweight machine learning algorithm and the number of Kalman filter iterations.

[0095] In some embodiments, the real-time water quality monitoring data is obtained by edge device after feature extraction and real-time optimization of raw data collected by multiple water quality sensors. This includes: the edge device performing time-series segmentation of the raw data using a sliding window algorithm, calculating time-domain statistical features, frequency-domain energy features, and inter-sensor correlation features for each segment, using an adaptive Kalman filter algorithm to suppress noise in the raw data of each sensor, and dynamically adjusting the filter gain parameters by estimating the sensor measurement noise covariance matrix in real time to obtain optimized real-time water quality monitoring data.

[0096] In some embodiments, associating the real-time water quality monitoring data with historical sampling data in a pre-established water quality sample library includes: constructing a spatial distance index based on the geographical coordinates of the monitoring point; performing preliminary screening of the real-time data with historical data from the same monitoring point or within a spatial distance threshold in the water quality sample library; extracting historical data within a preset time window based on the data collection timestamp; and performing multi-dimensional matching through water quality parameter types and data quality identifiers to form spatiotemporal parameter matching associated data pairs; the water quality parameter types include at least pH value, dissolved oxygen, and conductivity.

[0097] In some embodiments, the step of fitting and training the correlated real-time water quality monitoring data with historical sampling data in the water quality sample library to obtain a water quality prediction model includes: using a time-series decomposition algorithm to decompose the correlated data pairs into trend components, periodic components, and random components; constructing a time-series prediction model based on a long short-term memory network, using the decomposed components as input features, and learning the temporal evolution law of water quality parameters through end-to-end training; and using an attention mechanism to automatically weight the influence of historical data at different time steps on the current prediction, thereby optimizing the model's response capability to abnormal data.

[0098] In some embodiments, calculating the difference between the output of the water quality prediction model and the corresponding real-time water quality monitoring data includes: setting differentiated loss function weights for different water quality parameters based on their monitoring accuracy requirements; calculating the weighted root mean square error between the predicted and measured values, wherein the error weights corresponding to parameters with pollution levels exceeding a threshold are automatically amplified; and calculating the dynamic error mean through a sliding time window to eliminate the interference of random noise on the difference calculation.

[0099] In some embodiments, determining the parameter adjustment amount of the data processing model corresponding to the edge device based on the difference includes: calculating the gradient value of the difference with respect to the edge model parameters using the backpropagation algorithm; updating the gain coefficients of the state transition matrix and observation matrix using the gradient descent method for Kalman filter parameters; adjusting only the model weight parameters that contribute significantly to the current error using sparse optimization techniques for lightweight machine learning algorithms; and attenuating the gradient value using the exponential smoothing method to suppress oscillations during parameter adjustment.

[0100] In some embodiments, the edge device updates the parameters of the corresponding data processing model according to the parameter adjustment amount to achieve dynamic optimization of the edge data processing model, including: after receiving the parameter adjustment amount, the edge device first verifies the adjustment amount; if the verification fails, the historical parameters are used; an incremental update strategy is adopted to adjust the iterative parameters of the Kalman filter and the weight matrix of the machine learning model in batches; after the update is completed, sensor data for a period of time is collected for model verification; if the verification fails, a parameter rollback mechanism is triggered.

[0101] In some embodiments, the method further includes: deploying a reinforcement learning agent in a cloud server, wherein the agent uses the reduction rate of prediction error after edge model parameter adjustment as a reward signal to automatically optimize parameter adjustment strategies through a deep reinforcement learning algorithm; the reinforcement learning agent dynamically generates differentiated parameter adjustment rules for different water quality parameters and different edge device computing power states to form an adaptive parameter adjustment strategy library; when a sudden change pattern is detected in the water quality data, the agent automatically switches to an emergency adjustment strategy, increasing the adjustment step size of the model parameters and increasing the adjustment frequency.

[0102] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the cloud-based and edge-based collaborative water quality monitoring data dynamic optimization device and its modules described above can be referred to the corresponding processes in the embodiments of the cloud-based and edge-based collaborative water quality monitoring data dynamic optimization method described above, and will not be repeated here.

[0103] The aforementioned method for dynamically optimizing water quality monitoring data based on cloud and edge collaboration can be implemented as a computer program, which can be used in various ways, such as... Figure 2 It runs on the device shown.

[0104] Please see Figure 3 , Figure 3 This is a schematic block diagram of the structure of a cloud server provided in an embodiment of this application. The cloud server includes a processor, a memory, and a network interface connected via a device bus, wherein the memory may include a storage medium and internal memory.

[0105] The storage medium can store operating devices and computer programs. The computer program includes program instructions that, when executed, cause the processor to perform any dynamic optimization method based on cloud-edge collaborative water quality monitoring data.

[0106] The processor provides computing and control capabilities to support the operation of the entire cloud server.

[0107] The internal memory provides an environment for the execution of computer programs in non-volatile storage media. When the computer program is executed by the processor, it enables the processor to execute any dynamic optimization method for water quality monitoring data based on cloud and edge collaboration.

[0108] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the solution of this application and does not constitute a limitation on the terminal on which the solution of this application is applied. A specific cloud server may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0109] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0110] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps: The system receives real-time water quality monitoring data from multiple edge devices. The real-time water quality monitoring data is obtained by the edge devices after feature extraction and real-time optimization of the raw data collected by multiple water quality sensors. The system associates the real-time water quality monitoring data with historical sampling data in a pre-established water quality sample library. The water quality sample library contains historical water quality data from different times and different monitoring points, as well as corresponding water quality category or pollution level label information. A water quality prediction model is obtained by fitting and training the correlated real-time water quality monitoring data with historical sampling data in the water quality sample library. The fitting and training is based on the feature correlation and label consistency between the real-time data and the historical data. The difference between the output of the water quality prediction model and the corresponding real-time water quality monitoring data is calculated. Based on the difference, the parameter adjustment amount of the data processing model corresponding to the edge device is determined. The parameter adjustment amount includes the filter gain parameter of Kalman filter and the model weight parameter of lightweight machine learning algorithm. The parameter adjustment amount is sent to the corresponding edge device, so that the edge device updates the parameters of the corresponding data processing model according to the parameter adjustment amount, thereby realizing the dynamic optimization of the edge data processing model.

[0111] In some embodiments, the method further includes: during the process of receiving real-time data, associating historical data, fitting and training models, calculating parameter adjustments, and sending parameter adjustments, allocating computing tasks between the cloud server and the edge device through a preset computing power balancing and coordination strategy. The computing power balancing and coordination strategy dynamically adjusts the complexity of data processing on the edge device and the frequency of model training on the cloud server based on the real-time computing power status of the edge device and the load status of the cloud server. The real-time computing power status includes CPU utilization and memory usage, the load status includes the number of currently processed tasks and network bandwidth usage, and the complexity includes the feature extraction dimension of the lightweight machine learning algorithm and the number of Kalman filter iterations.

[0112] In some embodiments, the real-time water quality monitoring data is obtained by edge device after feature extraction and real-time optimization of raw data collected by multiple water quality sensors. This includes: the edge device performing time-series segmentation of the raw data using a sliding window algorithm, calculating time-domain statistical features, frequency-domain energy features, and inter-sensor correlation features for each segment, using an adaptive Kalman filter algorithm to suppress noise in the raw data of each sensor, and dynamically adjusting the filter gain parameters by estimating the sensor measurement noise covariance matrix in real time to obtain optimized real-time water quality monitoring data.

[0113] In some embodiments, associating the real-time water quality monitoring data with historical sampling data in a pre-established water quality sample library includes: constructing a spatial distance index based on the geographical coordinates of the monitoring point; performing preliminary screening of the real-time data with historical data from the same monitoring point or within a spatial distance threshold in the water quality sample library; extracting historical data within a preset time window based on the data collection timestamp; and performing multi-dimensional matching through water quality parameter types and data quality identifiers to form spatiotemporal parameter matching associated data pairs; the water quality parameter types include at least pH value, dissolved oxygen, and conductivity.

[0114] In some embodiments, the step of fitting and training the correlated real-time water quality monitoring data with historical sampling data in the water quality sample library to obtain a water quality prediction model includes: using a time-series decomposition algorithm to decompose the correlated data pairs into trend components, periodic components, and random components; constructing a time-series prediction model based on a long short-term memory network, using the decomposed components as input features, and learning the temporal evolution law of water quality parameters through end-to-end training; and using an attention mechanism to automatically weight the influence of historical data at different time steps on the current prediction, thereby optimizing the model's response capability to abnormal data.

[0115] In some embodiments, calculating the difference between the output of the water quality prediction model and the corresponding real-time water quality monitoring data includes: setting differentiated loss function weights for different water quality parameters based on their monitoring accuracy requirements; calculating the weighted root mean square error between the predicted and measured values, wherein the error weights corresponding to parameters with pollution levels exceeding a threshold are automatically amplified; and calculating the dynamic error mean through a sliding time window to eliminate the interference of random noise on the difference calculation.

[0116] In some embodiments, determining the parameter adjustment amount of the data processing model corresponding to the edge device based on the difference includes: calculating the gradient value of the difference with respect to the edge model parameters using the backpropagation algorithm; updating the gain coefficients of the state transition matrix and observation matrix using the gradient descent method for Kalman filter parameters; adjusting only the model weight parameters that contribute significantly to the current error using sparse optimization techniques for lightweight machine learning algorithms; and attenuating the gradient value using the exponential smoothing method to suppress oscillations during parameter adjustment.

[0117] In some embodiments, the edge device updates the parameters of the corresponding data processing model according to the parameter adjustment amount to achieve dynamic optimization of the edge data processing model, including: after receiving the parameter adjustment amount, the edge device first verifies the adjustment amount; if the verification fails, the historical parameters are used; an incremental update strategy is adopted to adjust the iterative parameters of the Kalman filter and the weight matrix of the machine learning model in batches; after the update is completed, sensor data for a period of time is collected for model verification; if the verification fails, a parameter rollback mechanism is triggered.

[0118] In some embodiments, the method further includes: deploying a reinforcement learning agent in a cloud server, wherein the agent uses the reduction rate of prediction error after edge model parameter adjustment as a reward signal to automatically optimize parameter adjustment strategies through a deep reinforcement learning algorithm; the reinforcement learning agent dynamically generates differentiated parameter adjustment rules for different water quality parameters and different edge device computing power states to form an adaptive parameter adjustment strategy library; when a sudden change pattern is detected in the water quality data, the agent automatically switches to an emergency adjustment strategy, increasing the adjustment step size of the model parameters and increasing the adjustment frequency.

[0119] The embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions, and the processor executing the program instructions to implement the steps of the dynamic optimization method for water quality monitoring data based on cloud and edge collaboration provided in the above embodiments of this application.

[0120] The computer-readable storage medium can be an internal storage unit of the cloud server described in the foregoing embodiments, such as the hard drive or memory of the cloud server. Alternatively, the computer-readable storage medium can be an external storage device of the cloud server, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the cloud server.

[0121] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A dynamic optimization method for water quality monitoring data based on cloud and edge collaboration, applied to a cloud server, wherein the cloud server is communicatively connected to multiple edge devices, and each edge device is equipped with multiple water quality sensors and a data processing model, characterized in that... The method includes: The system receives real-time water quality monitoring data from multiple edge devices. The real-time water quality monitoring data is obtained by the edge devices after feature extraction and real-time optimization of the raw data collected by multiple water quality sensors. The system associates the real-time water quality monitoring data with historical sampling data in a pre-established water quality sample library. The water quality sample library contains historical water quality data from different times and different monitoring points, as well as corresponding water quality category or pollution level label information. A water quality prediction model is obtained by fitting and training the correlated real-time water quality monitoring data with historical sampling data in the water quality sample library. The fitting and training is based on the feature correlation and label consistency between the real-time data and the historical data. The difference between the output of the water quality prediction model and the corresponding real-time water quality monitoring data is calculated. Based on the difference, the parameter adjustment amount of the data processing model corresponding to the edge device is determined. The parameter adjustment amount includes the filter gain parameter of Kalman filter and the model weight parameter of lightweight machine learning algorithm. The parameter adjustment amount is sent to the corresponding edge device, so that the edge device updates the parameters of the corresponding data processing model according to the parameter adjustment amount, thereby realizing the dynamic optimization of the edge data processing model.

2. The method according to claim 1, characterized in that, The method further includes: In the process of receiving real-time data, associating historical data, fitting and training models, calculating parameter adjustment amounts, and sending parameter adjustment amounts, the computing tasks of the cloud server and edge devices are allocated through a preset computing power balancing and coordination strategy. The computing power balancing and coordination strategy dynamically adjusts the complexity of data processing on the edge devices and the frequency of model training on the cloud server based on the real-time computing power status of the edge devices and the load of the cloud server. The real-time computing power status includes CPU utilization and memory usage; the load status includes the number of currently processed tasks and network bandwidth usage; and the complexity includes the feature extraction dimension of the lightweight machine learning algorithm and the number of Kalman filter iterations.

3. The method according to claim 1, characterized in that, The real-time water quality monitoring data is obtained by edge devices through feature extraction and real-time optimization of raw data collected by multiple water quality sensors, including: The edge device uses a sliding window algorithm to segment the raw data in time series. For each segment, it calculates the time-domain statistical characteristics, frequency-domain energy characteristics, and correlation characteristics between sensors. An adaptive Kalman filter algorithm is used to suppress noise in the raw data of each sensor. By estimating the sensor measurement noise covariance matrix in real time, the filter gain parameters are dynamically adjusted to obtain optimized real-time water quality monitoring data.

4. The method according to claim 1, characterized in that, The step of associating the real-time water quality monitoring data with historical sampling data in a pre-established water quality sample library includes: A spatial distance index is constructed based on the geographical coordinates of the monitoring points, and real-time data is initially screened with historical data from the same monitoring points or within the spatial distance threshold in the water quality sample bank. Based on the data collection timestamp, extract historical data that is within a preset time window and the time interval between the real-time data and the data collection timestamp. Multidimensional matching is performed using water quality parameter types and data quality identifiers to form spatiotemporal parameter matching associated data pairs; Water quality parameters include at least pH, dissolved oxygen, and conductivity.

5. The method according to claim 1, characterized in that, The process of fitting and training the correlated real-time water quality monitoring data with historical sampling data in the water quality sample library to obtain a water quality prediction model includes: The time-series decomposition algorithm is used to decompose the correlated data pairs into trend components, periodic components, and random components; A time series prediction model is constructed based on a long short-term memory network. The decomposed components are used as input features, and the time series evolution of water quality parameters is learned through end-to-end training. By using an attention mechanism to automatically weight the impact of historical data at different time steps on the current prediction, the model's ability to respond to abnormal data is optimized.

6. The method according to claim 1, characterized in that, The calculation of the difference between the output of the water quality prediction model and the corresponding real-time water quality monitoring data includes: Differentiated loss function weights are set to meet the monitoring accuracy requirements of different water quality parameters; Calculate the weighted root mean square error between the predicted and measured values, where the error weights of parameters whose contamination levels exceed the threshold are automatically amplified. The mean dynamic error is calculated by using a sliding time window to eliminate the interference of random noise on the difference calculation.

7. The method according to claim 1, characterized in that, The step of determining the parameter adjustment amount of the data processing model corresponding to the edge device based on the difference includes: The gradient values ​​of the difference pair edge model parameters are calculated using the backpropagation algorithm. For the Kalman filter parameters, the gain coefficients of the state transition matrix and the observation matrix are updated using the gradient descent method. For lightweight machine learning algorithms, sparse optimization techniques are used to adjust only the model weight parameters that contribute significantly to the current error. The gradient value is decayed by exponential smoothing to suppress oscillations during parameter adjustment.

8. The method according to claim 1, characterized in that, The edge device updates the parameters of the corresponding data processing model according to the parameter adjustment amount, thereby realizing dynamic optimization of the edge data processing model, including: After receiving the parameter adjustment, the edge device first verifies the adjustment. If the verification fails, the historical parameters are used. An incremental update strategy is adopted to adjust the iteration parameters of the Kalman filter and the weight matrix of the machine learning model in batches; After the update is completed, sensor data for a period of time is collected for model verification. If the verification fails, the parameter rollback mechanism is triggered.

9. The method according to claim 1, characterized in that, The method further includes: A reinforcement learning agent is deployed in a cloud server. The agent uses the reduction rate of prediction error after edge model parameter adjustment as a reward signal and automatically optimizes the parameter adjustment strategy through a deep reinforcement learning algorithm. The reinforcement learning agent dynamically generates differentiated parameter adjustment rules for different water quality parameters and different edge device computing power states, forming an adaptive parameter adjustment strategy library. When a sudden change in water quality data is detected, the agent automatically switches to an emergency adjustment strategy, increasing the adjustment step size of model parameters and increasing the adjustment frequency.

10. A dynamic optimization device for water quality monitoring data based on cloud and edge collaboration, applied to a cloud server, wherein the cloud server is communicatively connected to multiple edge devices, and each edge device is equipped with multiple water quality sensors and a data processing model, characterized in that... The apparatus is used to implement the method as described in any one of claims 1 to 9, the apparatus comprising: The data receiving unit is used to receive real-time water quality monitoring data sent by multiple edge devices. The real-time water quality monitoring data is obtained by the edge devices after feature extraction and real-time optimization of the raw data collected by multiple water quality sensors. The real-time water quality monitoring data is associated with historical sampling data in a pre-established water quality sample library. The water quality sample library contains historical water quality data at different times and different monitoring points, as well as corresponding water quality category or pollution degree label information. The fitting training unit is used to fit and train the correlated real-time water quality monitoring data with historical sampling data in the water quality sample library to obtain a water quality prediction model. The fitting training is based on the feature correlation and label consistency between real-time data and historical data. The unit calculates the difference between the output of the water quality prediction model and the corresponding real-time water quality monitoring data, and determines the parameter adjustment amount of the data processing model corresponding to the edge device based on the difference. The parameter adjustment amount includes the filter gain parameter of Kalman filtering and the model weight parameter of lightweight machine learning algorithm. The adjustment sending unit is used to send the parameter adjustment amount to the corresponding edge device, so that the edge device updates the parameters of the corresponding data processing model according to the parameter adjustment amount, thereby realizing the dynamic optimization of the edge data processing model.

Images