Eutrophication model self-update and iteration method and system based on machine learning

Abstract

The present invention provides a self-update and iteration method and system for eutrophication models based on machine learning. Through a full-process automation mechanism of automated data processing, seamless data transmission, automatic file generation and update, autonomous calculation and feedback, and parameter optimization and model iteration, the method and system can automatically obtain and format required data from local databases and external API interfaces, and use PHP edge computing methods to securely and reliably send data streams to the Python environment of the server. The MIKEIO library is used to generate or update terrain files, boundary condition input files, and initial field distribution files, and the model sandbox area instance is started through MzLaunch to feed back model results to the front end in real time. The water quality process parameters are initially calibrated and later optimized based on the sklearn machine learning library to realize the cyclic simulation and self-update of the model, thereby greatly improving the simulation accuracy and response speed. The system is suitable for water environment management scenarios that require real-time warning and prediction, such as red tides in seawater.

Description

Technical Field

[0001] The present invention belongs to the field of water environment technology, and in particular relates to a self-updating and iterative method and system for a eutrophication model based on machine learning. Background Art

[0002] In the field of numerical simulation of water environments, traditional modeling processes often involve tedious manual operations. Specifically, the preparation of initial conditions and input files, parameter calibration, and the extraction and comparative analysis of results all require extensive manual intervention. This manual process is not only time-consuming and labor-intensive, but also prone to human error, resulting in reduced model accuracy. Especially in the context of real-time early warning and prediction, manual operations cannot meet the requirements for rapid response, seriously affecting the practicality and reliability of the model.

[0003] For example, when using traditional hydrodynamic and water quality numerical simulation models like MIKE21, users must manually prepare terrain data, boundary condition input files, and initial field distribution files, and repeatedly adjust parameters and perform multiple simulations to ensure the accuracy of the model results. This process is not only complex and time-consuming, but also makes it difficult to dynamically adjust and continuously optimize the model in the face of constantly updating data sources. Summary of the Invention

[0004] The purpose of the present invention is to provide a self-update and iteration method and system for eutrophication models based on machine learning, which combines data acquisition from databases and API interfaces, data transmission bridges of PHP, file processing of the MIKEIO library in Python, and parameter optimization functions of the sklearn machine learning library, thereby realizing full-process automation from data acquisition to autonomous model update, so as to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above objectives, the present invention proposes a self-updating and iterative method for eutrophication model based on machine learning, comprising the following steps:

[0006] Obtain the data stream required by the water environment numerical simulation model from the local database and API interface, and send the data stream to the Python environment on the server through PHP;

[0007] Generate or update a terrain file, a boundary condition input file, and an initial field distribution file based on the data stream using the MIKEIO library in Python;

[0008] After completing the file preparation, a model calculation request is sent to the server and recorded in the task information file. The server-side self-made service monitors the task information file. Once a new calculation request is detected, MzLaunch is started to run the model sandbox area instance. The dfsu file in the running result is read using Python's MIKEIO library, and the simulation results are fed back to the front end in real time through PHP. The front-end system combines the simulation result data and the measured data in the database to generate the data set required for machine learning and feed it into the Python environment of sklearn on the server. The calibration verification effect and the water quality process parameters are adjusted through the sklearn machine learning library, and the data is passed to the MIKEIO to update the configuration file.

[0009] Based on machine learning methods, the water environment numerical simulation model is cyclically optimized to achieve self-update and iteration of the water environment numerical simulation model.

[0010] Preferably, the data stream required for the water environment numerical simulation model is obtained from the local database and API interface, including: the front end accesses the local database to extract pre-constructed meteorological data, hydrodynamic data, water quality data, terrain data and setting data; based on the extracted data, the data is formatted using the front-end interpretation code and sent to the back end in JSON format; the system calls the external data API interface from the front end to obtain the latest meteorological data and water quality monitoring data real-time information, and merges it with the formatted data into a unified data stream.

[0011] Preferably, the method of sending the data stream to the Python environment of the server through PHP includes: using Internet information services to open the door of the back-end server, and using PHP as a bridge for data transmission to receive the unified data stream from the front-end; in the PHP hub code, calling the popen process function to pass the unified data stream to the Python process.

[0012] Preferably, the use of the MIKEIO library in Python to generate or update the terrain file, boundary condition input file, and initial field distribution file according to the data stream includes: after the Python process receives the data stream, it starts to parse the data stream; based on the parsed data stream, the MIKEIO library is used to generate or update the terrain file, boundary condition input file, and initial field distribution file, ensuring that all files meet the requirements of the MIKE21 model and are stored in a preset directory; recording the status of file generation or update, and confirming that the file preparation is complete.

[0013] Preferably, after the file preparation is completed, a model calculation request is sent to the server and recorded in the task information file, including: based on the status of confirming that the file preparation is completed, a model calculation request is sent to the server; the model calculation request is recorded in the task information file, and the task information file contains all necessary parameters and configuration information for this calculation.

[0014] Preferably, the server-side self-made service monitors the task information file, and once a new computing request is detected, MzLaunch is started to run the model sandbox area instance, including: MzLaunch runs the MIKE21 model in the model sandbox area instance according to the parameters and configuration information in the task information file, and records the model running status to ensure that the model is executed as expected.

[0015] Preferably, the MIKEIO library of Python is used to read the dfsu file in the running results, and the simulation results are fed back to the front end in real time through PHP, including: parsing the dfsu file, extracting relevant data and formatting it; and feeding back the formatted data to the front end in real time through PHP for users to view and analyze.

[0016] Preferably, the front-end system combines the simulation result data and the measured data in the database to generate the data set required for machine learning and feeds it into the Python environment of sklearn on the server side. The calibration verification effect and the water quality process parameters are judged and adjusted through the sklearn machine learning library, and passed to the MIKEIO to update the configuration file, including: based on the extracted relevant data, using the ParameterGrid of the sklearn machine learning library to generate parameter combinations, and separating a set of theoretically better parameter combinations through train_test_split; according to the optimization results, adjusting the water quality process parameters, and passing them to the MIKEIO library to update the configuration file.

[0017] Preferably, the water environment numerical simulation model is cyclically optimized based on the machine learning method to achieve self-update and iteration of the water environment numerical simulation model, including: after each calculation example is completed, the root mean square error and goodness of fit between the simulated value of the pollutant concentration output by the evaluation model and the actual measured value provided by the front end are evaluated; if the evaluation result does not meet the predetermined standard, the model process parameters are directed reselected through the ParameterGrid and train_test_split of the machine learning library, and updated to the model parameter configuration file through the MIKEIO package to enter a new round of simulation; when the evaluation result meets the predetermined standard, the final round of model parameter configuration file is adopted and the autonomous continuous simulation stage is entered.

[0018] On the other hand, the present invention proposes a self-updating and iterative system for eutrophication models based on machine learning, comprising:

[0019] Data acquisition module, used to obtain the data stream required by the water environment numerical simulation model from the database and API interface;

[0020] Data transmission module, used to send data streams to the Python environment on the server through PHP;

[0021] A file generation module, for generating or updating a terrain file, a boundary condition input file, and an initial field distribution file according to the data stream using the MIKEIO library in Python;

[0022] The task request module is used to send a model calculation request to the server after completing file preparation and record it in the task information file;

[0023] The calculation execution module is used to monitor the task information file and start MzLaunch to run the model sandbox area instance once a new calculation request is detected;

[0024] The result feedback module is used to use Python's MIKEIO library to read the dfsu file in the running results and provide real-time feedback to the front end through PHP;

[0025] A parameter optimization module is used to adjust water quality process parameters through the sklearn machine learning library and pass them to the MIKEIO to update the configuration file;

[0026] The self-update and iteration module is used to cyclically optimize the water environment numerical simulation model based on machine learning methods to achieve self-update and iteration of the water environment numerical simulation model.

[0027] Technical effects and advantages of the present invention: The self-updating and iterative method and system for eutrophication models based on machine learning proposed in the present invention have the following advantages over the prior art:

[0028] The present invention significantly reduces manual intervention through a full-process automation mechanism of automated data processing, seamless data transmission, automatic file generation and updating, autonomous calculation and feedback, and parameter optimization and model iteration. The system can automatically obtain and format the required data from the local database and external API interface, use PHP to send the data stream safely and reliably to the Python environment of the server, use the MIKEIO library to generate or update the terrain file, boundary condition input file and initial field distribution file, and start the model sandbox area instance through MzLaunch, and feed back the model results to the front end in real time. Based on the sklearn machine learning library, the water quality process parameters are initially calibrated and later optimized to achieve cyclic simulation and self-update of the model, thereby greatly improving the simulation accuracy and response speed, and is particularly suitable for water environment management scenarios that require real-time warning and prediction, such as red tides in seawater. This method not only simplifies the operating process, but also improves the accuracy and practicality of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Flowchart of the self-updating and iterative method of the eutrophication model based on machine learning of the present invention;

[0030] Figure 2 This is a flow chart of the monitoring mode of the present invention;

[0031] Figure 3 A flow chart showing the update rate of the model of the present invention;

[0032] Figure 4 A block diagram of the self-updating and iterative system of the eutrophication model based on machine learning of the present invention;

[0033] Figure 5 This is a diagram showing data acquired by the front end of the system of the present invention;

[0034] Figure 6 Generate a data display diagram for the server of the present invention;

[0035] Figure 7 This is a schematic diagram of the present invention monitoring a system simulation request and autonomously starting the MIKE21 model to perform simulation;

[0036] Figure 8 Return the simulation results of the present invention to the front-end for display;

[0037] Figure 9 This is a diagram of the model self-update parameter calibration cycle process of the present invention;

[0038] Figure 10 This is a diagram showing the early warning prediction effect of the iterative simulation cycle of the model of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0040] The present invention provides a self-update and iteration method for eutrophication models based on machine learning. Taking the MIKE21 model as an example, this method develops a B / S architecture system. The system calls data resources, returns data to Python on the server via PHP, completes input and output control through Python's MIKEIO package, and automatically generates the initial and input condition files required by MIKE21 and stores them in the model sandbox area.

[0041] Generally, the system directly calls existing data from the database, such as topographic and water depth mapping, fine-tuned grids, historical monitoring data, etc., and obtains the latest meteorological and water quality monitoring data from the API interface. After these data are formatted through the system's functional code, they are sent to the server backend in JSON format via PHP.

[0042] The PHP hub code on the server backend receives data streams from the system and calls Python through the popen process function, passing in data parameters. The Python process runs and parses the data stream provided by PHP, packaging the data into corresponding files according to the file interface provided by the MIKEIO package. For example, if the input condition file is a one-dimensional time series file, the time series data is packaged into a dfs0 file. If the initial condition file is a two-dimensional surface file, the spatial distribution data of the water body at the initial time is interpolated onto each grid and packaged into a dfsu file containing only a single time point.

[0043] For subsequent simulations, initial files can be created by directly accessing the spatial data at the last moment in the previous simulation result file using the MIKEIO package. Because Python has local access permissions, these prepared initial and input condition files can be written directly to the corresponding folders in the server's pre-set model sandbox area. Typically, the sandbox area is subdivided into case folders corresponding to different model application scenarios.

[0044] According to the communication path of B / S system, PHP, and back-end Python, model parameter configuration resources can be created or modified on the front end of the system, and file updates in the back-end model sandbox area can be completed through the PFS file interface; at the same time, task request information that requires simulation calculations is returned to the task preparation area on the server.

[0045] A real-time monitoring service is generated on the server side to monitor task requests in the task preparation area. MzLaunch and system commands are used to control the MIKE21 computation engine code and monitor the model's progress and status. After the model runs, the results are interpreted and fed back to the system frontend via MIKEIO for multivariate visualization and statistical analysis.

[0046] The entire life cycle of the above model is constructed. At the same time, the initial parameter adjustment and calibration of the model are automatically and iteratively completed based on the Python sklearn machine learning library. After achieving the verification effect, the latest data resources are obtained in real time through the front-end system, synchronized to the back-end according to the above path and drive the model. The results are returned to the front-end to realize the fully autonomous model update, normalized continuous simulation and all-weather servo warning prediction. The details are as follows:

[0047] like Figure 1 As shown, the self-updating and iterative method of the eutrophication model based on machine learning in this embodiment includes the following steps:

[0048] Step 1: Obtain the data stream required by the water environment numerical simulation model from the local database and API interface. The system builds the relevant database in advance and designs the data fields according to the API interface instructions so that the data can be directly stored in the database. Then, the data stream required by the water environment numerical simulation model is obtained from the database and API interface. Specifically, it includes:

[0049] The front end accesses the local database to extract pre-built meteorological data (such as temperature, humidity, wind speed, etc.), hydrodynamic data (such as temperature, humidity, wind speed, etc.), water quality data (such as dissolved oxygen, ammonia nitrogen concentration, etc.), terrain data (such as water depth, terrain elevation, etc.) and setting data (such as simulation time step, grid resolution, etc.); these data are the basic input for numerical simulation of water environment.

[0050] Based on the extracted data, front-end interpretation code formats the data and sends it to the back-end in JSON format. The extracted data requires certain processing before it can be used directly in subsequent model calculations. The front-end system uses specially designed interpretation code to format the data to ensure it conforms to the MIKE21 model requirements. For example, this involves standardizing data from different sources to the same units and format. The formatted data is then converted to JSON and sent to the back-end server via HTTP requests or other communication protocols.

[0051] like Figure 5As shown in the figure, the system calls an external data API from the front end to obtain the latest real-time information on meteorological and water quality monitoring data, and merges this with the formatted data into a unified data stream. In addition to the static data in the local database, the system also needs to obtain the latest dynamic data, such as the latest weather forecast and water quality monitoring data. To achieve this, the system calls an external data API from the front end to obtain this real-time updated information.

[0052] The latest data obtained also needs to be formatted by the front-end interpretation code to ensure that it is consistent with the data format extracted from the local database.

[0053] Ultimately, all formatted data (including data extracted from local databases and real-time data obtained from external API interfaces) is merged into a unified data stream. This data stream contains all the input information required to run the model, ensuring data consistency and integrity.

[0054] During the system design phase, it's necessary to build the relevant database in advance and design the data fields according to the API interface specifications. This ensures that data extracted from the self-designed database can be directly stored in the database. Whenever new data comes in through the API, the front-end interpretation code and back-end conversion code can be directly combined to generate the latest files required by the MIKE21 model.

[0055] Because the local database data format is fixed, the code for extracting data from the database and passing it to Python to generate the MIKEIO file is also fixed. This allows the system to automatically combine the front-end interpretation code and back-end conversion code to generate the latest file required by the MIKE21 model every time new data comes in from the API.

[0056] Step 2: Send the data stream to the Python environment on the server side through PHP, which specifically includes: using Internet Information Service IIS to open the door of the back-end server, and using PHP as a bridge for data transmission to receive the unified data stream from the front-end; in the PHP hub code, call the popen process function to pass the unified data stream to the Python process.

[0057] On the back-end server, you first need to configure Internet Information Services (IIS) to ensure that the server can receive HTTP requests from the front-end. IIS is a powerful web server platform that can handle various types of requests and provide corresponding responses.

[0058] The front-end system sends a formatted, unified data stream to the back-end server via HTTP requests. PHP, an open-source server-side scripting language, is used to process these requests and facilitate data transmission. The PHP script receives the data stream from the front-end and stores it in memory or a temporary file for subsequent processing.

[0059] The PHP hub code is the core of the entire data transmission process. It is responsible for not only receiving the data stream from the front end, but also passing this data to the background Python process for further processing.

[0060] Using the popen function, PHP can start a new process to execute external commands. During this process, the popen function is used to call a Python script and pass a unified data stream to the Python process via standard input (stdin). Specifically, PHP parses the received JSON-formatted data stream into a string and passes it to the Python script through a pipe.

[0061] Step 3: Use the MIKEIO library in Python to generate or update the terrain file, boundary condition input file, and initial field distribution file based on the data stream; specifically, the following steps are included:

[0062] After receiving the data stream, the Python process begins parsing it. The Python process receives the unified data stream from the frontend via standard input (stdin). This data stream is transmitted in JSON format. After receiving the data stream, it must first be parsed. You can use Python's built-in json module to convert a JSON string into a Python dictionary or other easily processable data structure.

[0063] like Figure 6 As shown in the figure, based on the parsed data stream, the MIKEIO library is used to generate or update terrain files, boundary condition input files, and initial field distribution files, ensuring that all files meet the requirements of the MIKE21 model and are stored in a preset directory. The parsed data includes meteorological data, hydrodynamic data, water quality data, terrain data, etc. This data needs to be further processed and converted into files in the specific format required by the MIKE21 model.

[0064] Using the MIKEIO library (such as mikeio), various file types can be generated or updated based on the parsed data, including terrain files (Mesh files), boundary condition input files (Dfs0 files), initial field distribution files (Dfsu files), etc.

[0065] Record the status of file generation or updates to confirm that file preparation is complete. During the file generation or update process, the system needs to record the status of each file to ensure that all necessary files have been correctly generated or updated. Logging mechanisms can be used to track the status of file generation or updates. For example, a log can be written after each file is generated or updated, recording information such as the file name, generation time, and status (success or failure).

[0066] Step 4: After completing the file preparation, send a model calculation request to the server and record it in the task information file; specifically, it includes:

[0067] Based on the status of the confirmation file preparation completion, a model calculation request is sent to the server; the model calculation request is recorded in the task information file, which contains all necessary parameters and configuration information for this calculation.

[0068] In the previous step, the system has generated or updated all necessary terrain files, boundary condition input files, initial field distribution files, and configuration files (PFS files). To ensure that these files have been correctly generated and stored in the preset directory, a status check mechanism can be added during the file generation process.

[0069] Once all files are confirmed to have been successfully generated or updated, the system can trigger a model calculation request, which is usually sent to a specific interface on the server via an HTTP POST request or other suitable communication protocol.

[0070] To facilitate the management and tracking of the status of each model calculation, the system needs to record the relevant information of each calculation request in a task information file. This file contains all the necessary parameters and configuration information for the calculation, such as the time step of the model run, the grid resolution, the input file path, etc.

[0071] The task information file is not only used to record the status of the current computing task, but also serves as the basis for subsequent monitoring and scheduling to ensure that the task can be executed as expected.

[0072] Step 5: The server-side self-made service monitors the task information file. Once a new computing request is detected, MzLaunch is started to run the model sandbox area instance. Specifically, it includes:

[0073] like Figure 7 As shown in the figure, MzLaunch runs the MIKE21 model in the model sandbox area instance according to the parameters and configuration information in the task information file, and records the model running status to ensure that the model executes as expected.

[0074] like Figure 2As shown, a batch script-based monitoring service was created in advance on the server through NSSM. The task information file is scanned every 10 seconds in the form of a process to see if the task information content has been updated. The details are as follows:

[0075] After downloading and deploying the nssm software, you need to create a bat command that scans the task information file every 10 seconds. The file system checks that if the model run results folder path in the task information file exists, the case has already been run and monitoring continues without performing any action. If the model run results folder path does not exist, the system calls the MzLaunch process and passes it the model case path. MIKE21 will start the case simulation and log the run. Since the model run results folder path has been automatically established at this point, nssm will enter the next cycle of monitoring mode without restarting the case. If there is no valid information in the task information file, nssm will exit the current monitoring cycle and enter the next one. This process has a comprehensive fault tolerance mechanism and retry strategy.

[0076] Step 6: Use Python's MIKEIO library to read the dfsu file in the run results and use PHP to feed the simulation results back to the front end in real time. This includes:

[0077] like Figure 8 As shown, the dfsu file is parsed, relevant data is extracted, and formatted. The formatted data is fed back to the front-end in real time via PHP for user viewing and analysis. The dfsu file in the run results is read using the Python MIKEIO library, and the run results are fed back to the system front-end in real time via PHP for data and progress display.

[0078] like Figure 3 As shown, when the calculation progress is completed, MzLaunch completes the life cycle and automatically closes the process. The system front end also obtains the complete data of this simulation. By extracting the information on the grid nodes corresponding to the monitoring point locations, the simulated values ​​of the pollutant concentrations are obtained to form a training set. The actual measured values ​​of the pollutant concentrations at the monitoring points are obtained through the system to form a test set. These machine learning data sets are fed with data and sent to the sklearn machine learning library in Python in the service backend through PHP. The LinearRegression regression model is used to judge the effect and accuracy of the simulation of different pollutants. If the effect is not achieved, the adjustment direction of the corresponding process parameters pointed to by the simulation results of different pollutants (too large? too small?) is determined. The water quality process parameter dictionary encapsulated by prior knowledge and with a quadratic reduction interval is passed to ParameterGrid in a targeted manner, and a set of theoretically better parameter combinations is separated through train_test_split to allow the model to continue the next round of simulation.

[0079] The LinearRegression model regresses the true value of a statistical variable using observed values ​​from a series of experiments. Linear regression is a supervised learning algorithm that describes the relationship between a continuous dependent variable and a series of independent variables. Using LinearRegression, we can obtain the mean square error (MSE) and coefficient of determination (R²) for both the validation set (simulated results) and the test set (measured water quality data). These are the two key metrics for evaluating whether a model has exceeded the loss function.

[0080] ParameterGrid is a tool for generating parameter combinations. It can be used to iterate over all possible combinations of values ​​for a given parameter. It is commonly used in machine learning for hyperparameter optimization and grid search. It can help generate all possible combinations of nearly 40 water quality process parameters of MIKE21, after reducing their value ranges based on prior conditions, for selection by train_test_split.

[0081] train_test_split is a separator function in sklearn.model_selection, which is used to divide an array or matrix into a training set and a test set. Here it is used to select a set of water quality process parameter combinations for the MIKE21 model to use in the next round of simulation.

[0082] Step 7: The front-end system combines the simulation result data with the measured data in the database to generate the data set required for machine learning and feeds it to the server-side sklearn Python environment. The sklearn machine learning library is used to determine the calibration verification effect and adjust the water quality process parameters, and the data is passed to the MIKEIO to update the configuration file. This specifically includes: adjusting the water quality process parameters through the sklearn machine learning library and passing them to the MIKEIO to update the configuration file, including: based on the extracted relevant data, using the sklearn machine learning library's ParameterGrid to generate parameter combinations, and separating a set of theoretically better parameter combinations through train_test_split; adjusting the water quality process parameters based on the optimization results, and passing them to the MIKEIO library to update the configuration file. The instance configuration file is updated directly through the interface, and the initial file and input condition file are not changed during the parameter calibration process.

[0083] Step 8: Circularly optimize the water environment numerical simulation model based on machine learning methods to achieve self-update and iteration of the water environment numerical simulation model. Iteration refers to the completion of model parameter calibration (to meet simulation accuracy requirements). This set of model parameters is first solidified and only the latest model input data is updated to allow the model to continuously simulate. Specifically, it includes:

[0084] After each calculation case is completed, the root mean square error and goodness of fit between the simulated pollutant concentration values ​​output by the model and the measured values ​​are evaluated; if the evaluation result does not meet the predetermined standard, the model process parameters are directed reselected through the ParameterGrid and train_test_split of the machine learning library, and updated to the model parameter configuration file through the MIKEIO package to enter a new round of simulation; when the evaluation result meets the predetermined standard, the final round of model parameter configuration file is adopted and the autonomous continuous simulation stage is entered.

[0085] The system receives the feedback command and repeats the requested action. The model completes a new round of simulation according to the previous process, and the simulation result data is fed back into sklearn. After multiple parameter selections and iterative simulations, when the RMSE (root mean square error) and R² (goodness of fit) of the simulated and measured pollutant values ​​meet the requirements, the machine learning process stops and the training results are saved for subsequent use in scenarios.

[0086] like Figure 9 As shown in the figure, it is an example of automatic calibration of the EcoLab module of MIKE 21 through machine learning. It starts with the default parameter group. After 5 rounds of iteration, the water quality process parameters are within a reasonable range, and the water quality simulation results are excellent, meeting the requirements of jumping out of the calibration cycle.

[0087] Next, models that have been trained and meet simulation accuracy requirements can enter a closed-loop process of autonomous continuous simulation and unsupervised early warning prediction. Once the model begins its autonomous simulation cycle, the system automatically and regularly retrieves the latest data resources through the API. When the dataset required for simulation or prediction updates is complete, the system automatically sends the dataset to the service backend, where it uses the Python MIKEIO library to incrementally update the required model files.

[0088] like Figure 10 As shown, using a marine model as an example, the real-time early warning and forecasting implemented by this invention can provide an early warning of the comprehensive trophic status index (TLI). If the TLI is greater than 50, a marine eutrophication warning is triggered on the system's main interface, and can be linked to SMS delivery and push notifications via the accompanying app. After the server-side model simulation lifecycle completes, the system attains a completed status and the process jumps to the trained model that meets simulation accuracy requirements. This allows for the closed-loop process of autonomous continuous simulation and unsupervised early warning prediction, entering a new autonomous simulation cycle. This completes the model's self-update and iterative method path.

[0089] Taking MIKE21 as an example, this method builds an implementation path for self-updating and iteration of the water environment numerical simulation model, solving the problem of complicated and time-consuming repetitive operations required during the model construction process. Combining the system and machine learning, it realizes a new usage scenario of model self-construction, autonomous calibration, and spontaneous forecasting, greatly saving the time cost of model construction, and realizing deep and strong coupling between the model and the system, as well as continuous simulation and early warning prediction of the model without human intervention.

[0090] Incremental updates involve directly appending boundary condition input files; the initial field file uses the variable field distribution at the end of the previous simulation (dfsu file); the simulation start time in the PFS file is the end time of the previous simulation, and the simulation end time is the latest time of this update. The time step number in the output settings must also be set accordingly.

[0091] After the update is complete, the system receives the status and initiates a model calculation request. Task information is added to the service backend task information file. The monitoring service implemented through NSSM detects this task and launches the updated calculation instance in the model sandbox through MzLaunch. The sandbox area is not within the middleware access path and is completely isolated from the network. It can only be accessed by software or processes with local access permissions, such as Python on the server.

[0092] The system can perform multi-dimensional visualization through the real-time result data returned by the server MIKEIO, presenting the latest updated simulation results in real time, thereby realizing real-time early warning and forecasting of pollutant exceeding the standard, and realizing short-term prediction of the spatiotemporal changes of pollutants based on meteorological data products and historical water supply changes in the next few days.

[0093] Taking MIKE21 as an example, this method builds an implementation path for self-updating and iterating the eutrophication model based on sklearn machine learning, which solves the problem of complicated and time-consuming repetitive operations in the model construction process. Combining the system and machine learning, it realizes a new usage scenario of model self-construction, autonomous calibration, and spontaneous forecasting, greatly saving the time cost of model construction, and realizing deep and strong coupling between the model and the system, as well as continuous simulation and early warning prediction of the model without human intervention.

[0094] On the other hand, the present invention proposes a self-updating and iterative system for eutrophication models based on machine learning, such as Figure 4 Shown, including:

[0095] Data acquisition module, used to obtain the data stream required by the water environment numerical simulation model from the database and API interface;

[0096] Data transmission module, used to send data streams to the Python environment on the server through PHP;

[0097] A file generation module, for generating or updating a terrain file, a boundary condition input file, and an initial field distribution file according to the data stream using the MIKEIO library in Python;

[0098] The task request module is used to send a model calculation request to the server after completing file preparation and record it in the task information file;

[0099] The calculation execution module is used to monitor the task information file and start MzLaunch to run the model sandbox area instance once a new calculation request is detected;

[0100] The result feedback module is used to use Python's MIKEIO library to read the dfsu file in the running results and provide real-time feedback to the front end through PHP;

[0101] A parameter optimization module is used to adjust water quality process parameters through the sklearn machine learning library and pass them to the MIKEIO to update the configuration file;

[0102] The self-update and iteration module is used to cyclically optimize the water environment numerical simulation model based on the updated configuration method file, thereby realizing the self-update and iteration of the water environment numerical simulation model.

[0103] In addition, when executed, the above modules are also used to implement other steps of the above-mentioned self-update and iteration method of the eutrophication model based on machine learning, which will not be described one by one here.

[0104] In addition, the present invention also provides a terminal device. The self-update and iteration method of the eutrophication model based on machine learning involved in this embodiment is mainly applied to the terminal device, which can be a PC, portable computer, mobile terminal, etc. with display and processing functions.

[0105] Specifically, a terminal device may include a processor (e.g., a CPU), a communication bus, a user interface, a network interface, and memory. The communication bus is used to enable communication between these components; the user interface may include a display and an input unit such as a keyboard; the network interface may optionally include a standard wired interface or a wireless interface (e.g., a Wi-Fi interface); and the memory may be high-speed RAM or non-volatile memory, such as disk storage. The memory may also be a storage device independent of the processor.

[0106] Among them, the memory stores a readable storage medium, and the readable storage medium stores a self-update and iteration program. The processor can call the self-update and iteration program stored in the memory and execute the machine learning-based eutrophication model self-update and iteration method provided by an embodiment of the present invention.

[0107] It will be understood that a computer-readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0108] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0109] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0110] In addition, the abbreviations mentioned in this application have the following meanings:

[0111] PHP: An open source server-side scripting language that enables data interaction between front-end systems and back-end servers.

[0112] Python: a language for machine learning.

[0113] MIKEIO: A pre- and post-processing library for the MIKE simulation software in Python.

[0114] Mesh: A non-structured triangular mesh file format in MIKE21. Mesh terrain files can be generated and modified through MIKEIO.

[0115] Dfs0: The time series file format in MIKE21, generally used to record one-dimensional data that changes over time. It is suitable as the boundary input condition of a point source and can be generated, modified and updated through MIKEIO.

[0116] Dfsu: A two-dimensional surface file format in MIKE21, generally used to record spatiotemporal distribution data and suitable for creating initial field files. MIKE21 global simulation results are saved in dfsu format. MIKEIO can read the model result file and transmit the data back to the front end.

[0117] PFS: MIKE21 configuration file format. After being read by MIKEIO, it can interpret the tree structure and generate and modify the running configuration file of the updated model, including setting all the model parameters.

[0118] sklearn: A very powerful machine learning library provided by a third party Python, covering modules such as classification, regression, clustering, dimensionality reduction, model selection, and data preprocessing.

[0119] NSSM: A powerful small tool that allows users to register any executable file (including batch scripts, exe programs or dll dynamic link libraries) as a Windows system service and conveniently manage and control it.

[0120] MzLaunch: MIKE21's shell-less computing engine, which can be called through batch scripts to start instance computing.

[0121] Model sandbox: This service serves as an area where model computing instances are stored. Because Python permissions enable full disk access, the sandbox can be placed in an environment that is not exposed to the network, achieving a higher level of data security protection.

[0122] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A self-updating and iterative method for eutrophication model based on machine learning, characterized in that: The steps include: Obtain the data stream required by the water environment numerical simulation model from the local database and API interface, and send the data stream to the Python environment on the server through PHP; Generate or update a terrain file, a boundary condition input file, and an initial field distribution file based on the data stream using the MIKEIO library in Python; After completing the file preparation, a model calculation request is sent to the server and recorded in the task information file. The server-side self-made service monitors the task information file. Once a new calculation request is detected, MzLaunch is started to run the model sandbox area instance. The dfsu file in the running result is read using Python's MIKEIO library, and the simulation results are fed back to the front end in real time through PHP. The front-end system combines the simulation result data and the measured data in the database to generate the data set required for machine learning and feed it into the Python environment of sklearn on the server. The calibration verification effect and the water quality process parameters are adjusted through the sklearn machine learning library, and the data is passed to the MIKEIO to update the configuration file. It also includes: In advance, a monitoring service based on a batch script is created on the server through NSSM. The task information file is scanned every 10 seconds in the form of a process to check whether the task information content has been updated. The details are as follows: After downloading and deploying the nssm software, you need to create a bat command that is set to scan the task information file every 10 seconds. If the model running result folder path does not exist through the file system, the system will call the MzLaunch process and pass in the model case path. MIKE21 will start the case simulation and the system will record the run log. If the model running result folder path in the task information file exists, the case has been run and no operation will be performed to continue monitoring. Since the model running result folder path has been automatically established at this time, nssm will enter the monitoring mode of the next cycle and will not restart the case. If there is no valid information in the task information file, nssm will exit this round of monitoring and enter the next round of monitoring. Based on the machine learning method, the water environment numerical simulation model is cyclically optimized to achieve self-update and iteration of the water environment numerical simulation model. Specifically, after each calculation case, the root mean square error and goodness of fit between the simulated pollutant concentration values ​​output by the model and the measured values ​​provided by the front-end are evaluated; if the evaluation result does not meet the predetermined standard, the model process parameters are directed reselected through the ParameterGrid and train_test_split of the machine learning library, and updated to the model parameter configuration file through the MIKEIO package to enter a new round of simulation; when the evaluation result meets the predetermined standard, the final round of model parameter configuration file is adopted and the autonomous continuous simulation stage is entered; The method of obtaining the data stream required for the water environment numerical simulation model from the local database and API interface includes: the front end accesses the local database to extract pre-built meteorological data, hydrodynamic data, water quality data, terrain data, and setting data; based on the extracted data, the front end interpretation code is used to format the data and send it to the back end in JSON format; the system calls the external data API interface from the front end to obtain the latest meteorological data and water quality monitoring data in real time, and merges it with the formatted data into a unified data stream; The method of sending the data stream to the Python environment of the server through PHP includes: using Internet information services to open the door of the back-end server, and using PHP as a bridge for data transmission to receive the unified data stream from the front-end; in the PHP hub code, calling the popen process function to pass the unified data stream to the Python process; The method of using the MIKEIO library in Python to generate or update a terrain file, a boundary condition input file, and an initial field distribution file according to the data stream includes: after the Python process receives the data stream, starting to parse the data stream; based on the parsed data stream, using the MIKEIO library to generate or update the terrain file, the boundary condition input file, and the initial field distribution file, ensuring that all files meet the requirements of the MIKE21 model and storing them in a preset directory; recording the status of file generation or update, and confirming that the file preparation is complete; After the file preparation is completed, a model calculation request is sent to the server and recorded in the task information file, including: based on the status of confirming that the file preparation is completed, a model calculation request is sent to the server; the model calculation request is recorded in the task information file, and the task information file contains all necessary parameters and configuration information for this calculation.

2. The self-updating and iterative method for eutrophication model based on machine learning according to claim 1, characterized in that: The server-side self-made service monitors the task information file. Once a new computing request is detected, MzLaunch is started to run the model sandbox area instance, including: MzLaunch runs the MIKE21 model in the model sandbox area instance according to the parameters and configuration information in the task information file, and records the model running status to ensure that the model is executed as expected.

3. The self-updating and iterative method of eutrophication model based on machine learning according to claim 2, characterized in that: The method uses the MIKEIO library of Python to read the dfsu file in the running results and feeds back the simulation results to the front end in real time through PHP, including: parsing the dfsu file, extracting relevant data and formatting it; and feeding back the formatted data to the front end in real time through PHP for users to view and analyze.

4. The self-updating and iterative method for eutrophication model based on machine learning according to claim 3, characterized in that: The front-end system combines the simulation result data and the measured data in the database to generate the data set required for machine learning and feeds it to the Python environment of sklearn on the server side. The calibration verification effect and the adjustment of water quality process parameters are judged by the sklearn machine learning library, and are passed to the MIKEIO to update the configuration file, including: based on the extracted relevant data, using the ParameterGrid of the sklearn machine learning library to generate parameter combinations, and separating a set of theoretically better parameter combinations through train_test_split; according to the optimization results, the water quality process parameters are adjusted and passed to the MIKEIO library to update the configuration file.

5. A self-updating and iterative system for a eutrophication model based on machine learning for implementing the method according to any one of claims 1 to 4, characterized in that: include: Data acquisition module, used to obtain the data stream required by the water environment numerical simulation model from the database and API interface; Data transmission module, used to send data streams to the Python environment on the server through PHP; A file generation module, for generating or updating a terrain file, a boundary condition input file, and an initial field distribution file according to the data stream using the MIKEIO library in Python; The task request module is used to send a model calculation request to the server after completing file preparation and record it in the task information file; The calculation execution module is used to monitor the task information file and start MzLaunch to run the model sandbox area instance once a new calculation request is detected; The result feedback module is used to use Python's MIKEIO library to read the dfsu file in the running results and provide real-time feedback to the front end through PHP; The parameter optimization module is used to adjust the water quality process parameters through the sklearn machine learning library and pass them to the MIKEIO to update the configuration file; the self-update and iteration module is used to cyclically optimize the water environment numerical simulation model based on the machine learning method to achieve self-update and iteration of the water environment numerical simulation model.

Images