Machine-learning-based management system for virtual numerical-tank experiment

Abstract

Disclosed in the present invention is a machine-learning-based management system for a virtual numerical-tank experiment. The system comprises: a data collection module, which is configured to acquire initial data of various scenarios in a virtual numerical-tank experiment, and randomly generate simulation data, wherein the simulation data is data to be processed of various hydrodynamic phenomenon scenarios, including water movements, water level changes and waves, in a real hydrological environment; a central storage module, which is connected to the data collection module and is configured to process the simulation data after receiving a storage task sent by the data collection module, and perform bottom-layer storage on the processed simulation data; and an automated storage module, which is connected to the central storage module and is configured to automatically adjust a storage resource configuration according to a predictive-analysis result, and maintain data consistency of a plurality of copies. The present invention increases the storage capacity of a system for experiment data, improves the accuracy and efficiency of an experiment, and realizes automated management of a virtual numerical-tank experiment.

Description

A numerical water pool virtual test management system based on machine learning Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and distributed systems, and in particular relates to a numerical water pool virtual test management system based on machine learning. Background Art

[0002] Numerical water tank virtual test data refers to the data generated when hydrodynamic phenomena are numerically simulated using computer simulation methods. In hydrodynamic research, numerical water tank virtual testing is an important means. Through computer simulation, hydrodynamic phenomena such as water flow movement, water level changes, and waves in actual hydrological environments can be simulated. Numerical water tank virtual test data usually includes simulated hydrological environment parameters, hydrodynamic parameters such as water flow velocity, water level height, wave shape in the simulation results, and related numerical calculation errors. These data can be used to establish, verify, and optimize hydrodynamic models, and can also be used in applications in engineering design, water resources management, disaster warning, and other fields. The acquisition and processing of numerical water tank virtual test data requires knowledge and techniques from multiple fields such as numerical calculation, computer science, and mathematics, and is difficult and complex.

[0003] Driven by technologies like cloud computing, big data, and artificial intelligence, the demand for data storage in data pools is rapidly increasing. To meet this demand, distributed storage systems are becoming mainstream, but their management and maintenance are cumbersome and complex. Existing distributed storage systems typically require manual management and maintenance, are unable to adapt to diverse usage scenarios and workloads, and struggle to achieve efficient and automated storage management.

[0004] Summary of the Invention

[0005] The purpose of the present invention is to establish a numerical water pool virtual experiment management system based on machine learning automated distributed storage to manage numerical water pool virtual experiments more efficiently and accurately. Through machine learning technology, the command system can automatically collect, process and analyze large amounts of data, thereby realizing the automated management of numerical water pool virtual experiments. In this way, researchers can focus more on analyzing the test results without spending a lot of time and energy to process the data. At the same time, in order to improve the storage capacity and the accuracy and efficiency of the test, the system adopts distributed storage technology, which can distribute data storage tasks among multiple computer nodes, thereby increasing the system's storage capacity for test data. At the same time, the system can also automatically identify and process outliers and noise in the data through machine learning technology, thereby improving the accuracy and reliability of the test.

[0006] To achieve the above objectives, the present invention provides the following solution: a numerical water pool virtual test management system based on machine learning, comprising:

[0007] The data collection module is used to obtain initial data under various scenarios in the digital virtual water tank test and randomly generate simulated data; the simulated data is the data to be processed for various scenarios of hydrodynamic phenomena in the actual hydrological environment, including water flow movement, water level changes, and waves;

[0008] A central storage module, connected to the data collection module, for processing the simulation data and storing the processed simulation data in the underlying storage after receiving the storage task sent by the data collection module;

[0009] The automated storage module is connected to the central storage module and is used to automatically adjust storage resource configuration according to the prediction analysis results and maintain data consistency of multiple copies.

[0010] Preferably, the data collection module includes a data transmission unit, an intelligent agent unit, a data collection unit, a mode conversion unit, a data blocking unit, and a communication unit connected in sequence;

[0011] The data transmission unit is used to transmit the initial data in various scenarios of the digital virtual water pool test to the intelligent agent unit, and is also used to transmit the simulated data after data segmentation to the central storage module through the communication unit;

[0012] The intelligent agent unit is used to automatically identify the data categories in various scenarios; the data collection unit is used to collect initial data in various scenarios;

[0013] The mode conversion unit is used to determine the data storage model according to different data categories and provide an ordered traversal method;

[0014] The data block unit is used to block according to the key, dividing the entire key value space into many segments, each segment is a series of continuous keys, and a certain continuous key is stored on a storage node. Each segment is divided into a block area, which is described by a left-closed and right-open interval such as [start key, end key).

[0015] Preferably, the central storage module includes a machine learning unit, a data distribution unit, and an underlying storage unit MicaDB;

[0016] The machine learning unit is used to predict and optimize the operating status and performance of the pool based on historical data and real-time data;

[0017] The data distribution unit is used to distribute data storage tasks among multiple computer nodes through distributed storage technology, and integrate and manage data from multiple pools;

[0018] The underlying storage unit MicaDB is used to store underlying data through Star protocol block areas.

[0019] Preferably, the machine learning unit includes a communication interface unit, a data preprocessing unit, a model training unit, and a prediction unit;

[0020] The communication interface unit is connected to the communication unit and is used to receive the analog data after data blocking;

[0021] The data preprocessing unit is used to perform data cleaning, feature selection, feature extraction and feature conversion on the simulated data after data segmentation;

[0022] The model training unit is used to train the machine learning model using the data processed by the data preprocessing unit to obtain a target prediction model;

[0023] The prediction unit is used to input real-time data into the target prediction model to perform prediction analysis on future storage requirements and obtain prediction results.

[0024] Preferably, the central storage module includes storage nodes and a central management node, wherein the storage nodes are deployed in the machine learning unit, and an intelligent agent is installed on each storage node, which automatically identifies and handles abnormal conditions of storage nodes and data and reports them to the central management node; the intelligent agent is also used to summarize and report usage information of storage nodes and data to the central management node;

[0025] The central management node is used to automatically analyze the usage patterns of storage nodes and data to discover and predict possible faults, and automatically start the corresponding fault handling program when a fault occurs.

[0026] Preferably, the central management node includes a machine learning engine and a controller;

[0027] The machine learning engine is used to use historical data and existing data for training, thereby automatically analyzing the usage patterns of storage nodes and data and predicting future usage;

[0028] The controller is used to formulate storage strategies based on the prediction results of the machine learning engine and send corresponding commands to the storage node intelligent agent for execution.

[0029] Preferably, the storage nodes can dynamically join or exit the system, and the communication between the intelligent agent of the storage node and the central management node is carried out in a secure encryption manner.

[0030] Preferably, the automated storage module adopts the Star protocol, which includes leader selection, log replication, and security measures;

[0031] The leadership election requirement is that a new leader must be elected if the existing leader fails;

[0032] Log replication refers to when the leader accepts commands from the client, records them as logs, and replicates them to other servers in the cluster, forcing the logs of other nodes to be consistent with the leader;

[0033] The security measures are used to ensure the security of the system through some measures.

[0034] Preferably, the node states in the Star protocol include leader, follower, and candidate;

[0035] The leader receives all requests from clients, including processing write requests, managing log replication, and continuously sending heartbeat messages to notify other nodes not to initiate new elections. The Star protocol ensures that there is only one leader at any time.

[0036] The follower is used to passively receive and process messages from the leader. When the leader's heartbeat times out, it takes the initiative to stand up and recommends itself as a candidate.

[0037] The candidate is used to elect a new leader; by sending voting RPC messages to other nodes, notifying other nodes to vote, and winning the majority of votes to be promoted to the leader.

[0038] Compared with the prior art, the present invention has the following advantages and technical effects:

[0039] (1) Improved test efficiency: Traditional numerical water tank tests require a lot of time and human resources, while the virtual test management system based on machine learning can automate the test process, greatly improving test efficiency.

[0040] (2) Improved test accuracy: Virtual testing can simulate numerical water tank tests under different parameter combinations, providing more accurate test data and thus improving test accuracy.

[0041] (3) Reduced test costs: Traditional numerical water tank tests require the establishment of actual test equipment and human resources, which is costly. However, virtual tests do not require actual equipment and human resources, which greatly reduces test costs.

[0042] (4) Improved data management efficiency: The machine learning-based virtual test management system can automatically store test data for easy management and analysis. In addition, the distributed storage system can provide more efficient and reliable data storage services.

[0043] (5) Fault tolerance: If the system experiences an abnormal situation such as downtime or data corruption, the Star protocol's fault tolerance mechanism can be used to retrieve data and logs from other nodes for recovery and replay. Furthermore, the system supports multi-copy backup, achieving fault tolerance and high availability by replicating data across multiple nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0045] FIG1 is a schematic diagram of a system structure according to an embodiment of the present invention;

[0046] FIG2 is a structural diagram of an intermediate storage module system according to an embodiment of the present invention;

[0047] FIG3 is a block diagram of a Star protocol module according to an embodiment of the present invention;

[0048] FIG4 is a flowchart illustrating a storage function execution according to an embodiment of the present invention;

[0049] FIG5 is a flowchart illustrating the execution of a machine learning module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0051] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0052] As shown in FIG1 , the numerical water pool virtual test management system based on machine learning provided by the present invention mainly consists of three modules: a data collection module, a central storage module, and an automatic storage module.

[0053] (1) Data collection module

[0054] The data collection module mainly includes an intelligent agent module, a data collection module, and a system performance module. As a system for storing data, the first thing to decide is the data storage model, that is, in what form the data is stored. This system chooses a key-value model and provides an ordered traversal method. There are two key points in data storage: 1. This is a huge Map, which means that key-value pairs are stored; 2. The key-value pairs in this Map are ordered according to the binary order of the keys, which means that the location of a certain key can be found, and then the sequential traversal method is continuously called in ascending order to obtain key values ​​larger than this key. The system can collect various storage data and shard them. This data will be transmitted to the central storage module for analysis and processing, making the system more scalable.

[0055] System scalability refers to the system's ability to expand in response to future changes in the storage requirements of digital water tank virtual tests. When new storage requirements arise, the distributed system can support them with minimal or no changes, eliminating the need for reconfiguration or rebuilding of the entire system. The intelligent agent in the data collection module automatically identifies the data category in various scenarios and paves the way for transmission to the central storage module. The function of the data collection module is to process the raw data from the digital virtual water tank test. Accordingly, the data acquisition module needs to comprehensively process the data from various scenarios. By randomly generating simulated data, it simulates various scenarios such as water flow movement, water level changes, waves, and other hydrodynamic phenomena in the actual hydrological environment. This allows for efficient and flexible testing of the digital water tank test system's data collection to determine whether it can correctly, completely, and stably process the data from different scenarios. This is a targeted test of scalability.

[0056] (2) Central Storage Module

[0057] The central storage module primarily consists of a machine learning module and the underlying storage module, MicaDB. The machine learning module consists of five modules: a communication interface, a data preprocessing module, a model training module, a prediction module, and a storage node selection module. The central storage module is located between the data collection module and the automated storage module. Upon receiving storage tasks from the data collection module, it processes the data and stores it in the underlying storage.

[0058] Data preprocessing is a very important step in the machine learning process, and typically includes multiple steps such as data cleaning, feature selection, feature extraction, and feature conversion. The purpose of data preprocessing is to process the raw data so that it is more suitable for training machine learning models.

[0059] The model training module uses historical data to train machine learning models so that future storage needs can be predicted. Model training is the core step of the machine learning module. Its function is to input a labeled data set and adjust the model parameters to make the model better fit the data, thereby achieving prediction or classification tasks.

[0060] The prediction module can predict future storage needs based on the prediction analysis results. After the model is loaded, the prediction module can receive input data, input it into the model for prediction, and obtain the prediction results.

[0061] After passing through the machine learning module, the data is sent to MicaDB for storage. With any persistent storage engine, data must ultimately be stored on disk, and this system is no exception. However, this embodiment does not write data directly to disk. Instead, it stores the data in MicaDB, which is responsible for the specific data delivery. Here, MicaDB can be simply considered a standalone key-value map.

[0062] (3) Automated storage module

[0063] In the automated management module, the system can automatically adjust storage resource configuration based on predictive analysis results, such as automatically increasing storage capacity and adjusting data allocation strategies. This module also addresses one of the core issues of distributed storage systems: maintaining data consistency across multiple replicas.

[0064] To maintain consistency, this system uses the Star consensus protocol. The algorithm is divided into several parts, including leader election, log replication, and safety measures. Leader election requires that a new leader must be elected if the existing leader fails. Log replication refers to when the leader accepts commands from the client, records them in a log, and replicates them to other servers in the cluster, forcing the logs of other nodes to remain consistent with the leader. Safety measures: System security is ensured through measures such as ensuring that all state machines execute the same commands in the same order.

[0065] In Star, nodes have the following three states: (1) Leader: Receives all requests from clients. The leader's daily work includes three parts: processing write requests, managing log replication, and continuously sending heartbeat messages to notify other nodes "I am the leader, I am still alive, and you should not initiate a new election now." The Star protocol ensures that there is only one leader at any time. (2) Follower: Passively receives and processes messages from the leader. When the leader's heartbeat times out, it takes the initiative to stand up and recommends itself as a candidate. (3) Candidate: Used to elect a new leader. The candidate sends a voting RPC message to other nodes to notify other nodes to vote. If it wins the majority of votes, it will be promoted to leader.

[0066] Example 1

[0067] As shown in Figure 1, this system is a key-value database based on machine learning distributed automated storage. It provides a distributed transaction interface that meets ACID constraints and ensures multi-replica data consistency and high availability through the Star protocol. MicaDB, as the system's underlying storage layer, provides persistence and read / write services for user data written to the system, and also stores relevant statistical data. Data can be automatically stored in MicaDB's underlying storage using the machine learning module.

[0068] As mentioned above, this system can be regarded as a huge ordered key-value table. In order to achieve horizontal expansion of storage, data will be distributed on multiple machines. For a key-value system, there are two typical solutions for distributing data on multiple machines: (1) Hashing: Hash according to the key and select the corresponding storage node according to the hash value. (2) Blocking: Block according to the key, a certain continuous key is stored on a storage node. This embodiment chooses the second method in the data collection module, dividing the entire key value space into many segments, each segment is a series of continuous keys, and each segment is called a block area, which can be described by a left-closed and right-open interval such as [start key, end key). The amount of data stored in each block area is maintained at about 96 MiB by default (can be modified through configuration). Data is then exchanged with the central storage module through the communication module.

[0069] In the machine learning module, this embodiment first performs data collection and processing. This process involves collecting data from various data sources, including real-time sensor data in the numerical water pool, historical data, meteorological data, hydrological data, and so on. This data needs to be preprocessed and cleaned to ensure data quality and integrity and to prepare it for subsequent analysis.

[0070] The data is then fed into the feature engineering module, a process that involves converting raw data into more meaningful features to improve the accuracy and performance of the machine learning model. For example, time series data can be converted into frequency domain features to improve the accuracy of time series forecasting.

[0071] After extracting data features, this embodiment performs model selection and training. This process involves selecting an appropriate machine learning algorithm and training the model using feature-engineered data. For example, a supervised learning-based regression model can be used to predict water level changes. This process may also involve hyperparameter tuning and model evaluation to ensure the model's generalization performance.

[0072] Finally, this example performs model deployment and inference. This involves deploying the trained model to an actual numerical water pool and using the model to perform real-time inference and prediction. For example, the model can be used to predict future water level changes and make corresponding adjustments based on the predicted results.

[0073] The model designed in this embodiment also requires monitoring and maintenance. This process involves monitoring the performance and health of the model and promptly identifying and addressing any anomalies. For example, if the model's predictive performance degrades, it may be necessary to retrain the model or reselect the algorithm.

[0074] Unlike traditional full-node backup methods, MicaDB's storage system uses a replication mechanism. Data is divided into roughly equal slices (hereinafter referred to as chunks) based on key ranges. Each slice has multiple replicas (usually three), which are maintained consistent through the Star protocol.

[0075] As shown in Figure 2, this embodiment deploys multiple storage nodes within the machine learning module and installs an intelligent agent on each storage node. The intelligent agent automatically identifies and handles abnormalities in storage nodes and data, reporting them to a central management node. Furthermore, the intelligent agent is responsible for summarizing and reporting usage information for storage nodes and data to the central management node.

[0076] Deploy a central management node and install a machine learning engine and controller. The machine learning engine can be trained using historical and existing data to automatically analyze usage patterns of storage nodes and data and predict future usage. The controller can formulate storage policies based on the machine learning engine's predictions and issue corresponding commands to the storage node intelligent agents for execution.

[0077] Communication between storage node smart agents and the central management node is conducted using secure encryption. This ensures the security and reliability of data during transmission. Storage node smart agents also undergo identity verification to ensure that only authorized smart agents can communicate with the central management node.

[0078] The machine learning engine automatically analyzes usage patterns of storage nodes and data to predict future usage patterns. It can analyze multiple factors, including data volume, data type, access patterns, and access frequency, to predict future usage.

[0079] The controller formulates storage policies based on the machine learning engine's predictions and issues corresponding commands to the storage node intelligent agent for execution. Storage policies can include operations such as data migration, backup, and recovery. For example, if the machine learning engine predicts a sharp increase in the future utilization of a storage node, the controller can formulate a corresponding data migration policy to migrate some data to an idle storage node, thereby avoiding storage node overload.

[0080] The intelligent agent can automatically identify and handle abnormal conditions of storage nodes and data, and report them to the central management node. For example, when a storage node experiences a hardware failure or data corruption, the intelligent agent can promptly report to the central management node and perform data recovery or reallocation of storage according to the controller's instructions.

[0081] Figure 3 shows the operational process of maintaining data consistency in the Star protocol. In the Star protocol, nodes exist in the following three states: (1) Leader: Receives all requests from clients. The leader's daily work includes three parts: processing write requests, managing log replication, and continuously sending heartbeat messages to notify other nodes "I am the leader, I am still alive, and you should not initiate a new election now." The Star protocol ensures that there is only one leader at any time. (2) Follower: Passively receives and processes messages from the leader. When the leader's heartbeat times out, it takes the initiative to stand up and recommends itself as a candidate. (3) Candidate: Used to elect a new leader. The candidate sends a voting RPC message to other nodes to notify other nodes to vote. If it wins the majority of votes, it will be promoted to the leader.

[0082] The following explains the three states: followers and candidates are passive, and leaders are active. The role of a node may change over time and with changes in network conditions. The following is a detailed description of the node role changes:

[0083] In the initial state, all nodes are set to follower state.

[0084] Follower->Candidate: If a follower does not receive a heartbeat message from the leader within a certain period of time, it will assume that the leader has failed and become a candidate. The candidate sends a message requesting votes to other nodes, asking them to vote for themselves to become the new leader.

[0085] Candidate->Leader: If a candidate receives votes from a majority of nodes in the cluster, it becomes the new leader. The leader sends heartbeat messages to other nodes to keep them in sync and begins processing requests from clients.

[0086] Leader -> Follower: If a leader finds itself unable to continue as a leader, such as due to network partition or downtime, it will fall back to being a follower. The leader will send a step-down message to inform other nodes that it is no longer a leader and to start following the new leader.

[0087] Candidate->Follower: If a candidate does not receive votes from a majority of nodes in the cluster, it will fall back to being a follower.

[0088] In the Star protocol, node role changes are implemented through a communication protocol and an election algorithm. Through these mechanisms, nodes can switch to different roles at different points in time to ensure cluster correctness and fault tolerance. Node role changes typically occur automatically, without the need for manual intervention.

[0089] The overall flow chart shown in Figure 4 primarily covers the user registration and login process: users must register and log in to the system to use its features. This process includes steps such as user information input, information verification, user account creation, and login token generation. The experiment management process includes steps such as creating new experiments, viewing existing experiments, modifying experiment parameters, running experiments, and deleting experiments. The data preprocessing process includes steps such as collecting data from sensors or other data sources, preprocessing and filtering the data, and storing the data in a distributed storage system. The data storage and query process includes steps such as storing data in a distributed storage system, querying and retrieving data from the storage system, and aggregating and processing the data. The machine learning process includes steps such as obtaining data from the storage system, preprocessing the data, selecting an appropriate machine learning algorithm, training the machine learning model, and using the model for prediction and inference. The visualization process includes steps such as obtaining data from the storage system, processing the data, generating charts and visualization tools, and presenting the visualization results to the user.

[0090] Figure 5 shows the execution flow chart of the machine learning module. The first is the data preprocessing stage: in this stage, the data needs to be cleaned, converted, and feature extracted to better prepare for model training. Data preprocessing includes the following steps: a. Data cleaning: Check whether there are invalid values, missing values, or outliers in the data, and remove them from the dataset or fill them with appropriate values. b. Data conversion: Convert the data into a format that the model can process, such as converting categorical variables to digital encoding, discretizing continuous variables, normalizing or standardizing, etc. c. Feature extraction: Extract features from the data to help the model learn. This may involve the use of statistical analysis, natural language processing, image processing, and other techniques to extract useful information from the data.

[0091] The second step is the model training phase: In this phase, you need to select a model that is appropriate for the dataset and problem, use the dataset to train the model, and adjust the model parameters for better performance. Model training includes the following steps: a. Model selection: Select a model that is appropriate for the problem and dataset, such as linear regression, decision tree, or neural network. b. Dataset partitioning: Divide the dataset into a training set, a validation set, and a test set. The training set is used to train the model, the validation set is used to adjust the model parameters, and the test set is used to evaluate model performance. c. Model training: Use the training set to train the model, and use the validation set to adjust the model parameters for better performance. d. Model saving: Save the trained model for subsequent prediction and inference.

[0092] The third step is the model evaluation phase: At this stage, the model's performance needs to be evaluated to determine whether the model can effectively solve the problem. Model evaluation includes the following steps: a. Model prediction: Use the test set to make model predictions and compare the predicted results with the actual results. b. Model performance evaluation: Use various evaluation metrics (such as accuracy, recall, F1 value, etc.) to evaluate the model's performance and compare the performance of different models. c. Model improvement: Based on the model performance evaluation results, adjust the model parameters and retrain the model.

[0093] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A numerical water pool virtual test management system based on machine learning, characterized in that: include: The data collection module is used to obtain the initial data of various scenarios in the digital virtual water tank test and randomly generate simulation data; the simulation data is the data to be processed in various scenarios of hydrodynamic phenomena including water flow movement, water level change, and waves in the actual hydrological environment; A central storage module, connected to the data collection module, is used to process the simulation data and store the processed simulation data in the bottom layer after receiving the storage task sent by the data collection module; The automated storage module is connected to the central storage module and is used to automatically adjust storage resource configuration according to the prediction analysis results and maintain data consistency of multiple copies.

2. The numerical water pool virtual test management system based on machine learning according to claim 1 is characterized in that: The data collection module includes a data transmission unit, an intelligent agent unit, a data collection unit, a mode conversion unit, a data block unit, and a communication unit connected in sequence; The data transmission unit is used to transmit the initial data in various scenarios in the digital virtual water pool test to the intelligent agent unit, and is also used to transmit the simulated data after data segmentation to the central storage module through the communication unit; The intelligent agent unit is used to automatically identify the data categories in various scenarios; the data collection unit is used to collect initial data in various scenarios; The mode conversion unit is used to determine the data storage model according to different data categories and provide an ordered traversal method; The data block unit is used to block by key, dividing the entire key value space into many segments, each segment is a series of continuous keys, a certain segment of continuous keys is stored on a storage node, and each segment is divided into a block area, which is described by a left-closed and right-open interval such as [start key, end key).

3. The numerical water pool virtual test management system based on machine learning according to claim 1 is characterized in that: The central storage module includes a machine learning unit, a data distribution unit, and an underlying storage unit MicaDB; The machine learning unit is used to predict and optimize the operation status and performance of the pool based on historical data and real-time data; The data distribution unit is used to distribute data storage tasks among multiple computer nodes through distributed storage technology, and integrate and manage the data of multiple pools; The underlying storage unit MicaDB is used to store underlying data through Star protocol block areas.

4. The numerical water pool virtual test management system based on machine learning according to claim 3 is characterized in that: The machine learning unit includes a communication interface unit, a data preprocessing unit, a model training unit, and a prediction unit; The communication interface unit is connected to the communication unit and is used to receive the analog data after data segmentation; The data preprocessing unit is used to perform data cleaning, feature selection, feature extraction and feature conversion on the simulated data after data segmentation; The model training unit is used to train the machine learning model through the data processed by the data preprocessing unit to obtain a target prediction model; The prediction unit is used to input real-time data into the target prediction model to perform prediction analysis on future storage requirements and obtain prediction results.

5. The numerical water pool virtual test management system based on machine learning according to claim 1 is characterized in that: The central storage module includes a storage node and a central management node, wherein the storage node is deployed in a machine learning unit, and an intelligent agent is installed on each storage node, through which the abnormal conditions of the storage node and data are automatically identified and processed, and reported to the central management node; the intelligent agent is also used to summarize the usage information of the storage node and data and report it to the central management node; The central management node is used to automatically analyze the usage patterns of storage nodes and data to discover and predict possible faults, and automatically start the corresponding fault handling program when a fault occurs.

6. The numerical water pool virtual test management system based on machine learning according to claim 5 is characterized in that: The central management node includes a machine learning engine and a controller; The machine learning engine is used to use historical data and existing data for training, so as to automatically analyze the usage patterns of storage nodes and data and predict future usage; The controller is used to formulate storage strategies based on the prediction results of the machine learning engine and send corresponding commands to the storage node intelligent agent for execution.

7. The numerical water pool virtual test management system based on machine learning according to claim 5 is characterized in that: The storage nodes can dynamically join or exit the system, and the communication between the intelligent agent of the storage node and the central management node is carried out in a secure encrypted manner.

8. The numerical water pool virtual test management system based on machine learning according to claim 5 is characterized in that: The automated storage module adopts the Star protocol, which includes leader selection, log replication, and security measures; The leadership election requires that a new leader must be elected if an existing leader fails; Log replication refers to when the leader accepts commands from the client, records them as logs, and replicates them to other servers in the cluster, forcing the logs of other nodes to be consistent with the leader; The security measures are used to ensure the security of the system through some measures.

9. The numerical water pool virtual test management system based on machine learning according to claim 8 is characterized in that: The node states in the Star protocol include leader, follower, and candidate; The leader is used to receive all requests from the client, including processing write requests, managing log replication, and continuously sending heartbeat information to notify other nodes not to initiate new elections. The Star protocol ensures that there is only one leader at any time. The follower is used to passively receive and process messages from the leader. When the leader's heartbeat times out, the follower takes the initiative to stand out and recommends itself as a candidate. The candidate is used to elect a new leader; by sending voting RPC messages to other nodes, notifying other nodes to vote, and winning the majority of votes to be upgraded to the leader.

Images