Water conservancy data processing method and system based on agent large model, terminal and medium
Through the water conservancy data processing method based on the agent's large model, the corresponding agent's large model is automatically called to process water conservancy data, which solves the problem of unintegrated data management and prediction functions in the existing system, and realizes the intelligence, automation and precision of water conservancy data processing.
Patent Information
- Application Number
- CN202510530824.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-25
- Publication Date
- 2025-07-29
AI Technical Summary
The existing water conservancy data processing system fails to effectively integrate data management and prediction functions, resulting in the inability to fully utilize the effectiveness of the agent's large model when facing complex water conservancy data processing tasks, and it is difficult to meet the intelligent, automated and precise needs of the water conservancy industry for data processing.
Through the water conservancy data processing method based on the agent big model, user requests are obtained and analyzed, pre-trained data management or prediction agent big model is called, and corresponding management reports or prediction results are generated according to the request type, so as to realize the automation and intelligence of water conservancy data management and prediction.
It improves data processing efficiency and accuracy, reduces the workload of manual sorting and analysis, realizes the automation and intelligence of water conservancy data management and prediction, and adapts to the intelligent needs of the water conservancy industry.
Smart Images
Figure CN120386800A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of large model data processing, and particularly to a water conservancy data processing method, system, terminal and medium based on an intelligent agent large model. Background Art
[0002] The processing and analysis of water conservancy data play a crucial role in the rational utilization of water resources, the safe operation of water conservancy facilities, and the prevention and response to water disasters. Traditional water conservancy data processing methods mainly rely on manual experience and technology, suffering from problems such as low efficiency, insufficient accuracy, and difficulty in coping with complex and changeable water conservancy situations. For example, water conservancy data management often requires a large amount of manpower for data sorting, classification, and analysis, and is prone to omissions and errors. In terms of water conservancy data prediction, although traditional methods such as time series analysis can predict data such as water levels and flows to a certain extent, when faced with complex non-linear relationships and the combined effects of multiple factors, their prediction accuracy and reliability often fail to meet actual needs.
[0003] Intelligent agent large models have powerful data processing capabilities and learning abilities, capable of automatically extracting features, discovering patterns from massive data, and making intelligent decisions and predictions based on this. Existing water conservancy data processing systems fail to effectively integrate data management and data prediction functions, resulting in the inability to fully utilize the effectiveness of intelligent agent large models when faced with complex water conservancy data processing tasks, and it is difficult to meet the intelligent, automated, and precise requirements of the water conservancy industry for data processing. Summary of the Invention
[0004] To solve the above problems, the present invention provides a water conservancy data processing method, system, terminal and medium based on an intelligent agent large model, which improves data processing efficiency, accuracy, and reliability, and realizes the automation and intelligence of water conservancy data management and prediction. It can automatically call the corresponding intelligent agent large model for processing according to different types of requests, and meet the intelligent, automated, and precise requirements of the water conservancy industry for data processing.
[0005] In the first aspect, the technical solution of the present invention is a water conservancy data processing method based on an intelligent agent large model, including the following steps: Obtain and parse the data processing request sent by the user, and retrieve the target water conservancy data from the database according to the parsing result and store it in the buffer area; Detect the type of data processing request; If the data processing request type is a water conservancy data management type, call the pre-trained data management intelligent agent large model to process the target water conservancy data in the buffer area and generate a water conservancy data management report; If the data processing request type is a water conservancy data prediction type, call the pre-trained data prediction intelligent agent large model to process the target water conservancy data in the buffer area and generate a water conservancy data prediction result.
[0006] In an alternative embodiment, the method further includes the step of training the data prediction intelligent agent large model, specifically including: Obtain several groups of historical water conservancy data and corresponding prediction target data; Use a clustering algorithm to cluster several groups of historical water conservancy data to generate a clustering result, and label a data pattern for each clustering result; Construct a training data set from each group of clustered historical water conservancy data and corresponding prediction target data; Use the training data set to train the teacher model and the knowledge-distilled student model of the data prediction intelligent agent large model; According to the training results of the teacher model and the student model for each type of training data set, assign an appropriate data pattern to the teacher model and the student model.
[0007] In an alternative embodiment, according to the training results of the teacher model and the student model for each type of training data set, assigning an appropriate data pattern to the teacher model and the student model specifically includes: Obtain the prediction accuracy and processing time of the teacher model and the student model; Perform a weighted sum of the prediction accuracy and processing time to obtain the prediction evaluation scores of the teacher model and the student model; Match and associate the model with the larger prediction evaluation score with the current data pattern.
[0008] In an alternative embodiment, calling the pre-trained data prediction intelligent agent large model to process the target water conservancy data in the buffer area and generate a water conservancy data prediction result specifically includes: Retrieve the target water conservancy data from the buffer area; Use a clustering algorithm to cluster the target water conservancy data to obtain a clustering result, and match the data pattern according to the clustering result; Filter and associate the teacher model or the student model according to the matched data pattern; Use the associated teacher model or student model to process the target water conservancy data in the buffer area and generate a water conservancy data prediction result.
[0009] In an alternative embodiment, calling the pre-trained data management intelligent agent large model to process the target water conservancy data in the buffer area and generate a water conservancy data management report specifically includes: Retrieve the water conservancy data management report template; Construct a prompt based on the target water conservancy data in the buffer area and the water conservancy data management report template; Input the prompt words, the target water conservancy data in the buffer, and the water conservancy data management report template into the pre-trained data management intelligent agent large model to generate a water conservancy data management report.
[0010] In an alternative embodiment, retrieve the target water conservancy data from the database according to the parsing result and store it in the buffer, specifically including: Obtain the target time period and the target water conservancy data type identifier from the parsing result; Construct an SQL statement according to the target time period and the target water conservancy data type identifier; Retrieve the target water conservancy data from the database based on the SQL statement; Store the target water conservancy data in the buffer.
[0011] In an alternative embodiment, detect the data processing request type, specifically including: Retrieve the data value of the target field from the parsing result; Judge the data processing request type according to the data value.
[0012] In a second aspect, the technical solution of the present invention provides a water conservancy data processing system based on an intelligent agent large model, including: A data processing request acquisition module, configured to acquire and parse a data processing request sent by a user, and retrieve the target water conservancy data from the database according to the parsing result and store it in the buffer; A data processing request type detection module, configured to detect the data processing request type; A water conservancy data management report generation module, configured to, if the data processing request type is a water conservancy data management type, call the pre-trained data management intelligent agent large model to process the target water conservancy data in the buffer and generate a water conservancy data management report; A water conservancy data prediction result generation module, configured to, if the data processing request type is a water conservancy data prediction type, call the pre-trained data prediction intelligent agent large model to process the target water conservancy data in the buffer and generate a water conservancy data prediction result.
[0013] In a third aspect, the technical solution of the present invention provides a terminal, including: A memory, configured to store a water conservancy data processing program based on an intelligent agent large model; A processor, configured to implement the steps of the water conservancy data processing method based on an intelligent agent large model as described in any one of the above when executing the water conservancy data processing program based on an intelligent agent large model.
[0014] Fourthly, the technical solution of the present invention provides a computer-readable storage medium, on which a water conservancy data processing program based on an agent large model is stored. When the water conservancy data processing program based on the agent large model is executed by a processor, the steps of the water conservancy data processing method based on the agent large model as described in any one of the above are implemented.
[0015] As can be seen from the above technical solutions, the present application has the following advantages: The user sends a data processing request, selects an agent large model according to the type of the data processing request, including a data management agent large model and a data prediction agent large model, and generates a corresponding water conservancy data management report or a water conservancy data prediction result according to the large model. The present invention processes water conservancy data management and prediction tasks by calling a pre-trained agent large model, reduces the workload of manually sorting, classifying, and analyzing data, and improves data processing efficiency; the agent large model can automatically extract features and discover rules from a large amount of water conservancy data, improves the accuracy and reliability of analyzing water conservancy data, and realizes the automation and intelligence of water conservancy data management and prediction. It can automatically call the corresponding agent large model for processing according to different types of requests without manual intervention, meeting the requirements of the water conservancy industry for intelligent, automated, and precise data processing. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solutions of the present application, the drawings required for description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0017] Figure 1 It is a schematic flowchart of a water conservancy data processing method based on an agent large model provided by an embodiment of the present invention.
[0018] Figure 2 It is a schematic block diagram of the structure of a water conservancy data processing system based on an agent large model provided by an embodiment of the present invention.
[0019] Figure 3 It is a schematic diagram of the structure of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0020] In order to make the application purpose, features, and advantages of the present application more obvious and understandable, the technical solutions protected by the present application will be clearly and completely described below by using specific embodiments and the accompanying drawings. Obviously, the embodiments described below are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.
[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this invention belongs. The terms used in the description of the present invention herein are for the purpose of describing specific embodiments only and are not intended to limit the present invention.
[0022] In view of the problem that the existing water conservancy data processing system fails to effectively integrate data management and data prediction functions, resulting in the inability to fully utilize the effectiveness of the intelligent agent large model when facing complex water conservancy data processing tasks and being difficult to meet the intelligent, automated, and precise requirements of the water conservancy industry for data processing, this application provides a water conservancy data processing method based on an intelligent agent large model, which acquires and parses a data processing request sent by a user, retrieves target water conservancy data from a database according to the parsing result and stores it in a buffer area; detects the type of the data processing request; if the data processing request type is a water conservancy data management type, calls a pre-trained data management intelligent agent large model to process the target water conservancy data in the buffer area and generates a water conservancy data management report; if the data processing request type is a water conservancy data prediction type, calls a pre-trained data prediction intelligent agent large model to process the target water conservancy data in the buffer area and generates a water conservancy data prediction result. The present invention improves data processing efficiency, accuracy, and reliability, and realizes the automation and intelligence of water conservancy data management and prediction, and can automatically call the corresponding intelligent agent large model for processing according to different types of requests, meeting the intelligent, automated, and precise requirements of the water conservancy industry for data processing.
[0023] The water conservancy data processing method of this application is implemented based on the intelligent water conservancy Internet of Things architecture of the SpringBoot and SpringAI frameworks and accesses the intelligent agent large model. This architecture is based on the SpringBoot framework of Java to build an extensible microservices architecture, integrates Spring AI, and accesses the intelligent agent large model, and realizes effective connection with various water conservancy Internet of Things devices, ensuring the ability to collect data such as water level, flow rate, temperature, and water quality in real time, and process the data formats of different devices, and realizes the intelligence and automation of water conservancy management based on the intelligent agent large model, providing functions such as real-time data collection, intelligent prediction, and dynamic report generation to support the rational utilization and management of water resources.
[0024] The intelligent water conservancy Internet of Things architecture includes the following hierarchical architectures.
[0025] (1) Presentation Layer: User interface: Built using React or Vue.js to provide user interaction.
[0026] API Interface: Provide RESTful APIs for the front-end to call.
[0027] (2) Service Layer: Business Logic Services: Implement functions such as data processing, intelligent prediction, and report generation. Specifically, there are the following 6 layers. 1) Controller: The controller is the entry point of the system, receives external requests, calls appropriate services for processing, provides front-end and device communication interfaces, and calls the service layer for processing.
[0028] 2) Service: The service layer contains the core business logic of the system and is responsible for handling business rules, data conversion, and business processes. For example, it is responsible for processing inbound and outbound data.
[0029] 3) Repository: The repository is responsible for interacting with the database and performing read and write operations on data.
[0030] 4) Entity: The entity layer defines the data models used in the system and maps to the table structures in the database.
[0031] 5) Integration: The integration layer is responsible for handling message passing.
[0032] 6) Configuration: The configuration layer contains the configuration information of the system, such as database connections, device connections, etc.
[0033] AI Service: Interact with the large language model of the intelligent agent.
[0034] (3) Data Access Layer: Data Storage: Use Spring Data JPA or MyBatis for database operations.
[0035] Data Model: Define data entities and data transfer objects (DTOs).
[0036] (4) IoT Layer: Device Management: Responsible for connecting to IoT devices and collecting data.
[0037] Data Acquisition Module: Real-time receive and process data from sensors.
[0038] (5) Infrastructure Layer: Message Queue: Use RabbitMQ or Kafka for asynchronous communication between services.
[0039] Monitoring and Logging: Use Spring Actuator, Prometheus, and Grafana for system monitoring.
[0040] (6)Security Layer: Authentication and Authorization: Use Spring Security to implement user authentication and permission management.
[0041] The business process based on the above intelligent water conservancy Internet of Things architecture is as follows.
[0042] (1)User Login and Authentication The user enters the username and password through the front-end interface.
[0043] The front-end sends the request to the authentication interface at the back-end.
[0044] The back-end verifies the user information, generates a JWT (JSON Web Token), and returns it to the front-end.
[0045] (2)Device Management and Data Acquisition The administrator registers new Internet of Things devices through the system interface.
[0046] The Internet of Things devices regularly send data (such as water level, flow rate, temperature, etc.) to the data acquisition service.
[0047] The data acquisition service receives the data, parses the format, and stores it in the database.
[0048] (3)Data Processing and Analysis The data processing service extracts the raw data from the database and performs cleaning and preprocessing.
[0049] Key features can be extracted to prepare for subsequent predictive analysis.
[0050] (4)Intelligent Analysis and Recommendations The user requests data analysis through the front-end, selects the time range and data type for analysis.
[0051] The data processing service sends the requested data to the API of the agent large model.
[0052] The agent large model performs data analysis, generates analysis results and recommendations, and returns them to the system.
[0053] The system stores the analysis results and recommendations in the database for the user to view.
[0054] (5) Intelligent Prediction The user requests intelligent prediction through the front end, selecting the time range and data type for analysis.
[0055] The data processing service sends the requested data to the API of the agent large model.
[0056] The agent large model returns the prediction results and stores them in the database.
[0057] (6) Report Generation The user requests to generate a report and selects the report type (such as daily report, weekly report, monthly report).
[0058] The report generation service extracts relevant data and prediction results from the database.
[0059] Uses a template engine to generate a report and returns it to the user.
[0060] (7) User Interaction and Feedback The user can query historical data, prediction results, and generated reports.
[0061] The user provides feedback on the system's prediction results and reports, and the system records the feedback for improvement.
[0062] (8) System Monitoring and Maintenance Use Spring Actuator, Prometheus, and Grafana to monitor system performance.
[0063] Record system operation logs and error logs for subsequent troubleshooting and maintenance.
[0064] In terms of technical implementation of the intelligent water conservancy Internet of Things architecture, Spring Boot, Spring AI, and Spring Security are used in the backend, React or Vue.js in the front end, MySQL or MongoDB in the database, RabbitMQ or Kafka in the message queue, and Spring Actuator, Prometheus, and Grafana in the monitoring tools. When implementing security, Spring Security is used to implement user authentication, support JWT authentication, and design role and permission management modules to ensure that users can only access the functions they are authorized to.
[0065] Figure 1 It is a schematic diagram of the process flow of a water conservancy data processing method based on an agent large model provided by an embodiment of the present invention. Among them, Figure 1The executing entity can be a water conservancy data processing system based on an agent large model. The water conservancy data processing method provided by the embodiments of the present invention is executed by a computer device. Correspondingly, the water conservancy data processing system based on the agent large model runs in the computer device. According to different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.
[0066] As Figure 1 shown, the method includes the following steps.
[0067] S1. Obtain and parse the data processing request sent by the user, and retrieve the target water conservancy data from the database according to the parsing result and store it in the buffer area.
[0068] In this step, the data processing request of the user is received and parsed, key information is extracted from it, the target water conservancy data is retrieved from the database based on this information, and then it is stored in the buffer area. By parsing the request, the required data can be accurately located, improving the pertinence of data acquisition. Storing the data in the buffer area can speed up the subsequent processing speed, reduce the data reading time, and improve the overall processing efficiency.
[0069] S2. Detect the type of the data processing request.
[0070] In this step, the parsed request is analyzed to determine whether it belongs to the water conservancy data management type or the water conservancy data prediction type. Clearly defining the specific type of the request provides a basis for subsequently calling the corresponding agent large model, ensuring the accuracy and efficiency of the data processing process, and avoiding resource waste and processing errors caused by incorrect model calls.
[0071] S3. If the data processing request type is the water conservancy data management type, call the pre-trained data management agent large model to process the target water conservancy data in the buffer area and generate a water conservancy data management report.
[0072] In this step, when it is determined that the request is of the water conservancy data management type, the pre-trained data management agent large model is used to process the target water conservancy data in the buffer area, and finally a water conservancy data management report is generated. With the powerful data processing ability of the agent large model, the automatic generation of the water conservancy data management report is realized. Compared with manual sorting and analysis, it can process data faster and more accurately, improve the generation efficiency and quality of the report, and provide a scientific basis for water conservancy data management.
[0073] S4. If the data processing request type is the water conservancy data prediction type, call the pre-trained data prediction agent large model to process the target water conservancy data in the buffer area and generate a water conservancy data prediction result.
[0074] If the request is for water conservancy data prediction, the pre-trained data prediction agent model is invoked to process the target water conservancy data in the cache and produce a prediction result. This agent model is used to discover data patterns and improve the accuracy and reliability of water conservancy data prediction.
[0075] The water conservancy data processing method based on the intelligent agent big model provided in this embodiment processes water conservancy data management and prediction tasks by calling a pre-trained intelligent agent big model, reducing the workload of manual sorting, classification and analysis of data, and improving data processing efficiency; the intelligent agent big model can automatically extract features and discover patterns from massive water conservancy data, improve the accuracy and reliability of water conservancy data analysis, and realize the automation and intelligence of water conservancy data management and prediction. It can automatically call the corresponding intelligent agent big model for processing according to different types of requests without manual intervention, and meet the water conservancy industry's needs for intelligent, automated and precise data processing.
[0076] Furthermore, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process in this embodiment, another water conservancy data processing method based on the intelligent agent large model is provided, which includes the following steps.
[0077] SS1, obtains and parses the data processing request sent by the user, and retrieves the target water conservancy data from the database and stores it in the cache area based on the parsing results.
[0078] SS1.1, obtain and parse the data processing request sent by the user.
[0079] In some optional implementations, the program starts a network interface and continuously listens for data processing requests sent by users. When a user initiates a request, the server receives the request data, which may be included in the request body or request parameters.
[0080] SS1.2. Obtain the target time period and target water conservancy data type identifier from the parsing results.
[0081] In the parsed request data, search for fields related to the target period and target water conservancy data type identifier. The names of these fields can be predefined in the interface documentation. For example, if the request data is a JSON object, it may contain the "target_period" and "target_data_type" fields, indicating the target period and target water conservancy data type identifier, respectively.
[0082] SS1.3, construct SQL statements based on the target time period and target water conservancy data type identifier.
[0083] Determine the database table name to be queried and the fields to be queried according to the target water conservancy data type identifier. For example, if the target water conservancy data type is water level data, the corresponding table name may be water_level_data, and the fields to be queried may include timestamp (timestamp), water_level (water level value), etc.
[0084] Use the target time period to construct the time range condition for the SQL query. For example, if the target time period is from start_time to end_time, the condition timestamp BETWEEN'start_time' AND 'end_time' can be constructed. Then combine the table name, fields, and query conditions into a complete SQL query statement.
[0085] SS1.4, Retrieve the target water conservancy data from the database based on the SQL statement.
[0086] Use the established database connection object to execute the constructed SQL statement, extract the query results from the database, and convert them into a data structure that can be processed internally by the program, such as a list or a dictionary.
[0087] SS1.5, Store the target water conservancy data in the buffer area.
[0088] Store the target water conservancy data retrieved from the database in the cache system in a suitable format (such as a JSON string). The key-value pair storage function of the cache system can be used to set a unique key for the data. In some alternative embodiments, set a reasonable expiration time for the data stored in the cache to ensure the timeliness of the data. When the data expires, it will be retrieved from the database again the next time a request is made.
[0089] SS2, Detect the data processing request type.
[0090] SS2.1, Retrieve the data value of the target field from the parsing result.
[0091] SS2.2, Determine the data processing request type based on the data value.
[0092] Establish a mapping relationship between the data value and the data processing request type. For example, define "management" corresponding to the water conservancy data management type and "prediction" corresponding to the water conservancy data prediction type. A dictionary can be used to implement this mapping, such as request_type_mapping = {"management": "water conservancy data management type", "prediction": "water conservancy data prediction type"}. Match the data value of the target field extracted in step SS2.1 with the defined mapping relationship.
[0093] If the data value exists in the key of the mapping relationship, the corresponding request type is determined according to the mapping relationship. For example, if the extracted data value is "management", it is determined that the request type is the "water conservancy data management type".
[0094] SS3, if the data processing request type is the water conservancy data management type, call the pre-trained data management intelligent agent large model to process the target water conservancy data in the buffer area and generate a water conservancy data management report.
[0095] It should be noted that the pre-trained data management intelligent agent large model specifically includes the following steps.
[0096] Step 1, obtain several groups of historical water conservancy data and corresponding prediction target data.
[0097] The types of water conservancy data required for training the data management intelligent agent large model include water level, flow rate, rainfall, evaporation, water quality indicators, etc., as well as the corresponding time range and space range. [[ID=X]]
[0098] Connect to the database storing historical water conservancy data, and extract several groups of required historical water conservancy data and corresponding prediction target data through SQL query statements. If the data is stored in files (such as CSV, Excel, etc.), use data reading tools or file operation functions of programming languages to import the data into the data processing environment.
[0099] Step 2, use a clustering algorithm to cluster several groups of historical water conservancy data to generate a clustering result, and label a data pattern for each clustering result.
[0100] Select variables that can reflect data characteristics from historical water conservancy data as clustering features, such as the water level change rate, the flow rate fluctuation range, the cumulative rainfall value, etc.
[0101] Perform normalization processing on the clustering feature data to eliminate the dimensional differences between different features and make the data within the same scale range. If the data dimension is high, dimensionality reduction techniques (such as principal component analysis PCA) can be used to reduce the data dimension and reduce the computational complexity.
[0102] Select a clustering algorithm. In some alternative implementation manners, the K-Means clustering algorithm is selected. Determine the algorithm parameters according to the data characteristics and clustering requirements, including the number of clustering categories K value in the K-Means algorithm.
[0103] Input the preprocessed data into the clustering algorithm, run the algorithm to cluster the data, and obtain the clustering results, including the clustering categories to which each data sample belongs. Based on the clustering results, label a representative data pattern for each clustering category. The labeling of the data pattern can be based on domain knowledge and the analysis of the clustering results, such as "rapid increase pattern", "slow decline pattern", "stable fluctuation pattern", etc.
[0104] Step 3: Construct a training dataset from the historical water conservancy data and the corresponding predicted target data for each group of clusters.
[0105] Group the historical water conservancy data and the corresponding predicted target data according to the clustering categories to form multiple training data subsets.
[0106] Perform feature engineering on the water conservancy data in each training data subset, extract feature variables that are helpful for model training, and perform feature encoding and transformation. Generate the corresponding labels or target variables according to the predicted target data for supervising the training process of the model.
[0107] Divide each training data subset into a training set, a validation set, and a test set, and allocate the data samples according to a certain ratio (such as 7:2:1) to ensure that the model can fully learn and validate during the training process.
[0108] Step 4: Use the training dataset to train the teacher model and the knowledge-distilled student model of the data prediction agent large model.
[0109] In some alternative embodiments, the teacher model of the data prediction agent large model uses the Manus agent large model.
[0110] Step 4.1: Train the teacher model.
[0111] Normalize the water conservancy data in the training dataset to eliminate the dimensional differences between different features and make the data within the same scale range. According to the input requirements of the Manus agent large model, perform format conversion and preprocessing on the data, such as converting the data into JSON format, etc.
[0112] Input the preprocessed training dataset into the Manus agent large model. Use an optimization algorithm (such as Adam) to fine-tune the model to adapt to the water conservancy data prediction task. During the fine-tuning process, the performance of the model can be optimized by adjusting hyperparameters such as the learning rate and batch size.
[0113] Regularly evaluate the performance of the model on the validation set, and adjust the model's structure and hyperparameters according to the evaluation results to avoid overfitting or underfitting. Evaluate the prediction performance of the teacher model (Manus agent large model) on the test set, and calculate prediction accuracy metrics (such as root mean square error RMSE, mean absolute error MAE, coefficient of determination R², etc.).
[0114] Step 4.2, Train the student model.
[0115] Select a student model architecture with a relatively simple structure, such as a lightweight neural network (such as MobileNet, EfficientNet, etc.) or other suitable machine learning models.
[0116] Use the output of the teacher model (Manus agent large model) (such as prediction results, intermediate layer feature representations) as soft labels, and input them into the student model together with the hard labels (actual prediction target data) of the training dataset. Use a knowledge distillation loss function to combine the soft labels and hard labels to train the knowledge-distilled student model. The knowledge distillation loss function can be a weighted sum of the hard label loss function and the soft label loss function.
[0117] During the training process, minimize the knowledge distillation loss function through an optimization algorithm (such as Adam) to update the parameters of the student model. Evaluate the performance of the student model on the validation set and the test set, and calculate metrics such as prediction accuracy metrics and processing time. According to the evaluation results, adjust the structure and hyperparameters of the student model to optimize the performance of the student model, so that it has a faster processing speed while ensuring a certain prediction accuracy.
[0118] Step 5, According to the training results of the teacher model and the student model for each type of training dataset, assign an appropriate data pattern to the teacher model and the student model.
[0119] Step 5.1, Obtain the prediction accuracy and processing time of the teacher model and the student model.
[0120] Step 5.2, Perform a weighted sum of the prediction accuracy and processing time to obtain the prediction evaluation scores of the teacher model and the student model.
[0121] Step 5.3, Match and associate the model with the larger prediction evaluation score with the current data pattern.
[0122] Specifically, calculate the prediction accuracy metrics of the teacher model and the student model on the test set, such as root mean square error (RMSE), mean absolute error (MAE), coefficient of determination (R²), etc. Record the processing time of the teacher model and the student model for each type of training dataset, including model inference time and data preprocessing time.
[0123] According to the importance weights of prediction accuracy and processing time, the prediction accuracy and processing time are weighted and summed to obtain the comprehensive evaluation scores of the teacher model and the student model. It should be noted that each training dataset corresponds to a teacher model evaluation score and a student model evaluation score.
[0124] According to the comprehensive evaluation scores, the teacher model and the student model are matched and associated with the corresponding data patterns. For each data pattern, the model with the highest comprehensive evaluation score is selected as the adapted model.
[0125] Exemplarily, the rapid rise pattern matches the teacher model, and the slow decline pattern and the stable fluctuation pattern match the student model.
[0126] Based on the above-trained teacher model and student model, the pre-trained data prediction intelligent agent large model is called to process the target water conservancy data in the buffer area to generate water conservancy data prediction results, which specifically include the following steps.
[0127] SS3.1, Retrieve the target water conservancy data from the buffer area.
[0128] According to the unique identifier or relevant parameters of the data processing request, quickly locate and lock the storage location of the target water conservancy data in the buffer area. Execute the data reading command to load the target water conservancy data from the buffer area into the data processing memory space to ensure the integrity and accuracy of the data.
[0129] SS3.2, Use the clustering algorithm to cluster the target water conservancy data to obtain the clustering result, and match the data pattern according to the clustering result.
[0130] Perform necessary preprocessing on the target water conservancy data read from the buffer area, including data cleaning, outlier removal, normalization and other operations to ensure the quality and consistency of the data. Extract feature vectors from the preprocessed water conservancy data, and these feature vectors should be consistent with the feature vectors used when training the clustering model so that the clustering model can accurately identify the data pattern. Call the pre-trained clustering algorithm model, and input the extracted feature vectors into the clustering model to obtain the clustering result.
[0131] According to the clustering result, match the target water conservancy data with the predefined data patterns. For example, if the clustering result indicates that the data belongs to the "rapid rise pattern", then the current data pattern is determined to be the "rapid rise pattern".
[0132] SS3.3, Filter and associate the relevant teacher model or student model according to the matched data pattern.
[0133] Query the corresponding model matching rules according to the matched data pattern. These rules define whether to use the teacher model or the student model under specific data patterns.
[0134] According to the queried model matching rules, select the teacher model or student model adapted to the current data pattern from the model library and load it into the memory to prepare for subsequent data processing.
[0135] SS3.4, Use the associated teacher model or student model to process the target water conservancy data in the buffer area to generate the prediction result of water conservancy data.
[0136] According to the input requirements of the selected model (teacher model or student model), convert and organize the format of the target water conservancy data to form the input data format that the model can directly process. Input the prepared input data into the loaded model to start the model inference process. The model will calculate and analyze the input data according to its internal parameters and structure to generate the prediction result of water conservancy data.
[0137] Perform necessary post-processing operations on the prediction result output by the model, including data format conversion, result interpretation, etc., to make it meet the requirements of the actual application scenario and present it to the user in an intuitive and understandable way.
[0138] SS4, If the data processing request type is the water conservancy data prediction type, call the pre-trained data prediction intelligent agent large model to process the target water conservancy data in the buffer area to generate the prediction result of water conservancy data.
[0139] SS4.1, Retrieve the water conservancy data management report template.
[0140] Read the template file from the database or file system according to the storage location. The template file contains predefined formats, layouts, and placeholders for generating the final management report.
[0141] SS4.2, Build prompts based on the target water conservancy data in the buffer area and the water conservancy data management report template.
[0142] Parse the read template file, identify the placeholders and dynamic content areas in it, and determine the specific positions and format requirements for filling data. Extract the fields and values corresponding to the template placeholders from the target water conservancy data in the buffer area, such as key information like water level, flow rate, timestamp, etc. Integrate the extracted water conservancy data into prompts according to the prompt structure and requirements in the template. The prompts can be natural language descriptions, data labels, or other forms of indication information for guiding the model to generate report content.
[0143] SS4.3, Input the prompts, the target water conservancy data in the buffer area, and the water conservancy data management report template into the pre-trained data management intelligent agent large model to generate the water conservancy data management report.
[0144] Integrate the generated prompt words, target water conservancy data in the buffer, and the read report template into an input data packet to ensure that the data format and structure meet the input requirements of the data management agent large model. Send the input data packet to the pre-trained data management agent large model through an application programming interface (API) or other call methods. The data management agent large model performs inference and generation operations based on the input prompt words, water conservancy data, and report template, and automatically generates a water conservancy data management report.
[0145] In some alternative embodiments, the data management agent large model can be the Manus agent large model.
[0146] The embodiments of a water conservancy data processing method based on an agent large model have been described in detail above. Based on the water conservancy data processing method based on the agent large model described in the above embodiments, the embodiments of the present invention also provide a water conservancy data processing device corresponding to this method.
[0147] Figure 2 This is a schematic block diagram of the structure of a water conservancy data processing system based on an agent large model provided by the embodiments of the present invention. In this embodiment, the water conservancy data processing system 200 based on the agent large model can be divided into multiple functional modules according to the functions it performs. The modules referred to in the present invention refer to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, and are stored in the memory.
[0148] The data processing request acquisition module 210 is used to acquire and parse the data processing request sent by the user, and retrieve the target water conservancy data from the database according to the parsing result and store it in the buffer.
[0149] The data processing request type detection module 220 is used to detect the data processing request type.
[0150] The water conservancy data management report generation module 230 is used to, if the data processing request type is a water conservancy data management type, call the pre-trained data management agent large model to process the target water conservancy data in the buffer and generate a water conservancy data management report.
[0151] The water conservancy data prediction result generation module 240 is used to, if the data processing request type is a water conservancy data prediction type, call the pre-trained data prediction agent large model to process the target water conservancy data in the buffer and generate a water conservancy data prediction result.
[0152] The water conservancy data processing system based on the agent large model in this embodiment is used to implement the aforementioned water conservancy data processing method based on the agent large model. Therefore, the specific implementation manners in this system can be seen in the embodiment part of the water conservancy data processing method based on the agent large model in the foregoing text. Therefore, its specific implementation manners can be referred to the descriptions of the corresponding various part embodiments, and will not be elaborated here.
[0153] In addition, since the water conservancy data processing system based on the agent large model in this embodiment is used to implement the aforementioned water conservancy data processing method based on the agent large model, its functions correspond to those of the above method, and will not be elaborated here.
[0154] Figure 3 FIG. 7 is a schematic structural diagram of a terminal 300 provided by an embodiment of the present invention, including: a processor 310, a memory 320, and a communication unit 330. When the processor 310 is used to implement the water conservancy data processing program stored in the memory 320, the flow steps of the above-mentioned embodiment of the water conservancy data processing method based on the agent large model are implemented.
[0155] The terminal 300 includes a processor 310, a memory 320, and a communication unit 330. These components communicate through one or more buses. Those skilled in the art can understand that the structure of the server shown in the figure does not constitute a limitation on the present invention. It can be a bus structure, a star structure, and can also include more or fewer components than shown in the figure, or combine some components, or different component arrangements.
[0156] Among them, the memory 320 can be used to store the execution instructions of the processor 310. The memory 320 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the terminal 300 can execute some or all of the steps in the above method embodiments.
[0157] The processor 310 is the control center of the storage terminal, connecting various parts of the entire electronic terminal through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 320, and by invoking data stored in the memory, it performs various functions of the electronic terminal and / or processes data. The processor may be composed of an integrated circuit (IC), for example, it may be composed of a single packaged IC, or it may be composed of multiple packaged ICs with the same or different functions connected together. For example, the processor 310 may include only a central processing unit (CPU). In the embodiment of the present invention, the CPU may be a single arithmetic core or may include multiple arithmetic cores.
[0158] The communication unit 330 is used to establish a communication channel so that the storage terminal can communicate with other terminals. It receives user data sent by other terminals or sends user data to other terminals.
[0159] The present invention also provides a computer storage medium, and the storage medium mentioned here may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), etc.
[0160] The present invention also provides a computer storage medium, and the storage medium mentioned here may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), etc.
[0161] The computer storage medium stores a water conservancy data processing program based on the agent large model. When the water conservancy data processing program based on the agent large model is executed by the processor, it implements the process steps of the above-mentioned embodiment of the water conservancy data processing method based on the agent large model.
[0162] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solutions in the embodiments of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc., which can store program codes, and includes several instructions to enable a computer terminal (which can be a personal computer, a server, or a second terminal, a network terminal, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.
[0163] In several embodiments provided by the present invention, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.
[0164] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0165] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.
[0166] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for processing water conservancy data based on an agent large model, characterized in that It includes the following steps: Obtain and parse the data processing request sent by the user, and retrieve the target water conservancy data from the database according to the parsing result and store it in the buffer; Detect the type of data processing request; If the data processing request type is the water conservancy data management type, call the pre-trained data management intelligent agent large model to process the target water conservancy data in the buffer and generate a water conservancy data management report; If the data processing request type is the water conservancy data prediction type, call the pre-trained data prediction intelligent agent large model to process the target water conservancy data in the buffer and generate a water conservancy data prediction result.
2. The water conservancy data processing method based on the intelligent agent large model according to claim 1, wherein This method also includes the step of training the data prediction intelligent agent large model, specifically including: Obtain several groups of historical water conservancy data and corresponding predicted target data; Use the clustering algorithm to cluster several groups of historical water conservancy data to generate a clustering result, and label each clustering result with a data pattern; Construct a training data set from each group of clustered historical water conservancy data and the corresponding predicted target data; Use the training data set to train the teacher model and the knowledge distillation student model of the data prediction intelligent agent large model; According to the training results of the teacher model and the student model for each type of training data set, assign a suitable data pattern to the teacher model and the student model.
3. The water conservancy data processing method based on the intelligent agent large model according to claim 2, wherein According to the training results of the teacher model and the student model for each type of training data set, assign a suitable data pattern to the teacher model and the student model, specifically including: Obtain the prediction accuracy and processing time of the teacher model and the student model; Perform a weighted sum of the prediction accuracy and processing time to obtain the prediction evaluation scores of the teacher model and the student model; Match and associate the model with the larger prediction evaluation score with the current data pattern.
4. The water conservancy data processing method based on the agent large model according to claim 2 or 3, characterized in that, Call the pre-trained data prediction intelligent agent large model to process the target water conservancy data in the buffer and generate a water conservancy data prediction result, specifically including: Retrieve the target water conservancy data from the buffer; Use the clustering algorithm to cluster the target water conservancy data to obtain a clustering result, and match the data pattern according to the clustering result; Filter and associate the teacher model or the student model according to the matched data pattern; Use the associated teacher model or student model to process the target water conservancy data in the buffer and generate a water conservancy data prediction result.
5. The method for processing water conservancy data based on the intelligent agent large model according to claim 1, wherein, Call the pre-trained data management intelligent agent large model to process the target water conservancy data in the buffer and generate a water conservancy data management report, specifically including: Retrieve the water conservancy data management report template; Construct a prompt based on the target water conservancy data in the buffer and the water conservancy data management report template; Input the prompt, the target water conservancy data in the buffer, and the water conservancy data management report template into the pre-trained data management intelligent agent large model to generate a water conservancy data management report.
6. The water conservancy data processing method based on the intelligent agent large model according to claim 1, wherein Retrieve the target water conservancy data from the database according to the parsing result and store it in the buffer, specifically including: Obtain the target time period and the target water conservancy data type identifier from the parsing result; Construct an SQL statement according to the target time period and the target water conservancy data type identifier; Retrieve the target water conservancy data from the database based on the SQL statement; Store the target water conservancy data in the buffer.
7. The method for processing water conservancy data based on the intelligent agent large model according to claim 6, wherein, Detect the type of data processing request, specifically including: Retrieve the data value of the target field from the parsing result; Judge the type of data processing request according to the data value.
8. A water conservancy data processing system based on an intelligent agent large model, characterized in that, Including: A data processing request acquisition module, configured to acquire and parse a data processing request sent by a user, and retrieve target water conservancy data from a database according to the parsing result and store it in a buffer; A data processing request type detection module, configured to detect the type of data processing request; A water conservancy data management report generation module, configured to, if the data processing request type is a water conservancy data management type, call a pre-trained data management agent large model to process the target water conservancy data in the buffer and generate a water conservancy data management report; A water conservancy data prediction result generation module, configured to, if the data processing request type is a water conservancy data prediction type, call a pre-trained data prediction agent large model to process the target water conservancy data in the buffer and generate a water conservancy data prediction result.
9. A terminal, characterized in that, Including: A memory, configured to store a water conservancy data processing program based on an agent large model; A processor, configured to implement the steps of the water conservancy data processing method based on an agent large model according to any one of claims 1 to 7 when executing the water conservancy data processing program based on an agent large model.
10. A computer-readable storage medium, characterized in that, A water conservancy data processing program based on an agent large model is stored on the readable storage medium, and when the water conservancy data processing program based on an agent large model is executed by a processor, the steps of the water conservancy data processing method based on an agent large model according to any one of claims 1 to 7 are implemented.