Power question query method based on multi-model fusion

Abstract

The invention relates to the related field of power systems, in particular to a power question query method based on multi-model fusion, which realizes accurate adaptation to each task without large-scale adjustment of the whole model by setting specific LoRA parameter layers for different tasks to perform fixed-parameter fine adjustment. According to the strategy, video memory occupation is remarkably reduced, so that the method disclosed by the invention can be operated on a machine with a smaller video memory, and the hardware range of model deployment is expanded; and finally, the LoRA parameter layers of the tasks are combined, so that a unified multi-task model is formed. The fusion model can meet the requirements of multiple tasks at the same time, the universality and flexibility of the model are improved, and dependence on a single task model is reduced; according to the method, overall task data is further adopted to carry out combined secondary fine tuning on the combined model. The step ensures that the accuracy of the model in processing the whole query task is improved, and the reliability and accuracy of the model in practical application are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of power systems, and in particular to a method for querying power data based on multi-model fusion. Background Art

[0002] In the current power system, data query is a crucial link, which involves multiple aspects such as power production, transmission, distribution and consumption. Traditional data query methods mainly rely on manual operations and traditional database query technology. At present, the use of artificial intelligence technology to assist data query has become the main improvement method. Among these methods, disassembling data query tasks and using multiple modules for division of labor has become a more feasible technical solution. At the same time, with the application of large models, fine-tuning large models for each task to obtain separate task large models to solve the accuracy of a single task has become the current mainstream technical solution, but the existing technology has the following defects:

[0003] 1. Each model needs to be deployed separately, resulting in a large amount of video memory usage and cannot be applied under some conditions with small video memory.

[0004] 2. The overall query task is not considered during fine-tuning training of a single model, resulting in a high accuracy rate for a single model, but a lower query accuracy rate after overall collusion.

[0005] 3. The number of deployment task calls is small, resulting in a certain amount of resource waste. Summary of the invention

[0006] The purpose of the present invention is to provide a method for querying electric power data based on multi-model fusion to solve the problems raised in the above background technology.

[0007] To achieve the above object, the present invention provides the following technical solution: a method for querying electric power data based on multi-model fusion, comprising the following steps:

[0008] Step S1, using a preprocessing module for power data, preprocessing the single-task supervised data and multi-task joint supervised data of the power module, wherein the preprocessing of the preprocessing module includes data cleaning, data deduplication, and data feature extraction;

[0009] Step S2: According to step S1, supervised data processed for different single tasks are fine-tuned by selecting a base model for supervised model training, and specifically, LORA's fine-tuning technology is used for SFT training;

[0010] After the fine-tuning training is completed in step S3 and step S2, the supervised fine-tuning training parameters fine-tuned for different single tasks are obtained, and then the trained multiple supervised fine-tuning training parameters are integrated into the model to perform the first stage of joint training, that is, the base model is used again to perform joint fine-tuning of the multi-task model;

[0011] Step S4: After completing the fusion of the multi-model parameters in step S3, further joint fine-tuning is performed on the fusion model, and two-stage joint training is performed again to optimize the overall performance of the model when processing cross-task queries;

[0012] Step S5, completing the two-stage joint training in step S4 to obtain the final multi-task joint large model;

[0013] Step S6: The multi-task joint large model obtained in step S5 is deployed and scheduled through the model deployment and scheduling system.

[0014] Preferably, in step S1, the preprocessing module is responsible for receiving the raw data from the power system, and cleaning and standardizing the raw data to ensure data quality. The cleaning process includes removing invalid or erroneous data records, and standardizing the data from different sources into a unified format; the preprocessing module extracts features from the preprocessed raw data. Specifically, the preprocessing module uses statistical analysis and pattern recognition technology to extract features preset for the power data query task from the raw data. The extracted features are used to generate single-task supervised training data and multi-task joint supervised training data to provide input for subsequent model training.

[0015] Preferably, the first stage of joint training in step S3 is fine-tuning of individual task models, that is, the large model is fine-tuned individually for each specific query task in the power system, including load forecasting, fault detection, and energy management, specifically, multiple supervised fine-tuning training parameters that have been trained are fine-tuned so that they can more accurately capture the data patterns and features related to the specific task, wherein the fine-tuning technology includes but is not limited to transfer learning.

[0016] Preferably, the second stage joint training in step S4 is a multi-task model joint fine-tuning, specifically integrating multiple supervised fine-tuning training parameters after the first stage fine-tuning, wherein the multi-task model joint fine-tuning adopts LoRA technology to achieve effective fusion of model parameters. LoRA performs low-rank decomposition on the weight matrix of the multi-task model and introduces additional low-rank matrices to adjust the multi-task model parameters, that is, adaptive adjustment of multiple tasks is achieved without significantly increasing the size of the multi-task model.

[0017] Preferably, the model deployment scheduling system in step S6 is built based on k8s and docker technologies. The deployment scheduling system dynamically adjusts the deployment and resource allocation of the model according to the current query load and resource usage. The deployment scheduling system first collects the system's performance indicators and resource usage through monitoring components, and then the scheduler decides how to allocate resources to meet the query requirements based on this information and preset strategies. Specifically, when the load of the query task increases, the system automatically expands the corresponding computing resources, or reschedules the model to balance the load.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] Optimizing video memory usage: The present invention achieves precise adaptation to each task by setting specific LoRA parameter layers for different tasks and fine-tuning parameters without large-scale adjustments to the entire model. This strategy significantly reduces video memory usage, allowing the method of the present invention to run on machines with smaller video memory, expanding the hardware range for model deployment.

[0020] Enhanced multi-task adaptability: By finally merging the LoRA parameter layers of each task, the present invention forms a unified multi-task model. This fusion model can meet the needs of multiple tasks at the same time, improve the versatility and flexibility of the model, and reduce the dependence on a single task model.

[0021] Improvement of overall query task accuracy: After fine-tuning the multi-task large model, the present invention further uses the overall task data to perform joint secondary fine-tuning on the merged model. This step ensures that the accuracy of the model is improved when processing the overall query task, and enhances the reliability and accuracy of the model in practical applications.

[0022] Improved machine utilization: The model deployment and scheduling system of the present invention achieves unified calling and parallel processing by distributing the merged multi-task large model to multiple graphics cards. This deployment method improves machine utilization, optimizes the allocation of computing resources, and thus improves the operating efficiency of the entire system.

[0023] Reduce resource waste: Through intelligent resource management and scheduling, the present invention avoids over-allocation of resources on a single task and reduces resource waste. The system can dynamically adjust resource allocation according to real-time query requirements and resource usage to achieve optimal resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the structure of the method flow of the present invention;

[0025] Figure 2 Schematic diagram of the multi-task model joint fusion training framework structure of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] See also Figure 1-2 The present invention provides a technical solution: a method for querying electric power data based on multi-model fusion, comprising the following steps:

[0028] Step S1, using a preprocessing module for power data, preprocessing the single-task supervised data and multi-task joint supervised data of the power module, wherein the preprocessing of the preprocessing module includes data cleaning, data deduplication, and data feature extraction;

[0029] The preprocessing module in step S1 is responsible for receiving the raw data from the power system, and cleaning and standardizing the raw data to ensure data quality. The cleaning process includes removing invalid or erroneous data records, and standardizing the data from different sources into a unified format. The preprocessing module extracts features from the preprocessed raw data. Specifically, the preprocessing module uses statistical analysis and pattern recognition technology to extract features preset for the power data query task from the raw data. The extracted features are used to generate single-task supervised training data and multi-task joint supervised training data to provide input for subsequent model training.

[0030] Step S2: According to step S1, supervised data processed for different single tasks are fine-tuned by selecting a base model for supervised model training, and specifically, LORA's fine-tuning technology is used for SFT training;

[0031] After the fine-tuning training is completed in step S3 and step S2, the supervised fine-tuning training parameters fine-tuned for different single tasks are obtained, and then the trained multiple supervised fine-tuning training parameters are integrated into the model to perform the first stage of joint training, that is, the base model is used again to perform joint fine-tuning of the multi-task model;

[0032] The first stage of joint training in step S3 is fine-tuning of individual task models, that is, fine-tuning the large model individually for each specific query task in the power system, including load forecasting, fault detection, and energy management. Specifically, multiple supervised fine-tuning training parameters that have been trained are fine-tuned so that they can more accurately capture the data patterns and features related to the specific task, and the fine-tuning technology includes but is not limited to transfer learning.

[0033] Step S4: After completing the fusion of the multi-model parameters in step S3, further joint fine-tuning is performed on the fusion model, and two-stage joint training is performed again to optimize the overall performance of the model when processing cross-task queries;

[0034] The second stage joint training in step S4 is the joint fine-tuning of the multi-task model, which specifically integrates the multiple supervised fine-tuning training parameters after the fine-tuning in the first stage. The joint fine-tuning of the multi-task model adopts the LoRA technology to achieve effective fusion of model parameters. LoRA performs low-rank decomposition on the weight matrix of the multi-task model and introduces an additional low-rank matrix to adjust the multi-task model parameters, that is, to achieve adaptive adjustment of multiple tasks without significantly increasing the size of the multi-task model.

[0035] Step S5, completing the two-stage joint training in step S4 to obtain the final multi-task joint large model;

[0036] Step S6: The multi-task joint large model obtained in step S5 is deployed and scheduled through the model deployment and scheduling system.

[0037] In step S6, the model deployment and scheduling system is built based on k8s and docker technologies. The deployment and scheduling system dynamically adjusts the deployment and resource allocation of the model according to the current query load and resource usage. The deployment and scheduling system first collects the system's performance indicators and resource usage through monitoring components, and then the scheduler decides how to allocate resources to meet the query requirements based on this information and preset strategies. Specifically, when the load of the query task increases, the system automatically expands the corresponding computing resources or reschedules the model to balance the load.

[0038] In the above description, fine-tuning of individual task models and joint fine-tuning of multi-task models constitute a multi-task model joint fusion training framework, in which a unified multi-task fusion model is constructed based on the merging of LoRA parameters. This fusion model not only retains the expertise of each individual task model, but also learns common features across tasks through shared representations. Next, we further perform joint fine-tuning on this fusion model to optimize the overall performance of the model when processing cross-task queries;

[0039] In terms of fusion methods, a hierarchical fusion strategy is adopted. First, at the feature level, through feature selection and feature fusion technology, the relevant features of each task are integrated to enhance the model's understanding of the query task. Secondly, at the model level, the model parameters are adjusted through LoRA technology to achieve effective collaboration between models. Finally, at the decision-making level, a multi-task decision-making mechanism is designed, which can dynamically adjust the output weights of different task models according to the complexity and urgency of the query task to generate the optimal query result. Through this multi-level and multi-dimensional fusion method, the multi-task model joint fusion training framework of the present invention not only improves the efficiency and accuracy of power data query, but also significantly reduces the waste of resources through intelligent resource management and scheduling, and achieves significant improvement over the existing technology.

[0040] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for querying electric power data based on multi-model fusion, characterized in that: The following steps are involved: Step S1, using a preprocessing module for power data, preprocessing the single-task supervised data and multi-task joint supervised data of the power module, wherein the preprocessing of the preprocessing module includes data cleaning, data deduplication, and data feature extraction; Step S2: According to step S1, supervised data processed for different single tasks are fine-tuned by selecting a base model for supervised model training, and specifically, LORA's fine-tuning technology is used for SFT training; After the fine-tuning training is completed in step S3 and step S2, the supervised fine-tuning training parameters fine-tuned for different single tasks are obtained, and then the trained multiple supervised fine-tuning training parameters are integrated into the model to perform the first stage of joint training, that is, the base model is used again to perform joint fine-tuning of the multi-task model; Step S4: After completing the fusion of the multi-model parameters in step S3, further joint fine-tuning is performed on the fusion model, and two-stage joint training is performed again to optimize the overall performance of the model when processing cross-task queries; Step S5, completing the two-stage joint training in step S4 to obtain the final multi-task joint large model; Step S6: The multi-task joint large model obtained in step S5 is deployed and scheduled through the model deployment and scheduling system.

2. According to claim 1, a method for querying electric power data based on multi-model fusion is characterized in that: In step S1, the preprocessing module is responsible for receiving the raw data from the power system, and cleaning and standardizing the raw data to ensure data quality. The cleaning process includes removing invalid or erroneous data records, and standardizing data from different sources into a unified format. The preprocessing module extracts features from the preprocessed raw data. The specific preprocessing module uses statistical analysis and pattern recognition technology to extract the preset features for the power data query task from the raw data. The extracted features are used to generate single-task supervised training data and multi-task joint supervised training data to provide input for subsequent model training.

3. According to claim 1, a method for querying electric power data based on multi-model fusion is characterized in that: The first stage of joint training in step S3 is fine-tuning of individual task models, that is, fine-tuning the large model individually for each specific query task in the power system, including load forecasting, fault detection, and energy management. Specifically, multiple supervised fine-tuning training parameters that have been trained are fine-tuned so that they can more accurately capture the data patterns and features related to the specific task, wherein the fine-tuning technology includes but is not limited to transfer learning.

4. The method for querying electric power data based on multi-model fusion according to claim 1 is characterized in that: The second stage joint training in step S4 is the joint fine-tuning of the multi-task model, which specifically integrates the multiple supervised fine-tuning training parameters after the fine-tuning in the first stage. The joint fine-tuning of the multi-task model adopts the LoRA technology to achieve effective fusion of model parameters. LoRA performs low-rank decomposition on the weight matrix of the multi-task model and introduces an additional low-rank matrix to adjust the multi-task model parameters, that is, to achieve adaptive adjustment of multiple tasks without significantly increasing the size of the multi-task model.

5. The method for querying electric power data based on multi-model fusion according to claim 1 is characterized in that: In step S6, the model deployment and scheduling system is built based on k8s and docker technologies. The deployment and scheduling system dynamically adjusts the deployment and resource allocation of the model according to the current query load and resource usage. The deployment and scheduling system first collects the system's performance indicators and resource usage through monitoring components, and then the scheduler decides how to allocate resources to meet the query requirements based on this information and preset strategies. Specifically, when the load of the query task increases, the system automatically expands the corresponding computing resources or reschedules the model to balance the load.

Images