Large model adaptation method and device, equipment and storage medium

Abstract

The invention discloses a large model adaptation method and device, equipment and a storage medium. The big model adaptation method comprises the steps of performing parameter fine tuning on a pre-trained big model based on historical main network operation instruction data to obtain a fine-tuned main network big model; executing the major network operation task by adopting the fine-tuning major network large model to obtain an output result of the fine-tuning major network large model and feedback data based on the output result; and based on the output result and the feedback data, performing reinforcement learning on the fine-tuning major network large model to obtain a major network operation task large model. According to the technical scheme provided by the embodiment of the invention, the understanding capability of a large model on a power grid instruction can be improved.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a large model adaptation method, device, equipment and storage medium. Background Art

[0002] Large Language Model (LLM), hereinafter referred to as "large model", refers to a deep learning model with a large number of parameters and complex structure used in the fields of natural language processing and machine learning. Large models are currently widely used in various fields.

[0003] Large models are good at text processing, but they are prone to deviations when directly solving specific tasks. For example, large models may make mistakes in understanding or executing tasks in the main grid of the power grid. Therefore, how to improve the large language model's understanding and execution of the main grid's operation task instructions is very important for improving the accuracy and efficiency of the main grid dispatching task execution. Summary of the invention

[0004] The present invention provides a large model adaptation method, device, equipment and storage medium to solve the problem of low execution accuracy of large models in power grid main network scheduling tasks.

[0005] According to one aspect of the present invention, a large model adaptation method is provided, comprising:

[0006] Based on the historical main network operation instruction data, fine-tune the parameters of the pre-trained large model to obtain the fine-tuned main network large model;

[0007] The fine-tuning main network large model is used to execute the main network operation task, and an output result of the fine-tuning main network large model and feedback data based on the output result are obtained;

[0008] Based on the output results and the feedback data, reinforcement learning is performed on the fine-tuning main network large model to obtain a main network operation task large model.

[0009] According to another aspect of the present invention, there is provided a large model adaptation device, comprising:

[0010] The parameter fine-tuning module is used to fine-tune the parameters of the pre-trained large model based on the historical main network operation instruction data to obtain the fine-tuned main network large model;

[0011] A feedback acquisition module, used to use the fine-tuning main network large model to perform the main network operation task, and obtain the output result of the fine-tuning main network large model and feedback data based on the output result;

[0012] A reinforcement learning module is used to perform reinforcement learning on the fine-tuning main network large model based on the output results and the feedback data to obtain a main network operation task large model.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the large model adaptation method described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the large model adaptation method described in any embodiment of the present invention when executed.

[0018] According to another aspect of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the large model adaptation method of any embodiment of the present disclosure is implemented.

[0019] The technical solution of the embodiment of the present invention is to fine-tune the parameters of the pre-trained big model based on the historical main grid operation instruction data to obtain the fine-tuned main grid big model, and then use the fine-tuned main grid big model to execute the main grid operation task, to obtain the output result of the fine-tuned main grid big model and feedback data based on the output result, and finally based on the output result and the feedback data, to perform reinforcement learning on the fine-tuned main grid big model to obtain the main grid operation task big model. The parameters of the pre-trained big model are fine-tuned by the historical main grid operation instruction data, so as to improve the big model's ability to understand the power grid instructions, and further strengthen the learning based on the feedback data, so that the output of the big model can be more in line with actual business needs.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 is a flow chart of a large model adaptation method provided according to the first embodiment of the present invention;

[0023] Figure 2 is a flow chart of a large model adaptation method provided according to the second embodiment of the present invention;

[0024] Figure 3 is a structural schematic diagram of a large model adaptation device provided according to Embodiment 3 of the present invention;

[0025] Figure 4 It is a structural schematic diagram of an electronic device for implementing the large model adaptation method of an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Embodiment 1

[0029] Figure 1A flowchart of a large model adaptation method is provided for the first embodiment of the present invention. This embodiment is applicable to the case where the large model is optimized by using power grid instruction data and user feedback data. The method can be executed by a large model adaptation device. The large model adaptation device can be implemented in the form of hardware and / or software. The large model adaptation device can be configured in various general computing devices. Figure 1 As shown, the method includes:

[0030] S110. Based on the historical main network operation instruction data, fine-tune the parameters of the pre-trained large model to obtain a fine-tuned main network large model.

[0031] The historical main network operation instruction data is used to fine-tune the parameters of the pre-trained large model. The historical main network operation instruction data is instruction data re-acquired from the main network dispatching system of the power grid. In order to improve the accuracy of the large model's understanding of the main network operation instructions of the power grid, the historical main network operation instruction data needs to cover common operation scenarios in the main network of the power grid, such as line switching, equipment start and stop, or load adjustment. Exemplarily, the historical main network operation instruction data is a text instruction, an operation command, or a device control instruction. Among them, the text instruction can be "close line 123" or "start generator set A"; the operation command is a specific system instruction, for example, "close_line(123)"; the device control instruction, for example, "open_circuit_breaker(CB12)".

[0032] The pre-trained large model is a large model that has completed preliminary training using a large-scale data set. During the pre-training process, the pre-trained large model learns general features and knowledge, and does not perform targeted learning based on the knowledge of the power grid's main network operation instructions. This makes the pre-trained large model prone to misunderstandings and execution deviations when processing power grid main network operation tasks.

[0033] In an embodiment of the present invention, in order to enable the large model to more accurately understand and execute the power grid main network operation tasks, historical main network operation instruction data can be collected, and the historical main network operation instruction data can be used to further fine-tune the parameters of the pre-trained large model to obtain a fine-tuned main network large model. Specifically, the historical main network operation instruction data can be input into the pre-trained large model to obtain the output result of the pre-trained large model, and then based on the deviation between the output result and the actual operation corresponding to the historical main network operation instruction, a loss function is formed, and finally the loss function is optimized by back propagation, and the parameters of the pre-trained large model are fine-tuned, so that the performance of the pre-trained large model in understanding and generating power grid main network operation task instructions is improved.

[0034] The loss function may include the deviation between the model output result and the actual task, and may also include the deviation between the model output result and the actual task, as well as the power grid security penalty item of the model output result.

[0035] In addition, during the parameter fine-tuning process of the pre-trained large model, only the model parameters of the set number of layers close to the model input layer can be fine-tuned. For example, only the parameters of the layers used for feature extraction in the model are fine-tuned, and the model parameters of the remaining layers are frozen. This can improve the model's ability to understand the main grid operation tasks, while reducing the amount of calculation in the model fine-tuning process and improving the efficiency of parameter fine-tuning.

[0036] S120, using the fine-tuned main network large model to execute the main network operation task, and obtaining the output result of the fine-tuned main network large model and feedback data based on the output result.

[0037] In the embodiment of the present invention, after fine-tuning the parameters of the pre-trained large model to obtain the fine-tuned main network large model, the fine-tuned main network large model is used to execute the main network operation task, and the output result of the fine-tuned main network large model is obtained. In addition, in the main network dispatching operation based on the output result, task execution feedback data is collected, for example, the feedback data includes task completion rate, abnormal practice records, and operator feedback.

[0038] In order to improve the efficiency of model training, the feedback data can be further cleaned to remove noise and invalid information, and the remaining valid feedback data can be labeled with the corresponding generation tasks to form a training data set for reinforcement learning.

[0039] S130. Based on the output results and feedback data, reinforcement learning is performed on the fine-tuned main network large model to obtain the main network operation task large model.

[0040] In an embodiment of the present invention, after obtaining the feedback data, the fine-tuned main network large model is further subjected to reinforcement learning based on the output result of the fine-tuned main network large model and the feedback data for the output result, so as to obtain the main network operation task large model. Specifically, based on the feedback data, a reinforcement learning key can be constructed, and the output result of the fine-tuned main network large model is used as an action, and the feedback data is used as a reward to construct a reward function. Based on the reward function, a deep Q-network (DQN, referred to as DQN) is used for iterative training to adjust the task generation strategy of the model so that the model output is more in line with the actual business needs of the power grid main network dispatching, and finally a main network operation task large model is obtained.

[0041] Among them, the reward function can include task completion rewards and task execution safety rewards. On the basis of reinforcement learning, the model parameters are gradually adjusted to make the model output more in line with actual operation needs, and the model generation strategy is optimized to avoid the generation of high-risk operation tasks.

[0042] In addition, after obtaining the main grid operation task model, the effect of parameter fine-tuning can be evaluated through indicators such as task execution success rate and safety score to confirm that the model can generate efficient and safe operation tasks in various power grid main grid dispatching scenarios.

[0043] The technical solution of the embodiment of the present invention is to fine-tune the parameters of the pre-trained big model based on the historical main grid operation instruction data to obtain the fine-tuned main grid big model, and then use the fine-tuned main grid big model to execute the main grid operation task, to obtain the output result of the fine-tuned main grid big model and feedback data based on the output result, and finally based on the output result and the feedback data, to perform reinforcement learning on the fine-tuned main grid big model to obtain the main grid operation task big model. The parameters of the pre-trained big model are fine-tuned by the historical main grid operation instruction data, so as to improve the big model's ability to understand the power grid instructions, and further strengthen the learning based on the feedback data, so that the output of the big model can be more in line with actual business needs.

[0044] Embodiment 2

[0045] Figure 2 This is a flowchart of a large model adaptation method provided in the second embodiment of the present invention. This embodiment further refines the above embodiment and provides specific steps for fine-tuning the parameters of the pre-trained large model based on the historical main network operation instruction data to obtain the fine-tuned main network large model, and the specific steps for reinforcing the fine-tuned main network large model based on the output results and feedback data to obtain the main network operation task large model. Figure 2 As shown, the method includes:

[0046] S210: Preprocess the historical main network operation instruction data to obtain an instruction text sequence.

[0047] In the embodiment of the present invention, in order to improve the model fine-tuning efficiency and enable the model to correctly understand and execute the power grid instructions, the historical main grid operation instruction data is first preprocessed to obtain the instruction text sequence. Specifically, the historical main grid operation instruction data is first obtained from the power grid main grid dispatching system. For example, the historical main grid operation instruction can be a text instruction "close line 123", "start generator set A", or an operation command "close_line(123)", or a device control instruction "open_circuit_breaker(CB12)".

[0048] Furthermore, historical main network operation instruction data in different formats are converted into a standard text sequence format, for example, [CLS] close line 123 [SEP], to facilitate large model processing.

[0049] After conversion to text sequence format, each instruction in the text sequence can be further cleaned to remove invalid content and standardize the quality to a unified format. All instructions use the same operator and parameter representation, for example, "[CLS] Operation content Operation object type Operation object identifier [SEP]". Finally, a standardized instruction text sequence is obtained, for example, "[CLS] Start generator set A [SEP]".

[0050] S220, input the instruction text sequence into the pre-trained large model to obtain the output result of the pre-trained large model.

[0051] S230, forming a loss function based on the deviation of the output result and the instruction text sequence associated with the actual task, and the power grid operation safety of the output result.

[0052] In the embodiment of the present invention, after obtaining the instruction text sequence, the instruction text sequence is input into the pre-trained large model to obtain the output result of the pre-trained large model. Based on the deviation of the output result and the actual task associated with the quality text sequence, and the power grid operation safety of the output result, a loss function is formed.

[0053] Optionally, the loss function is as follows:

[0054] L(θ)=α×L task +β×L safety

[0055] Among them, L task is the deviation between the output of the pre-trained large model and the actual task associated with the instruction text sequence, L safety is the grid operation safety penalty term of the output result of the pre-trained large model, and α and β are weight coefficients.

[0056] In this optional embodiment, the specific formula of the loss function is provided as follows:

[0057] L(θ)=α×L task +β×L safety

[0058] Among them, L task is the deviation between the output of the pre-trained large model and the actual task associated with the instruction text sequence, L safety is the grid operation safety penalty term of the output result of the pre-trained large model, and α and β are weight coefficients.

[0059] By considering both output deviation and grid operation safety in the loss function, the accuracy of the model's understanding of grid instructions can be improved, while the safety of the model output can be improved.

[0060] S240. Based on the loss function, fine-tune the parameters of the pre-trained large model to obtain a fine-tuned main network large model.

[0061] In an embodiment of the present invention, after constructing the loss function, the parameters of the pre-trained large model are fine-tuned through the instruction text sequence, and the above-mentioned loss function is optimized through back propagation, and the model parameters are fine-tuned, so that the performance of the model in understanding and generating power grid main network operation task instructions is improved.

[0062] Specifically, in order to improve the efficiency of parameter adjustment, parameters of some layers in the pre-trained large model can be fine-tuned according to actual needs. For example, only the parameters of the convolutional layer can be adjusted so that the model can better learn the characteristics of the main grid operation tasks and improve the adaptability of the model in the power grid field.

[0063] Optionally, based on the loss function, fine-tune the parameters of the pre-trained large model to obtain a fine-tuned main network large model, including:

[0064] Freeze the model parameters of a set number of layers starting from the output layer in the pre-trained large model;

[0065] Based on the loss function, the model parameters other than the frozen model parameters in the pre-trained large model are fine-tuned to obtain the fine-tuned main network large model.

[0066] In this optional embodiment, a specific method is provided for fine-tuning the parameters of the pre-trained large model based on the loss function to obtain a fine-tuned main network large model: freezing the model parameters of a set number of layers starting from the output layer in the pre-trained large model, for example, freezing the model parameters of a set number of layers starting from the output layer in the direction from the output layer to the input layer. Then, based on the loss function, fine-tuning the model parameters other than the frozen model parameters in the pre-trained large model is performed to obtain a fine-tuned main network large model. In this way, only the model parameters close to the input layer in the model can be fine-tuned to improve the efficiency of model parameter fine-tuning, and since the model parameters close to the output layer are used for feature extraction, only fine-tuning this part of the model parameters can improve the adaptability of the model in the main grid operation tasks.

[0067] Optionally, after fine-tuning the parameters of the pre-trained large model to obtain the fine-tuned main network large model, the following is also included:

[0068] Input the verification main network operation instruction data into the fine-tuning main network large model, obtain the output result of the fine-tuning main network large model, and determine the parameter indicators of the fine-tuning main network large model according to the output result;

[0069] If the parameter indicators do not meet the training requirements, return to execute based on the historical main network operation instruction data, fine-tune the parameters of the pre-trained large model, and obtain the operation of fine-tuning the main network large model until the parameter indicators meet the training requirements.

[0070] In this optional embodiment, specific steps are provided after fine-tuning the parameters of the pre-trained large model to obtain the fine-tuned main network large model: after collecting historical main network operation instruction data, the data can be divided into two parts, one part is used as a training set for fine-tuning model parameters, and the other part is used as verification of the main network operation instruction data to verify the model parameter indicators.

[0071] The verification main network operation instruction data is input into the fine-tuning main network large model to obtain the output result of the fine-tuning main network large model, and the parameter indicators of the fine-tuning main network large model are determined according to the output result. For example, the parameter indicators are model performance that can be quantified by indicators such as precision and recall rate. In the case that the parameter indicators do not meet the training requirements, return to execute based on the historical main network operation instruction data, fine-tune the parameters of the pre-trained large model, and obtain the operation of the fine-tuning main network large model until the parameter indicators meet the training requirements. During the iteration process, all parameters in the current fine-tuning main network large model can also be fine-tuned, or only some model parameters that need to be optimized can be fine-tuned, for example, only the convolution layer parameters are fine-tuned, so that the model can better learn the specific features of the input data.

[0072] S250, using the fine-tuned main network large model to execute the main network operation task, and obtaining the output result of the fine-tuned main network large model and feedback data based on the output result.

[0073] S260: Taking the output result as an action, and determining the task reward based on the feedback data and the reward function; the reward function includes a task completion reward and a task execution safety reward.

[0074] In the embodiment of the present invention, the output result is used as an action, and the task reward is determined based on the feedback data and the reward function. The reward function includes a task completion reward and a task execution safety reward. Considering both the task completion status and the task execution safety at the same time, the model can complete the task while ensuring the safe operation of the power grid.

[0075] Optionally, the reward function is as follows:

[0076] R=γ×S task +(1-γ)×S safety

[0077] Among them, S task It is the reward for completing the task of fine-tuning the main network model. safety is the task execution safety reward, and γ is the weight factor.

[0078] In this optional embodiment, a specific formula of the reward function is provided:

[0079] R=γ×S task +(1-γ)×Ssafety

[0080] Among them, S task It is the reward for completing the task of fine-tuning the main network model. safety is the task execution safety reward, and γ is the weight factor.

[0081] By considering both the task completion reward and the task execution safety reward in the reward function, the model can complete the task while ensuring the safe operation of the power grid.

[0082] S270. Based on the action and task rewards, the fine-tuned main network large model is reinforced with learning to obtain the main network operation task large model.

[0083] In the embodiment of the present invention, the output result is used as an action, and based on the action and task reward, the DQN method is used to perform reinforcement learning on the fine-tuned main network large model to obtain the main network operation task large model.

[0084] Optionally, based on reinforcement learning, the model's task generation strategy can be adjusted through multiple iterative training to make its output more in line with the actual business needs of the power grid main grid dispatching.

[0085] Optionally, the operating status of the main power grid can be monitored in real time to identify changing scenarios, including equipment status, line topology changes, and load changes. Dynamically adjust fine-tuning strategies based on identified scenario changes. For example, when a device failure or emergency power outage is detected, the system will prioritize the strategy optimized by reinforcement learning to ensure the safety of task generation and execution success rate.

[0086] The adapted task generation results and actual execution feedback are analyzed in a closed loop to further adjust and optimize the model parameters, forming a closed-loop feedback mechanism for continuous improvement. Through multiple cycles, the model adaptation performance is continuously enhanced.

[0087] Dynamic adaptation effect evaluation: By real-time monitoring of indicators such as task completion rate and operational risk, the effect of dynamic adaptation is evaluated to ensure that the model can maintain efficient task generation capabilities in various complex scenarios.

[0088] The technical solution of the embodiment of the present invention pre-processes the historical main network operation instruction data to obtain an instruction text sequence, inputs the instruction text sequence into a pre-trained large model, obtains the output result of the pre-trained large model, and forms a loss function based on the deviation between the output result and the instruction text sequence and the actual task, as well as the power grid operation safety of the output result. Based on the loss function, the parameters of the pre-trained large model are fine-tuned to obtain a fine-tuned main network large model. Further, the fine-tuned main network large model is used to perform the main network operation task to obtain the output result of the fine-tuned main network large model and feedback data based on the output result. The output result is used as an action, and the task reward is determined based on the feedback data and the reward function. Based on the action and the task reward, the fine-tuned main network large model is reinforced learning to obtain a main network operation task large model. By simultaneously considering the accuracy and safety of the model in executing the task, the model is fine-tuned to improve the adaptability of the large model under the power grid main network operation task.

[0089] Embodiment 3

[0090] Figure 3 This is a schematic diagram of the structure of a large model adaptation device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:

[0091] The parameter fine-tuning module 310 is used to fine-tune the parameters of the pre-trained large model based on the historical main network operation instruction data to obtain a fine-tuned main network large model;

[0092] A feedback acquisition module 320 is used to use the fine-tuning main network large model to perform the main network operation task, and obtain the output result of the fine-tuning main network large model and feedback data based on the output result;

[0093] The reinforcement learning module 330 is used to perform reinforcement learning on the fine-tuning main network large model based on the output results and the feedback data to obtain the main network operation task large model.

[0094] The technical solution of the embodiment of the present invention is to fine-tune the parameters of the pre-trained big model based on the historical main grid operation instruction data to obtain the fine-tuned main grid big model, and then use the fine-tuned main grid big model to execute the main grid operation task, to obtain the output result of the fine-tuned main grid big model and feedback data based on the output result, and finally based on the output result and the feedback data, to perform reinforcement learning on the fine-tuned main grid big model to obtain the main grid operation task big model. The parameters of the pre-trained big model are fine-tuned by the historical main grid operation instruction data, so as to improve the big model's ability to understand the power grid instructions, and further strengthen the learning based on the feedback data, so that the output of the big model can be more in line with actual business needs.

[0095] Optionally, the parameter fine-tuning module 310 includes:

[0096] A data preprocessing unit, used to preprocess the historical main network operation instruction data to obtain an instruction text sequence;

[0097] A model output result acquisition unit, used for inputting the instruction text sequence into the pre-trained large model to obtain the output result of the pre-trained large model;

[0098] A loss function construction unit, configured to form a loss function based on the deviation of the output result and the instruction text sequence in associating the actual task, and the power grid operation safety of the output result;

[0099] A parameter fine-tuning unit is used to fine-tune the parameters of the pre-trained large model based on the loss function to obtain a fine-tuned main network large model.

[0100] Optionally, the loss function is as follows:

[0101] L(θ)=α×L task +β×L safety

[0102] Among them, L task is the deviation between the output of the pre-trained large model and the actual task associated with the instruction text sequence, L safety is the grid operation safety penalty term of the output result of the pre-trained large model, and α and β are weight coefficients.

[0103] Optionally, the reinforcement learning module 330 is specifically used for:

[0104] Taking the output result as an action, and determining a task reward based on the feedback data and a reward function; the reward function includes a task completion reward and a task execution safety reward;

[0105] Based on the actions and task rewards, reinforcement learning is performed on the fine-tuned main network large model to obtain the main network operation task large model.

[0106] Optionally, the reward function is as follows:

[0107] R=γ×S task +(1-γ)×S safety

[0108] Among them, S task It is the reward for completing the task of fine-tuning the main network model. safety is the task execution safety reward, and γ is the weight factor.

[0109] Optional parameter fine-tuning unit, specifically used for:

[0110] Freezing model parameters of a set number of layers starting from the output layer in the pre-trained large model;

[0111] Based on the loss function, other model parameters except the frozen model parameters in the pre-trained large model are fine-tuned to obtain a fine-tuned main network large model.

[0112] Optionally, the large model adaptation device also includes:

[0113] A parameter index acquisition module is used to input the verification main network operation instruction data into the fine-tuned main network large model after fine-tuning the parameters of the pre-trained large model to obtain the fine-tuned main network large model, obtain the output result of the fine-tuned main network large model, and determine the parameter index of the fine-tuned main network large model according to the output result;

[0114] The main network large model fine-tuning iteration module is used to return to execute the historical main network operation instruction data to fine-tune the parameters of the pre-trained large model when the parameter indicators do not meet the training requirements, and obtain the operation of fine-tuning the main network large model until the parameter indicators meet the training requirements.

[0115] The large model adaptation device provided in the embodiment of the present invention can execute the large model adaptation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0116] In the technical solution of the present invention, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0117] Embodiment 4

[0118] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.

[0119] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, applicators, blade applicators, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0120] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0121] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0122] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a large model adaptation method.

[0123] In some embodiments, the large model adaptation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the large model adaptation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the large model adaptation method in any other appropriate manner (e.g., by means of firmware).

[0124] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or application.

[0126] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0128] The systems and techniques described herein may be implemented in a computing system that includes a backend component (e.g., as a data applier), or a computing system that includes a middleware component (e.g., an application applier), or a computing system that includes a frontend component (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0129] A computing system may include a client and an application server. The client and the application server are generally remote from each other and usually interact through a communication network. The relationship between the client and the application server is generated by computer programs running on corresponding computers and having a client-application server relationship with each other. The application server may be a cloud application server, also known as a cloud computing application server or a cloud host, which is a host product in a cloud computing application system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS applications.

[0130] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0131] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A large model adaptation method, characterized in that: include: Based on the historical main network operation instruction data, fine-tune the parameters of the pre-trained large model to obtain the fine-tuned main network large model; The fine-tuning main network large model is used to execute the main network operation task, and an output result of the fine-tuning main network large model and feedback data based on the output result are obtained; Based on the output results and the feedback data, reinforcement learning is performed on the fine-tuning main network large model to obtain a main network operation task large model.

2. The method according to claim 1, characterized in that Based on the historical main network operation instruction data, fine-tune the parameters of the pre-trained large model to obtain the fine-tuned main network large model, including: Preprocess the historical main network operation instruction data to obtain the instruction text sequence; Inputting the instruction text sequence into the pre-trained large model to obtain the output result of the pre-trained large model; Based on the deviation of the output result and the actual task associated with the instruction text sequence, and the power grid operation safety of the output result, a loss function is formed; Based on the loss function, the parameters of the pre-trained large model are fine-tuned to obtain a fine-tuned main network large model.

3. The method according to claim 2, characterized in that The loss function is as follows: L(θ)=α×L task +β×L safety Among them, L task is the deviation between the output of the pre-trained large model and the actual task associated with the instruction text sequence, L safety is the grid operation safety penalty term of the output result of the pre-trained large model, and α and β are weight coefficients.

4. The method according to claim 1, characterized in that: Based on the output result and the feedback data, the fine-tuning main network large model is subjected to reinforcement learning to obtain a main network operation task large model, including: Taking the output result as an action, and determining a task reward based on the feedback data and a reward function; the reward function includes a task completion reward and a task execution safety reward; Based on the actions and task rewards, reinforcement learning is performed on the fine-tuned main network large model to obtain the main network operation task large model.

5. The method according to claim 4, characterized in that The reward function is as follows: R=γ×S task +(1-γ)×S safety Among them, S task It is the reward for completing the task of fine-tuning the main network model. safety is the task execution safety reward, and γ is the weight factor.

6. The method according to claim 2, characterized in that Based on the loss function, fine-tune the parameters of the pre-trained large model to obtain the fine-tuned main network large model, including: Freezing model parameters of a set number of layers starting from the output layer in the pre-trained large model; Based on the loss function, other model parameters except the frozen model parameters in the pre-trained large model are fine-tuned to obtain a fine-tuned main network large model.

7. The method according to claim 1, characterized in that After fine-tuning the parameters of the pre-trained large model to obtain the fine-tuned main network large model, it also includes: Inputting the verification main network operation instruction data into the fine-tuning main network large model to obtain the output result of the fine-tuning main network large model, and determining the parameter index of the fine-tuning main network large model according to the output result; When the parameter index does not meet the training requirements, return to execute based on the historical main network operation instruction data, fine-tune the parameters of the pre-trained large model, and obtain the operation of fine-tuning the main network large model until the parameter index meets the training requirements.

8. A large model adaptation device, characterized in that: include: The parameter fine-tuning module is used to fine-tune the parameters of the pre-trained large model based on the historical main network operation instruction data to obtain the fine-tuned main network large model; A feedback acquisition module, used to use the fine-tuning main network large model to perform the main network operation task, and obtain the output result of the fine-tuning main network large model and feedback data based on the output result; A reinforcement learning module is used to perform reinforcement learning on the fine-tuning main network large model based on the output results and the feedback data to obtain a main network operation task large model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the large model adaptation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the large model adaptation method according to any one of claims 1 to 7 when executed.

Images