Knowledge-guided large-model enhanced fine-tuning power distribution network dynamic reconstruction method and related equipment

Abstract

The embodiment of the invention provides a power distribution network dynamic reconstruction method based on knowledge-guided large model enhanced fine tuning and related equipment, and belongs to the technical field of smart power grids and artificial intelligence. The method comprises the following steps: constructing a power distribution network dynamic knowledge graph, and providing structured knowledge guidance for model training; subgraph sampling is carried out based on the timestamp and converted into a fine tuning sample, and a training data set is generated; utilizing the data set to supervise, finely adjust and preheat the large language model; designing a multi-dimensional reward function of fusion format accuracy, economy and security based on mechanism knowledge in the knowledge graph and expert experience; a group relative strategy optimization mechanism is adopted to carry out reinforced fine tuning on the large language model, and the large language model is guided to output a safe, reliable and economical dynamic reconstruction strategy in interaction with the environment. According to the method, the problems of lack of training data, lack of physical knowledge guidance and insufficient decision reliability of a large language model in the power grid field are solved, and the intelligent level and decision quality of dynamic reconstruction of the power distribution network are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent power grid optimal operation and artificial intelligence, and in particular relates to a power distribution network dynamic reconstruction method of knowledge-guided large model reinforcement fine-tuning and related equipment. BACKGROUND

[0002] With large-scale access of distributed new energy and electric vehicles to the power distribution network, the strong randomness and volatility thereof lead to uneven temporal and spatial distribution of net load, causing problems such as increased network loss, voltage out-of-limit, new energy curtailment, and seriously threatening the safe and economic operation of the system. Power distribution network dynamic reconstruction realizes real-time optimization of the topology structure by flexibly adjusting the switch state, and is a key means to improve the flexible regulation and control capability of the system.

[0003] The power distribution network dynamic reconstruction is essentially a high-dimensional, complex mixed integer nonlinear stochastic optimization problem. Existing methods mainly include model-driven mathematical optimization methods and data-driven reinforcement learning methods. Mathematical optimization methods realize solvability through model relaxation, but are time-consuming to solve in large-scale power distribution networks, and are difficult to meet the real-time decision-making requirements. Although the reinforcement learning method improves the decision-making efficiency, its physical embedding depends on the design of the penalty term in the reward function, and it is difficult to effectively understand and integrate domain knowledge such as power distribution network operation procedures, expert experience, and other complex temporal and causal logic, resulting in limited strategy exploration capability and insufficient decision-making reliability.

[0004] Large language models (LLM) provide a new approach to the above problems due to their strong semantic understanding, context learning, and generative decision-making capabilities. However, there are three major challenges in directly applying LLM to power distribution network dynamic reconstruction: 1) High-quality fine-tuning dataset generation relies on manual annotation, making it difficult to efficiently model complex relationships in multi-source heterogeneous data; 2) Lack of prior knowledge injection, making it difficult for the model to understand the semantic mapping between task instructions and optimal actions, resulting in an excessively large strategy search space; 3) Existing LLM reinforcement fine-tuning lacks feedback mechanisms for physical constraints and operational objectives, making it difficult to ensure decision-making reliability. SUMMARY

[0005] The main purpose of the embodiments of the present application is to provide a power distribution network dynamic reconstruction method of knowledge-guided large model reinforcement fine-tuning and related equipment, which provides structured knowledge guidance by introducing a dynamic knowledge graph, and improves the decision-making capability, reliability, and physical consistency of the model in complex dynamic environments by combining supervised fine-tuning and group relative strategy optimization.

[0006] To achieve the above purpose, one aspect of the embodiments of the present application provides a power distribution network dynamic reconstruction method of knowledge-guided large model reinforcement fine-tuning, which comprises: construct a dynamic knowledge graph for dynamic reconfiguration of a power distribution network, the dynamic knowledge graph being configured to explicitly represent key entities in the power distribution network and coupling relationships therebetween; sample subgraphs in the dynamic knowledge graph based on timestamps, and convert the sampled structured subgraphs into fine-tuning samples described in natural language to generate a fine-tuning dataset; perform supervised fine-tuning on a pre-trained large language model using the fine-tuning dataset to obtain a preheated large language model; based on mechanism knowledge and expert experience in the dynamic knowledge graph, construct a multi-dimensional reward function for evaluating power distribution network reconfiguration strategies; based on the multi-dimensional reward function, perform reinforcement fine-tuning on the preheated large language model using a group relative strategy optimization mechanism to obtain a final model capable of outputting power distribution network dynamic reconfiguration strategies.

[0007] In some embodiments, the dynamic knowledge graph for dynamic reconfiguration of a power distribution network is constructed, comprising: top-down construction of a pattern layer of the knowledge graph, the pattern layer being constructed based on an ontology extracted from a power distribution network dynamic reconfiguration optimization model, the ontology including at least one of a substation ontology, a switch ontology, a load ontology, a distributed photovoltaic ontology, and a wind power ontology; bottom-up construction of a data layer of the knowledge graph, entities and relationships are extracted from multi-source heterogeneous data of the power distribution network through knowledge extraction technology, and are supplemented to the pattern layer; fuse the pattern layer and the data layer to construct a dynamic knowledge graph containing dynamic quadruples, the dynamic quadruples being <head entity, relationship, tail entity, timestamp>.

[0008] In some embodiments, the structured subgraphs sampled are converted into fine-tuning samples described in natural language, comprising: convert key entities, relationships and their evolution processes in the subgraphs into text pairs in the format of "instruction-input-output"; wherein the instruction includes task description, objective function and constraint conditions of the power distribution network dynamic reconfiguration; the input includes at least one of initial topology structure, line impedance information, node load prediction data, photovoltaic output prediction data and wind power output prediction data of the power distribution network; and the output is a reconfiguration scheme of the power distribution network.

[0009] In some embodiments, the pre-trained large language model is supervised fine-tuned using the fine-tuning dataset, specifically using a low-rank adaptive parameter efficient fine-tuning method.

[0010] In some embodiments, the multi-dimensional reward function includes a reward part and a penalty part; The reward part includes a format reward for evaluating the format accuracy of the model output, and an economic reward for evaluating the reduction of the system operation cost after reconstruction. The penalty part includes at least one of a voltage penalty for punishing the node voltage out-of-limit, a current penalty for punishing the branch current out-of-limit, and a looped network penalty for punishing the looped network formed by the power grid topology.

[0011] In some embodiments, the economic reward is calculated based on the switching action cost, the network loss cost, the new energy curtailment cost, and the power distribution network power purchase cost.

[0012] In some embodiments, the reinforcement fine-tuning of the preheated large language model by using the group relative strategy optimization mechanism comprises: For a given input, a set of candidate actions is sampled from the preheated large language model; The reward value of each action in the set of candidate actions is calculated based on the multi-dimensional reward function; The relative advantage of each action is calculated with the average reward value of the set of candidate actions as a reference benchmark; The strategy of the preheated large language model is optimized by maximizing the objective function constructed based on the relative advantage.

[0013] To achieve the above object, another aspect of the embodiments of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the above method when executing the computer program.

[0014] To achieve the above object, another aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above method.

[0015] To achieve the above object, another aspect of the embodiments of the present application provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the above method.

[0016] The embodiments of the present application at least have the following beneficial effects: the present application provides a knowledge-guided large model reinforcement fine-tuning power distribution network dynamic reconstruction method, electronic device, storage medium and program product, which enhances the semantic understanding and causal reasoning ability by introducing a knowledge graph, combines the context generation advantage of a large language model and the reinforcement learning strategy optimization mechanism, and improves the intelligence, real-time performance and reliability of the power distribution network dynamic reconstruction strategy. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1is a flowchart of a power distribution network dynamic reconfiguration method in which a knowledge-guided large model is reinforced and fine-tuned in embodiments of the present application. Figure 2 is a partial dynamic knowledge graph visualization display diagram in embodiments of the present application. Figure 3 is a hardware structure schematic diagram of an electronic device provided by embodiments of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present application clearer and more apparent, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with embodiments of the present application, but are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0020] Before the embodiments of the present application are described in detail, first, some nouns and terms involved in the embodiments of the present application are described, and the nouns and terms involved in the embodiments of the present application are applicable to the following explanations.

[0021] 1) Large Language Model (LLM) is a model that simulates human language rules based on massive text data through deep learning technology, and can understand and generate natural language text.

[0022] 2) Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning technique that updates model weight matrices through low-rank decomposition, used to make large pre-trained models (such as large language models) quickly adapt to specific tasks while reducing computational and storage costs.

[0023] With the accelerated implementation of the "double carbon" target, a large number of distributed new energy sources with strong random fluctuation characteristics and electric vehicles are connected to the power distribution network, significantly exacerbating the uneven distribution of net load in space and time, and thus causing problems such as increased network loss, voltage out-of-limit, new energy curtailment, and serious threats to safe and economic operation of the system. As an important means to address the above challenges, power distribution network dynamic reconfiguration achieves real-time optimization of the topology structure by flexibly adjusting the switch state, effectively improving the flexible regulation and control capability of the system.

[0024] Dynamic reconfiguration of distribution network involves discrete, continuous and random variables, contains a large number of nonlinear constraints, and is essentially a high-dimensional, complex mixed integer nonlinear stochastic optimization problem. In recent years, domestic and foreign scholars have carried out a large number of researches on dynamic reconfiguration of distribution network, including model-driven mathematical optimization methods and data-driven artificial intelligence methods. Mathematical optimization methods relax the model through model simplification techniques to achieve solvability, but when applied to large-scale distribution networks, the solution time is long and cannot meet the real-time operation and control requirements. Reinforcement learning methods improve decision-making efficiency to support real-time decision-making, but their physical embedding depends on the design of the penalty term in the reward function or the rules set in the simulation environment. Due to the difficulty of reinforcement learning methods to effectively process text knowledge, especially the lack of comprehensive understanding of domain knowledge such as power grid operation procedures and expert experience. These knowledge often contains complex temporal evolution and causal logic relationships, which are difficult to express through simple rules or penalty terms. Therefore, the strategy exploration ability of existing reinforcement learning methods is limited when dealing with complex dynamic environments, making it difficult to fully guarantee the reliability of decision-making.

[0025] In recent years, LLM has shown unique advantages in complex decision-making tasks due to its strong semantic understanding, context learning, and generative decision-making capabilities, providing a new solution to the above problems. Compared with traditional small models, the outstanding advantages of LLM as a decision-making subject lie in its strong semantic understanding ability, context learning, and the ability to quickly adapt to complex decision-making tasks. LLM not only can accurately parse complex instructions and text inputs, but also effectively integrate domain knowledge such as power grid operation procedures and expert experience. The context learning ability of LLM can support it to understand and maintain the context information of the task in the process of multiple rounds of interaction, effectively capturing the long-term dependencies of power grid decision-making tasks. Unlike traditional small models that rely on massive labeled samples, LLM can complete the rapid understanding and initial strategy learning of complex decision-making tasks through a small amount of high-quality samples for supervised fine-tuning. Through structured knowledge guidance and reinforcement fine-tuning, LLM can quickly respond to environmental changes and generate reliable decision-making strategies in the process of exploration and trial and error, further improving its autonomous decision-making ability in complex dynamic environments.

[0026] However, the direct implementation of end-to-end decision of complex tasks of power grid by LLM is still in its infancy. Taking dynamic reconfiguration of distribution network as an example, the application of LLM needs to solve the following key challenges: 1) The generation of existing LLM fine-tuning dataset highly depends on manual experience for sample labeling, which cannot efficiently model the complex time evolution and causal logic relationship contained in the multi-source heterogeneous operation data of distribution network, limiting the scale and diversity of sample generation; 2) Without prior knowledge injection, direct reinforcement fine-tuning of pre-trained LLM makes it difficult for the model to understand the semantic mapping relationship between dynamic reconfiguration task instructions and optimal switch actions, resulting in a too large strategy search space and an unclear optimization path; 3) Existing LLM reinforcement fine-tuning mostly focuses on the format specification and content accuracy of natural language, lacking feedback mechanism for physical constraints and operation goals of dynamic reconfiguration task, and the reliability of model decision cannot be guaranteed.

[0027] Therefore, the embodiment of the present application provides a knowledge-guided large model reinforcement fine-tuning method for dynamic reconfiguration of distribution network, electronic equipment, storage medium and program product. The scheme enhances semantic understanding and causal reasoning ability by introducing a knowledge graph, combines the context generation advantage of a large language model and the reinforcement learning strategy optimization mechanism, and improves the intelligence, real-time performance and reliability of the dynamic reconfiguration strategy of the distribution network. It belongs to the field of new-generation intelligent power grid decision support system and AI-enabled distribution network regulation technology.

[0028] The knowledge-guided large model reinforcement fine-tuning method for dynamic reconfiguration of distribution network provided by the embodiment of the present application relates to the comprehensive application of cross-technology in dynamic reconfiguration of distribution network, such as power system automation, knowledge graph construction, large language model fine-tuning and reinforcement learning optimization. The knowledge-guided large model reinforcement fine-tuning method for dynamic reconfiguration of distribution network provided by the embodiment of the present application can be applied in a terminal, can also be applied in a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, etc., but is not limited thereto; the server end can be configured as a standalone physical server, can also be configured as a server cluster or a distributed system composed of multiple physical servers, can also be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform, and the server can also be a node server in a blockchain network; the software can be an application that implements a knowledge-guided large model reinforcement fine-tuning method for dynamic reconfiguration of distribution network, etc., but is not limited to the above forms.

[0029] The application is operable in numerous general or special computer system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that can be suitable for use with the application include personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. The application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media including memory storage devices.

[0030] Figure 1 is an optional flowchart of a power distribution network dynamic reconfiguration method provided by an embodiment of the application, Figure 1 The method in the method can include but is not limited to steps S1 to S15.

[0031] S1: Construct a dynamic knowledge graph for power distribution network dynamic reconfiguration.

[0032] In some embodiments, a top-down and bottom-up combined method is adopted. First, structured knowledge such as task definition, objective function, constraint condition is extracted from the power distribution network dynamic reconfiguration optimization model to construct a mode layer including the ontology of substation, sectionalizing switch / interconnected switch, load, distributed photovoltaic, wind power, etc. Second, rules-driven and semi-supervised learning techniques are used to extract entities and relationships from historical operation data, operation mode change sheets, dispatching logs and expert experience texts, and supplement them to the mode layer to complete the data layer construction. Finally, the mode layer and the data layer are fused to construct a dynamic knowledge graph, whose basic unit is a dynamic quadruple in the form of <head entity, relationship, tail entity, timestamp>, and a Neo4j graph database is used for storage and visualization. The graph can represent the time sequence evolution of power distribution network topology, operating state and dispatching knowledge.

[0033] S2: Generate a high-quality fine-tuning dataset.

[0034] In some embodiments, based on the dynamic knowledge graph constructed in step S1, subgraphs are sampled along the time axis according to task requirements. The sampled subgraphs retain the key entities, relationships and state change contexts of the power distribution network in a specific time period. Then, according to the large language model fine-tuning specification, the subgraphs are converted into natural language text pairs in the format of "instruction-input-output".

[0035] 1) Instruction: For example, "Please generate the switch action strategy for each time point in the next 24 hours based on the given power distribution network data, with the goal of minimizing the total operating cost and meeting the constraints of radial, voltage, current, etc." 2) Input: Includes initial topology, line impedance, node load prediction for the next 24 hours, photovoltaic output prediction, wind power output prediction, etc.

[0036] 3) Output: A time-ordered sequence of switch actions, such as "Time T1: open lines L1, L2; close line L5."

[0037] This process has high automation, avoiding tedious and inefficient manual annotation, and generating high-quality fine-tuning datasets with accuracy and diversity.

[0038] S3: Supervised fine-tuning of large language models.

[0039] In some embodiments, a large open-source language model is selected as the base model. A low-rank adaptive fine-tuning method is used to train only the introduced low-rank matrix (e.g., rank r = 8, scaling factor α = 16), and the original pre-training weights are frozen. The fine-tuning dataset generated in step S2 is used for supervised training with cross-entropy as the loss function. This pre-training makes the large language model initially understand the task logic and output format of dynamic reconfiguration of the power distribution network, laying the foundation for subsequent reinforcement fine-tuning.

[0040] S4: Design multi-dimensional reward function.

[0041] Based on the mechanism knowledge and expert experience contained in the dynamic knowledge graph, a multi-dimensional reward function is designed for reinforcement fine-tuning , whose calculation formula is:

[0042] where, is the format reward, which is used to ensure that the model output conforms to the pre-set keywords, timestamp order, and line number range. is the economic reward, which is positively related to the reduction of the total operating cost after reconfiguration, including switch action cost, network loss cost, new energy curtailment cost, and electricity purchase cost. is the voltage out-of-limit penalty, which is triggered when any node voltage deviates from the rated value (e.g., 1.0 p.u.) beyond the allowed range. is the branch current out-of-limit penalty, which is triggered when any branch current exceeds its current carrying capacity limit. is the topology ring network penalty, which is triggered when the reconfigured network topology does not meet the radial operation requirement.

[0043] S5: Reinforcement fine-tuning based on group relative strategy optimization.

[0044] In some embodiments, the large language model preheated after step S3 is taken as an initial strategy model. For each state (i.e., power distribution network operation scenario) in the simulation environment, a set (e.g., 8) of candidate reconstruction strategies is sampled from the strategy model. The set of strategies is input into the power distribution network power flow calculation simulation environment, and the reward value of each strategy is calculated according to the multi-dimensional reward function of step S4. The average reward of the set of strategies is calculated, and the relative advantage of each strategy is calculated based on this. The strategy model is updated by optimizing the objective function, which aims to improve the probability of high-advantage strategies, while the new strategy is ensured not to deviate too much from the reference strategy (preheated model) through the constraint of KL divergence, to prevent strategy drift. Through multiple rounds of iteration, the model continuously optimizes in the trial-and-error interaction with the environment, and finally obtains the final model capable of generating safe, economic, and reliable reconstruction strategies in a complex dynamic environment.

[0045] The method of the embodiment is described in detail below in combination with the accompanying drawings and specific embodiments.

[0046] The embodiment of the application provides a power distribution network dynamic reconstruction method for knowledge-guided large model reinforcement fine-tuning, which comprises the following steps: Step 1: Construct a dynamic knowledge graph for power distribution network dynamic reconstruction, which explicitly represents key entities and their complex coupling relationships in the power distribution network, and provides structured knowledge guidance for LLM fine-tuning training.

[0047] The dynamic knowledge graph for power distribution network dynamic reconstruction adopts a top-down and bottom-up combined construction method. The construction process can be divided into three stages: mode layer construction, data layer construction, and graph integration implementation.

[0048] Firstly, structured dynamic reconstruction knowledge is extracted from the power distribution network dynamic reconstruction optimization model, including task definition, basic concepts, objective function, constraint conditions, operation procedures, etc. These knowledge provides theoretical support for the top-down construction of the graph ontology. Based on this, a knowledge graph mode layer including multiple ontologies is established, such as a substation ontology, a sectionalizing switch / tie switch ontology, a load ontology, a distributed photovoltaic ontology, and a wind power ontology, etc.

[0049] Secondly, a large amount of operation data, operation mode change orders, dispatching logs, and operation mode arrangement expert experience texts are accumulated in the process of power distribution network operation. These semi-structured and unstructured data contain rich knowledge of power distribution network dynamic reconstruction. Rule-driven and semi-supervised learning are combined to extract knowledge from the above-mentioned multi-source heterogeneous data, and the extracted knowledge is supplemented to the existing ontology structure through entity alignment and relationship mapping, etc., to complete the bottom-up data layer construction of the knowledge graph.

[0050] Finally, the fusion modeling of the mode layer and the data layer is completed, and the dynamic knowledge graph for dynamic reconfiguration of distribution networks is constructed. The timestamp information is introduced into the static triple of the knowledge graph to form a dynamic quadruple representation form of graph data, which effectively represents the time evolution and causal logic relationship of the distribution network. Based on the Neo4j graph database platform, efficient storage and visual display of graph data can be realized, as shown in FIG. 8. In addition, the graph query language of the Neo4j graph database supports fast retrieval of graph data, which can accurately track and analyze the dynamic change process of the distribution network. Figure 2

[0051] The completed dynamic knowledge graph mainly enhances the decision-making ability of the large language model from two aspects. First, the introduction of timestamps realizes the structural representation of the complex time sequence relationship of multi-source heterogeneous operation data, and generates a high-quality fine-tuning dataset for the time evolution and causal logic relationship of the LLM fine-tuning training. Second, the dynamic knowledge graph integrates multi-source heterogeneous knowledge such as scheduling experience, operation rules, and device constraints, providing structured prior knowledge for the design of the multi-dimensional reward function in the LLM reinforcement fine-tuning stage, guiding the model to realize more physically consistent strategy optimization in the interaction with the environment.

[0052] Step 2: Subgraph sampling in the knowledge graph based on timestamps, converting the structured subgraph into natural language description of fine-tuning samples, and generating a high-quality fine-tuning dataset to provide high-quality data support for the fine-tuning training of the LLM.

[0053] First, subgraph sampling is performed in the dynamic knowledge graph based on timestamps. The sampled subgraph not only retains the topological structure and semantic relationship in the original graph, but also captures the state change and context association before and after the dynamic reconfiguration of the distribution network. The subgraph sampled based on the timestamp can effectively reflect the time evolution and causal logic of the operation state of the distribution network. The fine-tuning samples generated based on the subgraph can effectively support the subsequent model fine-tuning training, helping the model to understand the time sequence dependence of the power grid state and capture the causal relationship, thereby improving the task understanding and decision-making ability of the model.

[0054] Second, according to the input and output format specification of the LLM fine-tuning training, the structured subgraph is converted into natural language description of fine-tuning samples. Specifically, the key entities, relationships, and evolution process in the subgraph are converted into "instruction-input-output" text pairs, and a high-quality fine-tuning dataset for LLM fine-tuning training is constructed. The instructions of the fine-tuning dataset include the task description of the dynamic reconfiguration of the distribution network, the objective function, the constraint condition, and the output format requirement; the input includes the initial topological structure, the line impedance information, the node load prediction data, the photovoltaic output prediction data, and the wind power output prediction data, which are the key influencing factors of the distribution network reconfiguration decision scheme; and the output is the optimal distribution network reconfiguration scheme.

[0055] ​High-quality samples not only require compliance with the input and output specifications of fine-tuning training in terms of format, but also reflect the accuracy and completeness of the content. Compared with ordinary samples or low-quality samples, the instruction description of high-quality samples is more accurate, and the input data comprehensively covers the key factors affecting the dynamic reconfiguration decision of the distribution network, thereby significantly improving the stability of fine-tuning training.

[0056] In the above subgraph sampling and semantic conversion process, the existing entity, relationship and attribute information in the knowledge graph can be directly mapped to the key elements in the fine-tuning sample, avoiding the cumbersome process of manually annotating scenes and constructing fine-tuning samples, thereby significantly reducing the workload and cost of manual annotation, and providing high-quality data support for efficient training of LLM in the dynamic reconfiguration task of the distribution network.

[0057] Step 3: Supervised fine-tuning training of the pre-trained LLM based on a small amount of high-quality fine-tuning samples, and improving the semantic understanding and instruction response ability of the LLM to the reconfiguration logic of the distribution network through preheating training.

[0058] Based on the completed fine-tuning dataset, a parameter-efficient fine-tuning method based on Low-Rank Adaptation (LoRA) fine-tuning is introduced to supervise the fine-tuning of the pre-trained LLM. LoRA fine-tuning introduces trainable low-rank matrices in the linear transformation layer and multi-head self-attention mechanism layer of the pre-trained model while freezing the original pre-training parameters, thereby efficiently embedding the knowledge of the dynamic reconfiguration field of the distribution network while maintaining the original expression ability of the model. Compared with the traditional full-parameter fine-tuning strategy, LoRA fine-tuning significantly reduces the parameter quantity and memory overhead required for model updating, effectively improving the fine-tuning efficiency and resource utilization.

[0059] For the weight matrix of the pre-trained model, it is frozen during training and no longer updated by gradient. The training process only updates the incremental weight matrix , , , . The matrices and are initialized with random Gaussian distribution and zero matrix respectively, and the matrix updated during the fine-tuning process can be represented as:

[0060] In the formula: is the rank of the incremental weight matrix ; is the scaling coefficient, which can control the scaling ratio of the low-rank matrix to the original weight.

[0061] After completing the LoRA fine-tuning, for the input , the output of the model may be expressed as:

[0062] The supervised fine-tuning adopts a cross-entropy loss function, the expression of which is as follows:

[0063] In the formula: is the number of batch training samples; is the true label of the i-th training sample; is the predicted probability value of the i-th training sample by the model.

[0064] Step 4: Based on the mechanism knowledge and expert experience in the dynamic knowledge graph, a multi-dimensional reward function is designed to guide the model to accumulate high-quality decision-making experience through trial and error in the interaction process with the power distribution network simulation environment.

[0065] After completing the supervised fine-tuning of the pre-trained LLM, the model has basic understanding of the dynamic reconfiguration task of the power distribution network and can generate power distribution network reconfiguration decision schemes according to task instructions and input data. However, due to the dependence of the supervised fine-tuning stage on a small number of high-quality samples, the model has problems such as conservative strategy and poor adaptability when facing unseen conditions and complex scenarios.

[0066] To further enhance the model's strategy exploration ability and dynamic environment adaptability, the reinforcement fine-tuning stage designs a multi-dimensional reward function based on the mechanism knowledge and expert experience in the dynamic knowledge graph. This reward function considers the output format specification, objective function, and key physical constraints of the power distribution network dynamic reconfiguration task, providing a structured reward feedback mechanism for the reinforcement fine-tuning process, which can guide the model to accumulate high-quality decision-making experience through trial and error in the interaction process with the power distribution network simulation environment. Through reinforcement fine-tuning iterative training, the model's decision-making ability in complex dynamic environments can be effectively improved.

[0067] The multi-dimensional reward function includes rewards and penalties, which are used to evaluate and guide the strategy update of the LLM.

[0068] 1) Reward part: The reward mainly consists of two parts: one is the accuracy reward of the model output result in following the output format The model output undergoes validation of keywords, timestamps, and lines to guide the model in generating responses that meet the requirements and are semantically complete, ensuring accuracy and reducing the probability of model misinterpretations. Specifically, the model output should include the keyword "The set of lines opened after dynamic reconfiguration of the distribution network is:", which is the first step in ensuring compliance. Secondly, the timestamps in the output must be listed chronologically and include the set of lines opened for each timestamp. Finally, the lines in the set of opened lines should be within the distribution network's line number range. If any of these validations fails, the reward is 0, indicating that the model output does not conform to the specifications.

[0069] Second, the reward for reduced system operating costs after restructuring. The reward is calculated using the following formula to guide the model in generating the optimal reconstruction strategy that meets economic requirements.

[0070]

[0071]

[0072] In the formula: , , and These are the switching operation cost, network loss cost, renewable energy curtailment cost, and distribution network power purchase cost, with their unit power cost coefficients being: , , and , for Electricity price at any given time; branch at time t On / off state; and Branch roads The resistance and current values; For branch set; A set of nodes connecting new energy units, including wind power and photovoltaic; This refers to the set of nodes connecting the substations. , They are nodes New energy units at Predicted values ​​and active power output at any given time; For nodes Substation Contributing effort at all times; The scheduling period is set to 24 hours. A scheduling period is defined as 1 hour.

[0073] 2) Punishment section: The penalty part mainly punishes the violation of key physical constraints in the dynamic reconfiguration process of the power distribution network, including node voltage out-of-limit, branch current out-of-limit, and power grid topology loop, etc. By imposing the above-mentioned penalties in the reinforcement fine-tuning stage, the model can be effectively guided to actively avoid solutions that do not meet the operational safety when generating reconfiguration strategies.

[0074] A higher penalty is given to the reconfiguration strategy that does not meet the node voltage and branch current constraints, and no penalty is set for actions within the constraint range.

[0075] Node voltage penalty is:

[0076] In the formula: is the node voltage penalty coefficient; is the rated voltage, which is 1.0 p.u.

[0077] Branch current penalty is:

[0078] In the formula: is the branch current penalty coefficient; Grid topology loop penalty is:

[0079] In the formula: is the grid topology loop penalty coefficient.

[0080] Comprehensive reward part and penalty part, multi-dimensional reward function is calculated as follows:

[0081] Step 5: Based on the multi-dimensional reward function design, the LLM is fine-tuned based on group relative policy optimization, effectively improving the decision-making ability of the model in complex dynamic environments.

[0082] Unlike traditional reinforcement learning methods that rely heavily on value functions for policy updates, Group Relative Policy Optimization (GRPO) introduces a policy optimization mechanism based on relative advantage evaluation within the group, significantly improving the model's policy exploration ability in complex environments. In addition, traditional reinforcement learning is prone to "policy drift" due to rapid policy updates, and GRPO introduces a reference model to continuously reference past action strategies during training, preventing new strategies from deviating from the reasonable boundaries of the task action space, effectively improving the convergence of training.

[0083] For each input sample in the fine-tuning dataset GRPO from supervised fine-tuning of the preheated policy model A set of candidate switching actions were sampled. The reward obtained by this group of samples in interacting with the environment is calculated, and the group's average reward is used as a benchmark to measure the advantage of each action relative to the group's average level. The policy model is optimized by maximizing the following objectives. :

[0084] In the formula: and For hyperparameters, It is the advantage function, which uses a set of rewards The calculations are performed to determine these rewards, which correspond to the output within each group. The advantage function is calculated as follows:

[0085] If the reward of an output within a group is higher than the group average, it is considered to have a positive advantage, and its sampling probability in the policy distribution will increase; conversely, it is considered to have a negative advantage, and its sampling probability will be suppressed.

[0086] GRPO effectively simplifies the training process by directly embedding the KL divergence between the policy model and the reference model into the loss function for regularization, rather than adding it as a penalty term to the reward function. KL divergence The calculation is as follows:

[0087] By using a strategy optimization mechanism based on intra-group relative advantage assessment, GRPO simplifies the reward estimation process and effectively avoids the reliance on global accurate value function modeling in traditional reinforcement learning methods. This helps to achieve more stable strategy optimization in distribution network operation and control scenarios with diverse state spaces, complex physical constraints, and sparse rewards.

[0088] In summary, compared with the prior art, the method of this embodiment has at least the following advantages and beneficial effects: 1) A dynamic knowledge graph for power distribution network dynamic reconfiguration is constructed by integrating massive multi-source heterogeneous data. A high-quality fine-tuning dataset with complex temporal evolution and causal logic is generated through subgraph sampling. A multi-dimensional reward function is designed to guide and strengthen fine-tuning training, which effectively enhances the decision-making ability of LLM in complex operation scenarios.

[0089] 2) Supervised fine-tuning pre-training based on a small number of high-quality fine-tuning samples effectively improves the semantic understanding and instruction response ability of the LLM for power distribution network reconstruction logic, effectively reducing the action search space in the reinforcement fine-tuning stage. Reinforcement fine-tuning training based on a large number of fine-tuning samples further enhances the model's strategy exploration ability and dynamic environment adaptability.

[0090] 3) By introducing a group relative strategy optimization mechanism for strategy updating, it exhibits stronger computational stability in complex decision-making tasks of power distribution networks with diverse state spaces, complex physical constraints, and sparse rewards, while maintaining good online decision-making efficiency.

[0091] The embodiment of the present application also provides an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the above method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0092] It can be understood that the content in the above method embodiments is applicable to the present device embodiments, the present device embodiments specifically implement the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0093] Please refer to Figure 3 , Figure 3 The hardware structure of the electronic device of another embodiment is illustrated, which includes: The processor 301 can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present application; The memory 302 can be implemented in the form of a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 302 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are saved in the memory 302 and called and executed by the processor 301 to implement the above method of the embodiments of the present application; The input / output interface 303 is used to realize information input and output; The communication interface 304 is configured to realize the communication between the device and other devices. The communication can be realized through a wired manner (for example, a USB, a network cable, and the like) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, and the like). The bus 305 is configured to transmit information between various components (for example, the processor 301, the memory 302, the input / output interface 303, and the communication interface 304) of the device. The processor 301, the memory 302, the input / output interface 303, and the communication interface 304 are connected to each other through the bus 305.

[0094] The computer readable storage medium stores a computer program. The computer program is executed by the processor to realize the method.

[0095] It can be understood that the contents in the above method embodiments are applicable to the storage medium embodiments. The storage medium embodiments specifically realize the functions of the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0096] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor. The remote memory can be connected to the processor through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0097] The computer program product includes a computer program. The computer program is executed by the processor to realize the method.

[0098] It can be understood that the contents in the above method embodiments are applicable to the program product embodiments. The program product embodiments specifically realize the functions of the above method embodiments, and achieve the same beneficial effects as the above method embodiments. The executable computer program code or "code" for executing various embodiments can be written in a high-level programming language such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (for example, Transact-SQL), Perl, or various other programming languages.

[0099] The embodiments described in the specification are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0100] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than shown in the figures, or combine certain steps, or different steps.

[0101] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0102] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the function modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0103] The terms "first", "second", "third", "fourth" and the like (if any) in the specification and above-described drawings of the present application are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0104] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B and A and B existing at the same time, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and rear associated objects. "At least one of the following" or the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b and c can be single or multiple.

[0105] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed mutual ones can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0106] The units described above as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0107] In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0108] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0109] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A knowledge-guided large-scale model-enhanced dynamic reconfiguration method for distribution networks, characterized in that, The method includes the following steps: A dynamic knowledge graph for dynamic reconfiguration of distribution networks is constructed, which is used to explicitly represent key entities and their coupling relationships in the distribution network. Subgraph sampling is performed on the dynamic knowledge graph based on timestamps, and the sampled structured subgraphs are transformed into fine-tuned samples with natural language descriptions to generate a fine-tuned dataset. The pre-trained large language model is supervised fine-tuned using the fine-tuning dataset to obtain a warmed-up large language model. Based on the mechanistic knowledge and expert experience in the dynamic knowledge graph, a multi-dimensional reward function is constructed to evaluate the distribution network reconfiguration strategy. Based on the multi-dimensional reward function, a group relative strategy optimization mechanism is used to strengthen and fine-tune the preheated large language model to obtain the final model that can output the dynamic reconfiguration strategy of the distribution network.

2. The method according to claim 1, characterized in that, The construction of a dynamic knowledge graph oriented towards dynamic reconfiguration of the distribution network includes: The top-down knowledge graph is constructed using a pattern layer, which is based on the ontology extracted from the dynamic reconfiguration optimization model of the distribution network. The ontology includes at least one of the following: substation ontology, switch ontology, load ontology, distributed photovoltaic ontology, and wind power ontology. The data layer of the knowledge graph is built from the bottom up. Entities and relationships are extracted from the multi-source heterogeneous data of the power distribution network through knowledge extraction technology and added to the pattern layer. The pattern layer and the data layer are merged to construct a dynamic knowledge graph containing dynamic quadruples, where the dynamic quadruples are <head entity, relation, tail entity, timestamp>.

3. The method according to claim 1, characterized in that, The step of converting the sampled structured subgraphs into fine-tuned samples with natural language descriptions includes: The key entities, relationships, and their evolution processes in the subgraph are transformed into text pairs in "instruction-input-output" format; The instructions include a task description, objective function, and constraints for dynamic reconfiguration of the distribution network; the inputs include at least one of the following: initial topology of the distribution network, line impedance information, node load prediction data, photovoltaic power output prediction data, and wind power output prediction data; and the output is a reconfiguration scheme for the distribution network.

4. The method according to claim 1, characterized in that, The pre-trained large language model is supervised and fine-tuned using the fine-tuning dataset, specifically by employing a low-rank adaptive parameter-efficient fine-tuning method.

5. The method according to claim 1, characterized in that, The multi-dimensional reward function includes a reward component and a penalty component; The reward component includes a format reward for evaluating the accuracy of the model output format and an economic reward for evaluating the reduction in system operating costs after reconstruction. The penalty component includes at least one of the following: a voltage penalty for penalizing node voltage exceeding limits, a current penalty for penalizing branch current exceeding limits, and a ring network penalty for penalizing the formation of a ring network in the power grid topology.

6. The method according to claim 5, characterized in that, The economic incentive is calculated based on the switching operation cost, network loss cost, renewable energy curtailment cost, and distribution network power purchase cost.

7. The method according to claim 1, characterized in that, The method of using a group-relative strategy optimization mechanism to enhance and fine-tune the preheated large language model includes: For a given input, a set of candidate actions is sampled from the preheated large language model; Calculate the reward value obtained for each action in the candidate action group based on the multi-dimensional reward function; Using the average reward value of the candidate actions in this group as a benchmark, calculate the relative advantage of each action; The strategy for optimizing the warmed-up large language model is to maximize the objective function constructed based on the relative advantages.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images