Intelligent agent optimal evolution method and related device

Abstract

The invention belongs to an agent optimization method, and the method comprises the steps: constructing a virtual power system, constructing a knowledge graph based on the text data of a power distribution network, selecting an agent evaluation index, setting an agent test scene in the virtual power system, and testing the intelligent level of an agent in the agent test scene. And on the basis of the intelligent level of the intelligent agent obtained by testing, continuously iteratively optimizing the hyper-parameter of the intelligent agent, and finally obtaining the intelligent agent with the highest intelligent level as the optimal intelligent agent. According to the priori knowledge guidance-based intelligent agent optimal evolution method, intelligent levels of different intelligent agents can be evaluated by utilizing a constructed intelligent quantitative evaluation index system, the intelligent agent with the highest intelligent level and the best power grid regulation and control capability is selected, the intelligent level of the intelligent agents is improved by adjusting hyper-parameters of the intelligent agents, and the optimal power grid regulation and control capability of the intelligent agents is improved. The corresponding power grid operation state is improved, and the priori knowledge guidance-based intelligent agent optimal evolution is realized.

Description

Technical Field

[0001] The present application relates to an intelligent agent optimization method, and specifically to an intelligent agent optimization evolution method and related devices. Background Art

[0002] Intelligent agents generally face problems that are difficult to understand, and thus difficult to evaluate and compare according to unified standards. Since the intelligence of intelligent agents is analogous to human intelligence, when quantitatively evaluating the intelligence level, it is necessary to refer to the evaluation method of human intelligence level test, and evaluate the skill acquisition and intelligence level of intelligent systems by determining the evaluation criteria for the ability of intelligent agents to achieve intelligent management and control of distribution networks and the efficiency of skill acquisition. Quantitative evaluation of intelligent agents can be used to compare the advantages and disadvantages of different intelligent systems, and then promote the optimal evolution among intelligent agents, that is, to adjust the model structure or parameter settings of the intelligent system, and ultimately improve the performance of the intelligent system, the management and control effect of the distribution network, and the intelligence level of the intelligent system.

[0003] J. Liang et al. proposed an evolutionary automatic machine learning framework called Learning Evolutionary AI Framework (LEAF). LEAF is an automatic machine learning system composed of three main components, including algorithm layer, system layer and problem domain layer. The algorithm layer and system layer work together to support the problem domain layer. LEAF is able to solve problems such as hyperparameter tuning, architecture search and complexity minimization. However, the results of LEAF cannot be generalized beyond the intended domain. The LEAF model needs to be manually designed by experts through a trial-and-error process, which is tedious, expensive and may even be wrong. Ashraf et al. used a swarm-based optimization algorithm, the Whale Optimization Algorithm (WOA), to optimize the hyperparameters of the Deep Deterministic Policy Gradient (DDPG) algorithm to achieve the best control strategy in the autonomous driving control problem. However, WOA is inefficient and usually requires thousands of CPU cores to achieve good results. In addition, there is no case where this method has been applied to both hyperparameters and neural network parameters. Summary of the invention

[0004] The present application provides an intelligent agent optimization evolution method and related devices to address the technical problems of existing intelligent agent optimization evolution methods, such as cumbersome methods, high costs, high error rates, and low efficiency.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions: In the first aspect, the present application proposes a method for optimizing the evolution of an intelligent agent, comprising: Construct a virtual power system corresponding to the actual power system; Build a knowledge graph based on distribution network text data as prior knowledge; Combined with the knowledge graph, agent evaluation indicators are selected according to the tasks performed by the agent, the agent's own characteristics, and the capabilities reflected in the agent's operation; Set up agent test scenarios in a virtual power system; Combine the agent evaluation index with the quantitative scoring function, and calculate the intelligence level of the agent in the agent test scenario according to the agent evaluation index; According to the intelligence level, the hyperparameters of the intelligent agent are optimized and adjusted, and the corresponding intelligence level is recalculated after each optimization and adjustment of the hyperparameters of the intelligent agent. After executing a preset number of times, the intelligent agent with the highest intelligence level is selected as the optimal intelligent agent.

[0006] Furthermore, the virtual power system comprises:

[0007]

[0008] in, is the power flow calculation equation of the virtual power system, is the observation equation of the virtual power system, It is the intelligent entity of grid control in the virtual grid. The actions and operations output at all times, is The actual control quantity of the power grid at any moment, It is the actual system observation and virtual system observation The difference between is The target grid electrical quantity observed by the intelligent agent in the actual power system at any moment, is The target grid electrical quantity observed by the intelligent agent in the virtual power system at the moment, It is the constraints of the power flow calculation equation, observation equation, intelligent agent control action, and actual power grid control variables. represents the number of iterations, represents the virtual power system, Represents the actual power system.

[0009] Furthermore, the construction of a knowledge graph based on distribution network text data includes: Extract data information from the fault / maintenance plan and dispatching regulations of the distribution network; Use knowledge graph technology to model the extracted data information. Use knowledge graph technology to model the extracted key information and build a knowledge graph.

[0010] Furthermore, the agent evaluation index is selected according to the tasks performed by the agent, the agent's own characteristics, and the ability of the agent in operation, including: The agent evaluation indicators are selected by comprehensively considering the similarity and superiority of the agent compared to prior knowledge, the agent's task completion, and the agent's intelligence level.

[0011] Furthermore, the selected agent evaluation indicators include expert sample utilization, active power margin and line overload rate.

[0012] Furthermore, the expert sample utilization rate includes:

[0013] The active power margin includes:

[0014] in, is the active power margin, is the sequence number of the time section, is the number of time sections, is the number of generators in the power system, is the generator serial number, For the i Generators t Maximum output at all times, For the i Generators t The effort of the moment; The line overload rate includes:

[0015] in, is the line overload rate, For the i The maximum load capacity of the overloaded line, For the r Overloaded lines t The load factor at the moment, M Indicates the number of overloaded lines. r The overload line number.

[0016] Furthermore, the agent evaluation index quantization scoring function includes:

[0017] in, Quantify the scoring function for the agent evaluation index, is the observation of the running status of the agent in the agent test scenario, In the process of evolution k The electrical control agent generated by the iteration is Testing scenarios for agents; The method for calculating the intelligence level includes:

[0018] in, For the intelligence level, is the indicator weight, M is the total number of agent evaluation indicators, and m is the serial number of the agent evaluation indicator.

[0019] In the second aspect, the present application proposes an intelligent agent optimization evolution system, comprising: System module, used to build a virtual power system corresponding to the actual power system; Graph module, used to build a knowledge graph based on distribution network text data as prior knowledge; The indicator module is used to select agent evaluation indicators based on the knowledge graph, the tasks performed by the agent, the agent's own characteristics, and the capabilities reflected in the agent's operation; Scenario module, used to set up agent test scenarios in virtual power systems; A testing module is used to combine the agent evaluation index to quantify the scoring function and calculate the intelligence level of the agent in the agent test scenario according to the agent evaluation index; The optimization module is used to optimize and adjust the hyperparameters of the intelligent agent according to the intelligence level, and recalculate the corresponding intelligence level after each optimization and adjustment of the hyperparameters of the intelligent agent. After executing a preset number of times, the intelligent agent with the highest intelligence level is selected as the optimal intelligent agent.

[0020] Furthermore, the virtual power system comprises:

[0021]

[0022] in, is the power flow calculation equation of the virtual power system, is the observation equation of the virtual power system, It is the intelligent entity of grid control in the virtual grid. The actions and operations output at all times, is The actual control quantity of the power grid at any moment, It is the actual system observation and virtual system observation The difference between is The target grid electrical quantity observed by the intelligent agent in the actual power system at any moment, is The target grid electrical quantity observed by the intelligent agent in the virtual power system at the moment, It is the constraints of the power flow calculation equation, observation equation, intelligent agent control action, and actual power grid control variables. represents the number of iterations, Represents a virtual system. Represents the actual system.

[0023] Furthermore, the construction of a knowledge graph based on distribution network text data includes: Extract data information from the fault / maintenance plan and dispatching regulations of the distribution network; Use knowledge graph technology to model the extracted data information. Use knowledge graph technology to model the extracted key information and build a knowledge graph.

[0024] Furthermore, the agent evaluation index is selected according to the tasks performed by the agent, the agent's own characteristics, and the ability of the agent in operation, including: The agent evaluation indicators are selected by comprehensively considering the similarity and superiority of the agent compared to prior knowledge, the agent's task completion, and the agent's intelligence level.

[0025] Furthermore, the selected agent evaluation indicators include expert sample utilization, active power margin and line overload rate.

[0026] Furthermore, the expert sample utilization rate includes:

[0027] The active power margin includes:

[0028] in, is the active power margin, is the sequence number of the time section, is the number of time sections, is the number of generators in the power system, is the generator serial number, For the i Generators t Maximum output at all times, For the i Generators t The effort of the moment; The line overload rate includes:

[0029] in, is the line overload rate, For the i The maximum load capacity of the overloaded line, For the r Overloaded lines t The load factor at the moment, M Indicates the number of overloaded lines. r The overload line number.

[0030] Furthermore, the agent evaluation index quantization scoring function includes:

[0031] in, Quantify the scoring function for the agent evaluation index, is the observation of the running status of the agent in the agent test scenario, In the process of evolution k The electrical control agent generated by the iteration is Testing scenarios for agents; The method for calculating the intelligence level includes:

[0032] in, For the intelligence level, is the indicator weight, M is the total number of agent evaluation indicators, and m is the serial number of the agent evaluation indicator.

[0033] In the third aspect, the present application proposes an electronic device, comprising: a memory, and one or more processors; the memory is coupled to the processor; wherein computer program code is stored in the memory, and the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the steps of the above-mentioned intelligent agent optimization evolution method.

[0034] In a third aspect, the present application proposes a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned intelligent agent optimization evolution method are implemented.

[0035] Compared with the prior art, this application has the following beneficial effects: This application proposes a method for the optimal evolution of intelligent agents, constructs a virtual power system, constructs a knowledge graph based on distribution network text data, selects intelligent agent evaluation indicators and sets up intelligent agent test scenarios in the virtual power system, tests the intelligence level of the intelligent agent in the intelligent agent test scenario, and then uses the intelligence level of the intelligent agent obtained by the test as a basis to continuously iterate and optimize the hyperparameters of the intelligent agent, and finally obtains the intelligent agent with the highest intelligence level as the optimal intelligent agent. This application proposes a method for the optimal evolution of intelligent agents guided by prior knowledge. By using the constructed intelligent quantitative evaluation index system, the intelligence levels of different intelligent agents can be evaluated, and the intelligent agent with the highest intelligence level and the best ability to regulate the power grid can be selected. By adjusting the hyperparameters of the intelligent agent, the intelligence level of the intelligent agent can be improved, the corresponding power grid operation status can be improved, and the optimal evolution of intelligent agents guided by prior knowledge can be realized.

[0036] The present application also proposes an intelligent agent optimization evolution system, an electronic device and a computer storage medium, which possess all the advantages of the above-mentioned intelligent agent optimization evolution method. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 This is a schematic diagram of the first process of the intelligent agent optimization evolution method of this application; Figure 2 This is a schematic diagram of the second process of the intelligent agent optimization evolution method of this application; Figure 3 A schematic diagram of the intelligent agent optimization evolution system of this application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0040] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0041] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0042] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invented product is usually placed when used. It is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0043] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0044] In the description of the embodiments of the present application, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0045] The evolution of intelligent agents towards excellence refers to the process in which intelligent agents gradually improve their performance and capabilities through continuous learning, adaptation and optimization. This process is similar to biological evolution. Under the selection pressure of the "environment", intelligent agents gradually evolve to a better state through mechanisms such as inheritance, mutation and natural selection.

[0046] At present, the evolution of intelligent agents mainly adopts the evolutionary method based on reinforcement learning, the evolutionary method based on Bayesian optimization and the method based on deep evolutionary reinforcement learning. Reinforcement learning algorithms can be used to evaluate the advantages and disadvantages of different intelligent agent behaviors, and evolutionary algorithms can be used to iteratively optimize the strategies of intelligent agents. For example, genetic algorithms can be used to encode the strategy parameters of intelligent agents, and new strategies can be generated through operations such as selection, crossover and mutation. Reinforcement learning algorithms are used to evaluate the performance of these new strategies. Through multiple iterations, intelligent agents with better performance can be gradually evolved. Furthermore, Bayesian optimization is an efficient global optimization algorithm that uses Bayesian theorem to estimate the posterior distribution of the objective function and guides the search process by constructing an agent model. In the evolution of intelligent agents, Bayesian optimization algorithms can be combined with evolutionary algorithms to guide the generation and optimization of intelligent agent strategies through agent models. In addition, the method based on deep evolutionary reinforcement learning is a new computing framework that combines deep learning and evolutionary algorithms. Deep learning models are used to represent the strategies of intelligent agents, and the parameters of these models are optimized through evolutionary algorithms. However, these methods have technical problems such as cumbersome processes, high costs, high error rates, and low efficiency.

[0047] Based on the above situation, the present application proposes an intelligent agent optimal evolution method and related devices, and the present application is described in detail below in conjunction with the embodiments and drawings.

[0048] like Figure 1 As shown, it is a first flow chart of an intelligent agent optimization evolution method of the present application, which may include: S101, constructing a virtual power system corresponding to the actual power system.

[0049] Create a virtual platform that can accurately simulate the operating environment and characteristics of the actual power system for subsequent testing and evaluation of the intelligent agent. In practical applications, relevant data such as the grid structure, equipment parameters, and operating rules can be obtained from the actual power system. Using these data, a virtual power system model corresponding to the actual power system can be constructed in the simulation software. The constructed virtual power system is then verified to ensure that it can accurately simulate the operating characteristics of the actual power system and be optimized and adjusted as needed.

[0050] S102, construct a knowledge graph based on the distribution network text data as prior knowledge.

[0051] Providing agents with rich prior knowledge helps to better understand and handle complex problems in the power system. In practical applications, information such as device names, device parameters, operating rules, and fault handling can be extracted from text data related to the distribution network. Using natural language processing technology and knowledge graph building tools, this information can be organized into a structured knowledge graph. Ensure that the device names in the distribution network accurately correspond to the device information in the knowledge graph so that the agent can quickly obtain relevant information when needed.

[0052] S103, combining the knowledge graph, selecting agent evaluation indicators according to the tasks performed by the agent, the agent's own characteristics, and the capabilities reflected in the agent's operation.

[0053] According to the task requirements, characteristics and operation capabilities of the intelligent agent in the power system, appropriate evaluation indicators can be selected to evaluate the intelligence level of the intelligent agent. In practical applications, the specific tasks of the intelligent agent in the power system can be clarified, such as fault detection, load forecasting, and optimal scheduling. Consider the characteristics of the intelligent agent itself, such as algorithm type, learning ability, and adaptability. Combined with the capabilities of the intelligent agent during operation, such as accuracy, efficiency, and stability. Based on the above analysis, select evaluation indicators that can fully reflect the intelligence level of the intelligent agent, such as accuracy, recall rate, response time, resource consumption, etc.

[0054] S104, setting an intelligent agent test scenario in the virtual power system.

[0055] Provide a real test environment for the agent, which can be used to evaluate the performance of the agent in different scenarios. In practical applications, various test scenarios can be designed according to the actual operation of the power system, such as normal operation, fault handling, load peak, etc. Configure the corresponding test environment in the virtual power system, including the grid structure, equipment status, load demand, etc. Deploy the agent to be tested into the virtual power system and set the corresponding operating parameters and initial conditions.

[0056] S105, combining the agent evaluation index with the quantified scoring function, and calculating the intelligence level of the agent in the agent test scenario according to the agent evaluation index.

[0057] Through the quantitative scoring function, the intelligence level of the agent in the test scenario can be calculated according to the agent evaluation index. During the test, the running data of the agent is collected, and the collected data is processed and analyzed according to the selected agent evaluation index and quantitative scoring function to calculate the intelligence level score of the agent in the test scenario.

[0058] S106, optimizing and adjusting the hyperparameters of the intelligent agent according to the intelligence level, and recalculating the corresponding intelligence level after optimizing and adjusting the hyperparameters of the intelligent agent each time, and after executing a preset number of times, selecting the intelligent agent with the highest intelligence level as the optimal intelligent agent.

[0059] By optimizing and adjusting the hyperparameters of the intelligent agent, its intelligence level is improved, and the optimal intelligent agent is selected. In practical applications, the hyperparameters of the intelligent agent can be adjusted using appropriate optimization algorithms (such as grid search, random search, Bayesian optimization, etc.) according to the intelligence level score of the intelligent agent in the test scenario. After each optimization and adjustment of the hyperparameters of the intelligent agent, it is redeployed to the virtual power system for testing, and the corresponding intelligence level score is calculated. The process of hyperparameter optimization and intelligence level calculation is repeated until the preset number of iterations is reached or the performance of the intelligent agent is no longer significantly improved. Among all the tested intelligent agents, the intelligent agent with the highest intelligence level score is selected as the optimal intelligent agent.

[0060] This application combines the methods and theories of prior knowledge to establish an intelligent agent quantitative evaluation index system to achieve quantitative evaluation of intelligence. Combining Bayesian optimization and automated reinforcement learning technology, as well as the objective function of the intelligent agent's optimal evolution, an intelligent agent evolution method guided by prior knowledge is proposed. Using the constructed intelligent quantitative evaluation index system, the intelligence level of different intelligent agents can be evaluated, and based on this, the intelligent agent with the highest intelligence level and the best power grid control ability can be selected from the intelligent agents with better training results. By adjusting the hyperparameters, the intelligence level of the intelligent agent can be improved, the corresponding power grid operation status can be improved, and the optimal evolution of the intelligent agent guided by prior knowledge can be achieved. In practical applications, the method of this application can also be further extended to the human-machine hybrid enhanced intelligent system to achieve the evaluation and optimization of human-machine hybrid enhanced intelligence.

[0061] For DRL (Deep Reinforcement Learning, DRL) agents that perform grid correction control tasks, such as Figure 2 As shown, it is a second flow chart of an intelligent agent optimization evolution method of the present application, which may include: S201, designing a virtual power system based on the actual power system.

[0062] The solution process of the general actual power system in the virtual power system is shown in the following formula:

[0063]

[0064] in, k represents the number of iterations, A represents the virtual power system, Rrepresents the actual power system, is the power flow calculation equation of the virtual power system, is the observation equation of the virtual power system, It is the intelligent entity of grid control in the virtual grid. The actions and operations output at all times, is The actual control quantity of the power grid at any moment, Represents the actual system observation and virtual system observation The difference between It means in The target grid electrical quantity observed by the intelligent agent in the virtual system at the moment, It means in The target grid electrical quantity observed by the agent in the actual system at any moment, It is the constraint condition of power flow calculation equation, observation equation, control action of intelligent agent and actual power grid control variable.

[0065] S202, analyze the text data of the distribution network, digitize it, and use it as prior knowledge to design intelligent agent evaluation indicators based on text data such as distribution network transfer related fault / maintenance plans.

[0066] Extract key information from text data such as fault / maintenance plans, dispatching regulations, etc., such as operation information, topology changes, operation modes, etc. Use knowledge graph technology to model the extracted key information. Specifically, the following methods can be used: Extract knowledge of power grid equipment topology entities and operation step entities from fault / maintenance plans to build a power grid fault handling entity library. The ontology can be called a knowledge base, and the ontology of the knowledge graph Include the type of the object , the type of attribute and the type of relationship , which can be expressed as:

[0067] in, Represents entities of the same type. (such as buses of different voltage levels) corresponds to a class of objects (such as buses), It represents the same type of attributes (such as the attributes of different entities: busbars, transformers, and lines all have the attribute of voltage level). It represents the same type of relationship (for example, busbar entities and disconnector entities of different voltage levels have the same type of relationship).

[0068] By extracting the equipment name information from the fault / maintenance plan text, the extracted information is integrated with the equipment information.

[0069] The following methods can be used to select agent evaluation indicators: (1) Select appropriate intelligence indicators to be tested from the indicator system based on the tasks performed by the intelligent agent, the characteristics of the intelligent agent, and the capabilities demonstrated by the intelligent agent during operation. (2) Refer to the design method of the operating steps in the fault / maintenance plan, analyze the logic and ideas behind it, and summarize the evaluation indicators that can reflect the similarity and superiority of the intelligent agent's control results with the expert's prior knowledge.

[0070] (3) Imitating the division method of human intelligence test indicators, referring to human intelligence evaluation theory and cognitive theory, and then, in accordance with the principles of representativeness and comprehensiveness, classifying and summarizing the expert sample utilization rate, active power margin, and line overload rate indicators from three aspects: similarity and superiority compared to human prior knowledge, task completion, and the intelligence level of the intelligent agent (to complete the task and design goals). In the subsequent quantitative evaluation of the intelligence level of each intelligent system, appropriate indicators to be tested can be selected from this indicator system:

[0071]

[0072] in, is the active power margin, is the sequence number of the time section, is the number of time sections, is the number of generators in the power system, is the generator serial number, For the i Generators t Maximum output at all times, For the i Generators t The effort of the moment.

[0073]

[0074] in, is the line overload rate, For the i The maximum load capacity of the overloaded line, For the r Overloaded lines t The load factor at the moment, M Indicates the number of overloaded lines. r The overload line number.

[0075] S203, setting the test scenario considering the capabilities corresponding to the agent indicators and the similarity of the training scenarios.

[0076] Considering the training scenarios of the target intelligent agent, the capabilities of the target intelligent agent corresponding to the intelligent indicators, the similarity of the training and test scenarios, and the difficulty of the test scenarios themselves, an intelligent agent test scenario is designed in the virtual power system.

[0077] S204, calculating the intelligence level of the intelligent agent according to the selected intelligent agent evaluation index.

[0078] Obtain observations of the agent's operating status in the target power grid test scenario through calculations and experiments ; Using smart indicators Quantitative scoring function , the intelligent indicators of the tested intelligent agent Test scenario Quantitative scoring of intelligence level in the process. The capabilities represented, and The grid operation status parameters contained in are used to select appropriate indicators to quantify the scoring function.

[0079] Based on the obtained indicators The intelligent scoring results, the comprehensive intelligence level of the intelligent agent As shown below:

[0080] in, For the intelligence level, is the indicator weight, M is the total number of agent evaluation indicators, and m is the serial number of the agent evaluation indicator.

[0081] S205, using the intelligence level of the agent and hyperparameters Update the Gaussian process, calculate information entropy and weights, and adjust the hyperparameters of the agent through Bayesian optimization to improve the performance of the agent.

[0082] The following methods can be used: By controlling the control quantity of the virtual power system and the actual power system and , to obtain the best intelligent quantitative evaluation results of the agent The process is shown in the following formula:

[0083] in, Mainly by Provided, therefore, by adjusting the control maximize The process is shown below:

[0084] Here is The control can be achieved by The adjustment of can be realized, and the basic framework of autonomous optimal evolution of the intelligent agent based on intelligent evaluation can be obtained, as shown in the following formula:

[0085] in, Indicates that in the evolutionary process The electrical control agent generated by the iteration is It is The hyperparameters of the agent generated in the iteration, It is The comprehensive intelligence level of the agent generated by the iteration.

[0086] S206, repeating steps S204 and S205 multiple times, calculating the agent level after each adjustment, and taking the agent with the highest intelligence level as the optimal agent.

[0087] As an example, we can let the total number of iterations be K =18000, execute steps S204 and S205 repeatedly, calculate the agent level after each adjustment, and take the agent with the highest intelligence level as the optimal agent.

[0088] Set a loop limit and calculate the intelligence level each time you loop , the agent and intelligence level Save to history, and find the one with the highest comprehensive intelligence level in limited iterations and the same intelligence indicators and test scenarios Agent and its hyperparameters , these hyperparameters are saved in the search space. During the evolution process, the grid control agent still needs to meet its design goals and constraints:

[0089] in, and Indicates the highest level of comprehensive intelligence The optimal agent and its hyperparameters; Indicates that in the evolutionary process The electrical control agent generated by the iteration; It is The hyperparameters of the agent generated in the iteration; It is The comprehensive intelligence level of the agent generated in the iteration; represents the number of evolutionary iterations; is the total number of iterative evolutions; Indicates control The reward function of is the test scenario used in intelligent evaluation, It is the intelligent indicator to be tested in the intelligent assessment.

[0090] like ,use and replace and , otherwise keep and constant; use and Update the Gaussian process (GP) and get the mean from the updated GP and variance ,use and Optimized .

[0091] This application uses prior knowledge to perform quantitative evaluation on the trained model, and iterates the model hyperparameters based on the quantitative evaluation results, achieving good results in convergence speed and convergence performance.

[0092] like Figure 3 As shown, it is a schematic diagram of an intelligent agent optimization evolution system of the present application, which may include: System module, used to build a virtual power system corresponding to the actual power system; Graph module, used to build a knowledge graph based on distribution network text data as prior knowledge; The indicator module is used to select agent evaluation indicators based on the knowledge graph, the tasks performed by the agent, the agent's own characteristics, and the capabilities reflected in the agent's operation; Scenario module, used to set up agent test scenarios in virtual power systems; A testing module is used to combine the agent evaluation index to quantify the scoring function and calculate the intelligence level of the agent in the agent test scenario according to the agent evaluation index; The optimization module is used to optimize and adjust the hyperparameters of the intelligent agent according to the intelligence level, and recalculate the corresponding intelligence level after each optimization and adjustment of the hyperparameters of the intelligent agent. After executing a preset number of times, the intelligent agent with the highest intelligence level is selected as the optimal intelligent agent.

[0093] In some embodiments of the intelligent agent optimization evolution system of the present application, the virtual power system may include:

[0094]

[0095] in, is the power flow calculation equation of the virtual power system, is the observation equation of the virtual power system, It is the intelligent entity of grid control in the virtual grid. The actions and operations output at all times, is The actual control quantity of the power grid at any moment, It is the actual system observation and virtual system observation The difference between is The target grid electrical quantity observed by the intelligent agent in the actual power system at any moment, is The target grid electrical quantity observed by the intelligent agent in the virtual power system at the moment, It is the constraints of the power flow calculation equation, observation equation, intelligent agent control action, and actual power grid control variables. represents the number of iterations, Represents a virtual system. Represents the actual system.

[0096] In some embodiments of the intelligent agent optimization evolution system of the present application, constructing a knowledge graph based on distribution network text data may include: Extract data information from the fault / maintenance plan and dispatching regulations of the distribution network; Use knowledge graph technology to model the extracted data information. Use knowledge graph technology to model the extracted key information and build a knowledge graph.

[0097] In some embodiments of the intelligent agent optimization evolution system of the present application, when selecting the intelligent agent evaluation index based on the tasks performed by the intelligent agent, the intelligent agent's own characteristics, and the capabilities demonstrated in the operation of the intelligent agent, it can include: comprehensively considering the similarity and superiority of the intelligent agent compared to prior knowledge, the task completion status of the intelligent agent, and the intelligence level of the intelligent agent, and selecting the intelligent agent evaluation index.

[0098] In some embodiments of the intelligent agent optimization evolution system of the present application, the selected intelligent agent evaluation indicators may include expert sample utilization, active power margin and line overload rate.

[0099] In some embodiments of the intelligent agent optimization evolution system of the present application, the expert sample utilization rate includes:

[0100] The active power margin includes:

[0101] in, is the active power margin, is the sequence number of the time section, is the number of time sections, is the number of generators in the power system, is the generator serial number, For the i Generators t Maximum output at all times, For the i Generators t The effort of the moment; The line overload rate includes:

[0102] in, is the line overload rate, For the i The maximum load capacity of the overloaded line, For the r Overloaded lines t The load factor at the moment, M Indicates the number of overloaded lines. r The overload line number.

[0103] In some embodiments of the intelligent agent optimization evolution system of the present application, the intelligent agent evaluation index quantitative scoring function may include:

[0104] in, Quantify the scoring function for the agent evaluation index, is the observation of the running status of the agent in the agent test scenario, In the process of evolution k The electrical control agent generated by the iteration is Testing scenarios for agents; The method for calculating the intelligence level includes:

[0105] in, For the intelligence level, is the indicator weight, M is the total number of agent evaluation indicators, and m is the serial number of the agent evaluation indicator.

[0106] It should be noted that in the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of each module is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another device, or some features can be ignored or not executed. The module described as a separate component may or may not be physically separated. The component displayed as a module may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed in multiple different places. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0107] In addition, each module in each embodiment of the present invention may be integrated into a processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0108] An embodiment of the present application also provides an electronic device, which may include one or more processors, a memory, and a communication interface.

[0109] The memory, the communication interface and the processor are coupled, for example, the memory, the communication interface and the processor may be coupled together via a bus.

[0110] The communication interface is used for data transmission with other devices. The memory stores computer program code. The computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the steps of the intelligent agent optimization evolution method in the embodiment of the present application.

[0111] Wherein, the processor can be a processor or a controller, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the present disclosure. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of DSP and microprocessors, and the like. The processor can be used to support electronic devices to execute the method steps provided in the above embodiments.

[0112] The bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The above bus may be divided into an address bus, a data bus, a control bus, etc.

[0113] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the intelligent agent optimization evolution method described in any of the above embodiments are implemented.

[0114] The computer-readable storage medium involved in the present application includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field.

[0115] For the description of the relevant parts of the intelligent agent optimization evolution system, electronic device and computer-readable storage medium provided in the embodiments of the present application, please refer to the detailed description of the corresponding parts in the intelligent agent optimization evolution method provided in the embodiments of the present application, which will not be repeated here. In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0116] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for optimizing the evolution of an intelligent agent, characterized in that: include: Construct a virtual power system corresponding to the actual power system; Build a knowledge graph based on distribution network text data as prior knowledge; Combined with the knowledge graph, agent evaluation indicators are selected according to the tasks performed by the agent, the agent's own characteristics, and the capabilities reflected in the agent's operation; Set up agent test scenarios in a virtual power system; Combine the agent evaluation index with the quantitative scoring function, and calculate the intelligence level of the agent in the agent test scenario according to the agent evaluation index; According to the intelligence level, the hyperparameters of the intelligent agent are optimized and adjusted, and the corresponding intelligence level is recalculated after each optimization and adjustment of the hyperparameters of the intelligent agent. After executing a preset number of times, the intelligent agent with the highest intelligence level is selected as the optimal intelligent agent.

2. The agent optimization evolution method according to claim 1, characterized in that: The virtual power system comprises: in, is the power flow calculation equation of the virtual power system, is the observation equation of the virtual power system, It is the intelligent entity of grid control in the virtual grid. The actions and operations output at all times, is The actual control quantity of the power grid at any moment, It is the actual system observation and virtual system observation The difference between is The target grid electrical quantity observed by the intelligent agent in the actual power system at any moment, is The target grid electrical quantity observed by the intelligent agent in the virtual power system at the moment, It is the constraints of the power flow calculation equation, observation equation, intelligent agent control action, and actual power grid control variables. represents the number of iterations, represents the virtual power system, Represents the actual power system.

3. The agent optimization evolution method according to claim 1, characterized in that: The construction of a knowledge graph based on distribution network text data includes: Extract data information from the fault / maintenance plan and dispatching regulations of the distribution network; Use knowledge graph technology to model the extracted data information. Use knowledge graph technology to model the extracted key information and build a knowledge graph.

4. The agent optimization evolution method according to claim 1, characterized in that: The agent evaluation index is selected based on the tasks performed by the agent, the agent's own characteristics, and the ability of the agent in operation, including: The agent evaluation indicators are selected by comprehensively considering the similarity and superiority of the agent compared to prior knowledge, the agent's task completion, and the agent's intelligence level.

5. The agent optimization evolution method according to claim 1, characterized in that: The selected agent evaluation indicators include expert sample utilization, active power margin and line overload rate.

6. The agent optimization evolution method according to claim 1, characterized in that: The expert sample utilization rate includes: The active power margin includes: in, is the active power margin, is the sequence number of the time section, is the number of time sections, is the number of generators in the power system, is the generator serial number, For the i Generators t Maximum output at all times, For the i Generators t The effort of the moment; The line overload rate includes: in, is the line overload rate, For the i The maximum load capacity of the overloaded line, For the r Overloaded lines t The load factor at the moment, M Indicates the number of overloaded lines. r The overload line number.

7. The agent optimization evolution method according to claim 1, characterized in that: The agent evaluation index quantification scoring function includes: in, Quantify the scoring function for the agent evaluation index, is the observation of the running status of the agent in the agent test scenario, In the process of evolution k The electrical control agent generated by the iteration is Testing scenarios for agents; The method for calculating the intelligence level includes: in, For the intelligence level, is the indicator weight, M is the total number of agent evaluation indicators, and m is the serial number of the agent evaluation indicator.

8. An intelligent agent optimization evolution system, characterized in that: include: System module, used to build a virtual power system corresponding to the actual power system; Graph module, used to build a knowledge graph based on distribution network text data as prior knowledge; The indicator module is used to select agent evaluation indicators based on the knowledge graph, the tasks performed by the agent, the agent's own characteristics, and the capabilities reflected in the agent's operation; Scenario module, used to set up agent test scenarios in virtual power systems; A testing module is used to combine the agent evaluation index to quantify the scoring function and calculate the intelligence level of the agent in the agent test scenario according to the agent evaluation index; The optimization module is used to optimize and adjust the hyperparameters of the intelligent agent according to the intelligence level, and recalculate the corresponding intelligence level after each optimization and adjustment of the hyperparameters of the intelligent agent. After executing a preset number of times, the intelligent agent with the highest intelligence level is selected as the optimal intelligent agent.

9. The intelligent agent optimization evolution system according to claim 8, characterized in that: The virtual power system comprises: in, is the power flow calculation equation of the virtual power system, is the observation equation of the virtual power system, It is the intelligent entity of grid control in the virtual grid. The actions and operations output at all times, is The actual control quantity of the power grid at any moment, It is the actual system observation and virtual system observation The difference between is The target grid electrical quantity observed by the intelligent agent in the actual power system at any moment, is The target grid electrical quantity observed by the intelligent agent in the virtual power system at the moment, It is the constraints of the power flow calculation equation, observation equation, intelligent agent control action, and actual power grid control variables. represents the number of iterations, Represents a virtual system. Represents the actual system.

10. The intelligent agent optimization evolution system according to claim 8, characterized in that: The construction of a knowledge graph based on distribution network text data includes: Extract data information from the fault / maintenance plan and dispatching regulations of the distribution network; Use knowledge graph technology to model the extracted data information. Use knowledge graph technology to model the extracted key information and build a knowledge graph.

11. The intelligent agent optimization evolution system according to claim 8, characterized in that: The agent evaluation index is selected based on the tasks performed by the agent, the agent's own characteristics, and the ability of the agent in operation, including: The agent evaluation indicators are selected by comprehensively considering the similarity and superiority of the agent compared to prior knowledge, the agent's task completion, and the agent's intelligence level.

12. The intelligent agent optimization evolution system according to claim 8, characterized in that: The selected agent evaluation indicators include expert sample utilization, active power margin and line overload rate.

13. The intelligent agent optimization evolution system according to claim 8, characterized in that: The expert sample utilization rate includes: The active power margin includes: in, is the active power margin, is the sequence number of the time section, is the number of time sections, is the number of generators in the power system, is the generator serial number, For the i Generators t Maximum output at all times, For the i Generators t The effort of the moment; The line overload rate includes: in, is the line overload rate, For the i The maximum load capacity of the overloaded line, For the r Overloaded lines t The load factor at the moment, M Indicates the number of overloaded lines. r The overload line number.

14. The intelligent agent optimization evolution system according to claim 8, characterized in that: The agent evaluation index quantification scoring function includes: in, Quantify the scoring function for the agent evaluation index, is the observation of the running status of the agent in the agent test scenario, In the process of evolution k The electrical control agent generated by the iteration is Testing scenarios for agents; The method for calculating the intelligence level includes: in, For the intelligence level, is the indicator weight, M is the total number of agent evaluation indicators, and m is the serial number of the agent evaluation indicator.

15. An electronic device, characterized in that: include: A memory and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the steps of the intelligent agent optimization evolution method as described in any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the intelligent agent optimization evolution method as described in any one of claims 1 to 7.

Images