Energy router state control method and device based on operation and maintenance agent

Abstract

The invention provides an energy router state control method and device based on an operation and maintenance agent, and relates to the technical field of energy internet. The operation and maintenance agent-based energy router state control method comprises the following steps: when a transfer condition of a state machine of a target energy router is satisfied, a target operation and maintenance agent associated with the target energy router acquires state data; the state machine of the target energy router is used for describing different states and conversion actions among the states in the operation process of the target energy router; according to the state data, determining a model by utilizing a strategy matched with the type of the state data, and obtaining a conversion action corresponding to the transfer condition; and instructing the target energy router to execute state transition according to the conversion action. According to the invention, the energy router in the energy internet can be accurately and efficiently controlled, so that the stability, economical efficiency and robustness of the operation of the energy internet are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of energy internet, and particularly relates to an energy router state control method and device based on an operation and maintenance intelligent agent. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, its application range in the field of information communication is continuously expanding, which has profoundly changed the operation mode of traditional industries. As a modern energy network that deeply integrates information communication technology and energy utilization technology, energy internet is gradually becoming an important direction for the development of future energy systems. Energy internet realizes efficient and flexible configuration and utilization of energy by integrating distributed energy resources (such as solar energy, wind energy, etc.), energy storage devices and intelligent loads. However, the complexity and dynamics of energy internet also put higher requirements on the operation and management of the system.

[0003] In the energy internet, the energy router as a core device undertakes the key task of energy distribution, conversion and control. In related technologies, there is still a lack of accurate modeling mechanism for the control algorithm of the energy router.

[0004] Therefore, how to accurately and efficiently control the energy router in the energy internet, so as to ensure the stability, economy and robustness of the operation of the energy internet, is a technical problem to be solved. SUMMARY

[0005] The present application provides an energy router state control method and device based on an operation and maintenance intelligent agent, to solve the above defects in the prior art and realize accurate and efficient modeling of the control algorithm of the energy router in the energy internet.

[0006] The present application provides an energy router state control method based on an operation and maintenance intelligent agent, comprising the following steps.

[0007] When the transition condition of the state machine of the target energy router is met, the target operation and maintenance intelligent agent associated with it acquires state data; wherein the state data includes the state data of the target energy router itself and / or the interaction data between the energy internet accessed by the target energy router and the target energy router; the state machine of the target energy router is used to describe different states in its running process and the transition actions between states; according to the state data, a strategy determination model matching the type of the state data is used to determine the transition action corresponding to the transition condition; wherein the strategy determination model is a machine learning model trained using historical state data of the same type as the state data; the target energy router is instructed to perform state transition according to the transition action.

[0008] The application provides an energy router state control method based on an operation and maintenance intelligent agent, and the policy determination model is obtained through the following method: acquiring a training data set from an established routing strategy knowledge base according to the type of state data processed by the policy determination model; and training an initial machine learning model by using the training data set to obtain the policy determination model.

[0009] The application provides an energy router state control method based on an operation and maintenance intelligent agent, and the routing strategy knowledge base comprises a plurality of knowledge bases divided according to execution time dimensions of conversion actions of the state machine; and the method comprises the following steps: determining the execution time dimension of a conversion action corresponding to the type of state data according to the type of the state data; and acquiring the training data set from a routing strategy knowledge base matched with the execution time dimension.

[0010] The application provides an energy router state control method based on an operation and maintenance intelligent agent, and the routing strategy knowledge base comprises an instantaneous action expert knowledge base, a medium and long-term operation expert knowledge base and a medium and long-term management knowledge base; the instantaneous action expert knowledge base corresponds to a first execution time dimension, the medium and long-term operation expert knowledge base corresponds to a second execution time dimension, and the medium and long-term management knowledge base corresponds to a third execution time dimension; the first execution time dimension is greater than the second execution time dimension, and the second execution time dimension is greater than the third execution time dimension; and the method comprises the following steps: if the execution time dimension is the first execution time dimension, the training data set is acquired from the instantaneous action expert knowledge base; if the execution time dimension is the second execution time dimension, the training data set is acquired from the medium and long-term operation expert knowledge base; and if the execution time dimension is the third execution time dimension, the training data set is acquired from the medium and long-term management knowledge base.

[0011] The application provides an energy router state control method based on an operation and maintenance intelligent agent, and the routing strategy knowledge base is established through the following method: acquiring historical state data of the target energy router and conversion actions corresponding to the historical state data; extracting parameter characteristic values from the historical state data; obtaining frequent item data groups of state data and conversion actions by using a big data analysis algorithm according to the parameter characteristic values of the historical state data and the conversion actions corresponding to the historical state data; and storing the frequent item data groups into routing strategy knowledge bases corresponding to execution time dimensions of the frequent item data groups.

[0012] According to the energy router state control method based on the operation and maintenance agent provided by the application, the enhanced learning algorithm is used to optimize the corresponding conversion action of the target energy router based on the accumulated state data in the operation of the target energy router, the accumulated state data and the corresponding conversion action are used to expand the routing strategy knowledge base, and the data in the routing strategy knowledge base is used to retrain the strategy determination model to obtain an optimized strategy determination model when the retraining condition is reached.

[0013] According to the energy router state control method based on the operation and maintenance agent provided by the application, the enhanced learning algorithm is used to optimize the corresponding conversion action of the target energy router based on the accumulated state data in the operation of the target energy router, the accumulated state data and the corresponding conversion action are used to expand the routing strategy knowledge base, and the data in the routing strategy knowledge base is used to retrain the strategy determination model to obtain an optimized strategy determination model when the retraining condition is reached.

[0014] The application further provides an energy router state control device based on an operation and maintenance agent, which comprises the following modules. The first acquisition module is used to make a target operation and maintenance agent associated with a target energy router acquire state data when a transfer condition of a state machine of the target energy router is met, wherein the state data comprises state data of the target energy router itself and / or interaction data between an energy internet accessed by the target energy router and the target energy router; the state machine of the target energy router is used to describe different states in the operation process of the target energy router and conversion actions between the states; the second acquisition module is used to obtain a conversion action corresponding to the transfer condition by using a strategy determination model matched with the type of the state data according to the state data; wherein the strategy determination model is a machine learning model trained by using historical state data of the same type as the state data; and the execution module is used to instruct the target energy router to perform state transfer according to the conversion action.

[0015] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the energy router state control method based on the operation and maintenance agent according to any of the above when executing the computer program.

[0016] The application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the energy router state control method based on the operation and maintenance agent according to any of the above.

[0017] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the energy router state control method based on the operation and maintenance intelligent agent.

[0018] The energy router state control method and device based on the operation and maintenance intelligent agent provided by the application determine the conversion action corresponding to the transition condition by using a policy determination model matched with the type of the state data according to the obtained state data. Since the policy determination model is a machine learning model trained by using historical state data of the same type as the current state data, the model can better understand the features and rules of the current state data, so as to more accurately determine the conversion action and improve the scientificity and rationality of the decision. By instructing the target energy router to perform state transition according to the determined conversion action, the precise control of the running state of the energy router is realized. The state transition mode based on intelligent analysis and decision can make the energy router run more efficiently and stably, adapt to the complex and changeable operation environment of the energy internet, and improve the operation efficiency and reliability of the entire energy system. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0020] Figure 1 is a flowchart of the energy router state control method based on the operation and maintenance intelligent agent provided by the application.

[0021] Figure 2 is a flowchart of the method for establishing the routing policy knowledge base provided by the application.

[0022] Figure 3 is a state machine diagram of the energy router provided by the application.

[0023] Figure 4 is a control flow diagram of the operation and maintenance intelligent agent provided by the application.

[0024] Figure 5 is a schematic diagram of the artificial intelligence classification method of the routing policy knowledge base.

[0025] Figure 6 is a structural diagram of the energy router state control device based on the operation and maintenance intelligent agent provided by the application.

[0026] Figure 7 is a structural diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0027] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are only part of, rather than all of, the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0028] The present application provides an energy router state control method based on an operation and maintenance intelligent agent. Figures 1-5 The present application provides an energy router state control method based on an operation and maintenance intelligent agent.

[0029] Figure 1 The present application provides an energy router state control method based on an operation and maintenance intelligent agent. Figure 1 The present application provides an energy router state control method based on an operation and maintenance intelligent agent. Step 101, when the transition condition of the state machine of the target energy router is met, the target operation and maintenance intelligent agent associated with it acquires state data.

[0030] The state data includes the state data of the target energy router itself and / or the interaction data between the target energy router and the energy internet to which the target energy router accesses. The state machine of the target energy router is used to describe different states in its running process and the conversion actions between the states.

[0031] The energy router is a core device in the energy internet, and undertakes the key task of energy distribution, conversion and control. Specifically, the energy router is an intelligent control and conversion device, which realizes efficient and flexible configuration and utilization of energy by integrating distributed energy resources (such as solar energy, wind energy, etc.), energy storage devices and intelligent loads, and is an important cornerstone of the operation of the energy internet.

[0032] The operation and maintenance intelligent agent is an intelligent control unit or software program based on artificial intelligence technology for managing and promoting the state machine of the energy router. In the embodiments provided by the present application, the operation and maintenance intelligent agent collects and analyzes the state data of the energy router, generates corresponding control strategies by using big data analysis and machine learning algorithms, so as to realize intelligent and efficient operation management and control of the energy router. The multiple operation and maintenance intelligent agents in the energy internet are distributedly deployed, and through cooperation and game, the overall intelligence and robustness of the energy internet system can be effectively improved.

[0033] The target energy router is an energy router that needs to generate a control algorithm for accessing the energy internet. The target operation and maintenance intelligent agent is an operation and maintenance intelligent agent associated with the target energy router.

[0034] State data refers to all data related to the operating state of the target energy router, which is used to describe the current state of the energy router and its interaction with the energy internet. State data is an important basis for decision-making and control by the operation intelligent agent. State data can include but is not limited to the following data: State data of the target energy router itself: including but not limited to real-time operating parameters such as voltage, current, power, temperature, as well as configuration parameters, fault status, etc.

[0035] Interaction data between the target energy router and the energy internet to which it is connected: such as energy input and output of the energy router, communication data with other devices in the energy internet, energy scheduling instructions, etc.

[0036] State machine is a mathematical model used to describe different states of energy router during operation and the transition actions between states. The state machine of the energy router is used to define various operating states of the energy router (such as Figure 3 The start running, steady state, emergency / fault state, repair state, and maintain steady state shown in the figure) and the transition conditions and transition actions between these states. Among them, the start running state is the initialization and preparation of the energy router to receive energy input; the steady state is the stable energy distribution, conversion and control of the energy router under normal operating conditions; the emergency / fault state is entered when the energy router detects a fault or abnormal condition, and the corresponding protection measures are taken; the repair state is entered by the energy router after the fault is repaired, and necessary checks and recovery operations are performed; the maintain steady state (optimization): after repair or adjustment, the energy router reenters steady state operation and performs energy distribution and control according to the optimization strategy.

[0037] Transition conditions of the state machine are the rules or triggering factors that the energy router relies on to transition between different states. Transition conditions can be based on time-driven, event-driven and state-driven, and can include but are not limited to: Transition conditions from "start running" to "steady state" The energy router has completed initialization, all system parameters and configurations have been correctly set, the system has detected no abnormal signals, and all components are operating normally. When the above transition conditions are met, the energy router transitions from the "start running" state to the "steady state" and begins normal power supply and energy conversion work.

[0038] The transition condition from the "steady state" to the "emergency / failure state" is that the system detects abnormal signals such as overload, short circuit, equipment failure, etc., or the operating parameters of the energy router exceed the preset safety range. When any of the above conditions are met, the energy router is urgently converted from the "steady state" to the "emergency / failure state", and the corresponding protection mechanism (i.e. transition action) is triggered, such as cutting off the power supply, starting the backup device, etc.

[0039] The transition condition from the "emergency / failure state" to the "repair state" is that the cause of the failure has been identified and the repair measures are ready, or the system confirms through self-checking and diagnosis that the failure has been eliminated and all components are operating normally. When the above transition conditions are met, the energy router is converted from the "emergency / failure state" to the "repair state" for necessary maintenance and debugging.

[0040] The transition condition from the "repair state" to the "steady state (optimization)" is that the repair work has been completed, and the system confirms through self-checking and performance testing that all components are operating normally and the performance meets or exceeds the preset standard.

[0041] In the implementation process, the transition conditions of the state machine can be prioritized, and based on the priority, the system can be robust, optimized, and controlled in operation and related strategy scheduling by combining FIFO (First In First Out), shortest completion time, and minimum failure interval. The priority design principle of the transition conditions of the state machine is instantaneous action > medium and long-term action > medium and long-term management.

[0042] By defining the states and transition conditions of the state machine of the energy router, the operation and maintenance intelligent agent can generate corresponding transition actions according to the real-time state of the energy router, and realize intelligent and efficient operation management and control of the energy router.

[0043] Step 102, according to the state data, using a strategy determination model matched with the type of state data, obtaining a transition action corresponding to a transition condition.

[0044] The strategy determination model is a machine learning model trained using historical state data of the same type as the state data. The strategy determination model is used to determine the best transition action based on real-time state data during the transition of the state machine of the energy router.

[0045] The input data of the strategy determination model is historical state data of the same type as the current state data, which may include operating parameters of the energy router (such as voltage, current, power, etc.), device status (such as failure, normal, standby, etc.), environmental conditions (such as temperature, humidity, etc.), and interaction data with the energy internet (such as energy input and output, scheduling instructions, etc.).

[0046] The output data of the policy determination model is a conversion action corresponding to the state data, which can include control instructions of the energy router (such as turning on / off devices, adjusting operating parameters, etc.), etc.

[0047] The policy determination model can adopt various machine learning model structures, depending on the complexity of the state data and the requirements of the conversion action, which can include but are not limited to: Decision Tree: suitable for cases where state data has clear feature division and conversion rules.

[0048] Random Forest: by constructing multiple decision trees and combining their prediction results, to improve the accuracy and robustness of the model.

[0049] Neural Network: such as Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN, suitable for image or sequence data) or Recurrent Neural Network (RNN, suitable for time series data), etc., suitable for cases where state data is complex and conversion action requires highly nonlinear mapping.

[0050] Reinforcement Learning Model: such as Q-learning, Deep Q Network (DQN) or A3C, etc., which learns the optimal policy through interaction with the environment, suitable for scenarios that require dynamic adjustment and optimization.

[0051] The state data can include various types, such as device status, such as whether the energy router is running normally, whether there is a fault, fault type, etc.; interaction data with the energy internet, such as energy input and output, scheduling instructions, communication status with other devices, etc.

[0052] In some embodiments, the policy determination model can be trained in the following way: According to the type of state data processed by the policy determination model, obtain a training data set from the established routing policy knowledge base; use the training data set to train the initial machine learning model to obtain the policy determination model.

[0053] In some embodiments, the routing policy knowledge base includes multiple knowledge bases divided according to the execution time dimension of the conversion action of the state machine, and the execution time dimension of the conversion action corresponding to the state data can be determined according to the type of state data processed by the policy determination model; obtain a training data set from the routing policy knowledge base matching the execution time dimension.

[0054] In some embodiments, the routing strategy knowledge base comprises an instantaneous action expert knowledge base, a medium and long-term operation expert knowledge base, and a medium and long-term management knowledge base; the instantaneous action expert knowledge base corresponds to a first execution time dimension, the medium and long-term operation expert knowledge base corresponds to a second execution time dimension, and the medium and long-term management knowledge base corresponds to a third execution time dimension; the first execution time dimension is greater than the second execution time dimension, and the second execution time dimension is greater than the third execution time dimension.

[0055] The first execution time dimension refers to the control time range corresponding to the instantaneous action expert knowledge base, which is usually between milliseconds, seconds and minutes. The knowledge in this instantaneous action expert knowledge base is used to handle real-time fault prediction, judgment and repair, and emergency operation, etc. The relevant control range involves load equipment, bus and regional system, and needs to ensure the real-time and reliability of system operation. Therefore, its execution time dimension is the shortest, between milliseconds, seconds and minutes, requiring quick response and handling of sudden situations.

[0056] The second execution time dimension refers to the control time range corresponding to the medium and long-term operation expert knowledge base, which is between seconds and hours. The knowledge in the medium and long-term operation expert knowledge base is used to realize supply and demand matching and ensure the long-term stable operation of the energy internet, and the control range involves energy internet bus equipment and system at all levels, for reducing low-frequency network fluctuation and improving network optimization performance. Compared with the first execution time dimension, the second execution time dimension is longer, focusing on the stable operation and optimization of the system in a longer time range.

[0057] The third execution time dimension refers to the control time range corresponding to the medium and long-term management knowledge base, which ranges from minutes to days, months and years, and the control range includes regional system, provincial, national to global system. The medium and long-term management knowledge base is used to realize medium and long-term operation management, and medium and long-term configuration and operation of energy internet transaction and management, which needs to ensure the safety, economy and sustainable operation of the system. Compared with the first and second execution time dimensions, the third execution time dimension is the longest, focusing on the overall management and optimization of the energy internet system in a longer time range.

[0058] The above three execution time dimensions correspond to different types of tasks and demands, and by classifying and establishing corresponding knowledge bases, intelligent and efficient operation management and control of the energy router can be realized.

[0059] In the specific implementation process, if the execution time dimension is the first execution time dimension, the training data set is obtained from the instantaneous action expert knowledge base; if the execution time dimension is the second execution time dimension, the training data set is obtained from the medium and long-term operation expert knowledge base; and if the execution time dimension is the third execution time dimension, the training data set is obtained from the medium and long-term management knowledge base.

[0060] For the process of establishing the routing strategy knowledge base, refer to the related content in Figure 2 , which will not be repeated here.

[0061] The training data set includes state data as sample data and conversion actions as labels.

[0062] In the specific implementation process, a loss function can be constructed to measure the difference between the prediction result output by the initial machine learning model for the sample data and the corresponding label of the sample data; a preset optimization algorithm (for example, a gradient descent optimization algorithm) is used to adjust the parameters of the initial machine learning model, so that the loss function gradually converges, and finally a trained strategy determination model is obtained.

[0063] Step 103, instruct the target energy router to perform state transition according to the conversion action.

[0064] In the specific implementation process, the target operation and maintenance agent can send the conversion action to the control system of the target energy router, and the control system performs state transition according to the control parameters described in the conversion action, for example, from state stable to state repair.

[0065] In some embodiments, in the energy internet, an operation and maintenance agent is associated with each energy router, and the carriers (i.e., hardware platforms running the operation and maintenance agents) of all operation and maintenance agents are connected through optical fibers as transmission media according to the connection topology of each energy router in the energy internet, forming an operation and maintenance agent network. The operation and maintenance agents can communicate with each other through the optical fiber-Infiniband communication protocol, which is a communication protocol combining optical fiber transmission medium and Infiniband high-speed interconnection technology. The target operation and maintenance agent can share the conversion action corresponding to the transition condition with other operation and maintenance agents in the energy internet through the operation and maintenance agent network.

[0066] In this way, the operation and maintenance agents associated with all energy routers can realize real-time sharing of knowledge, improving the intelligence and collaborative working ability of the entire energy internet system.

[0067] In the specific implementation process, if there are multiple conversion actions shared by operation and maintenance agents that conflict, a multi-expert voting mechanism (MOE) can be used to select the conversion action with the most votes or the highest score as the next candidate action.

[0068] In some embodiments, an enhanced learning algorithm can be used to optimize the corresponding conversion action of the target energy router based on the accumulated state data in the operation of the target energy router, expand the routing strategy knowledge base using the accumulated state data and the corresponding conversion action, and retrain the strategy determination model using the data in the routing strategy knowledge base when the retraining condition is reached, to obtain an optimized strategy determination model. In the specific implementation process, the A3C enhanced learning algorithm can be used to continue training the strategy determination model based on the distributed deployment characteristics of the operation and maintenance agent in the energy internet. In addition to enhanced learning, a pre-trained large model and a small amount of sample fine-tuning method, such as a typical transformer pre-training model architecture, can be used to continue training the strategy determination model.

[0069] In some embodiments, the energy internet is a source-network-load integrated energy internet, as shown in FIG. 1, and the embodiments provided by the present application establish routing strategy knowledge bases of different execution time dimensions, and then train a plurality of strategy determination models of different execution time dimensions using the routing strategy knowledge bases of different execution time dimensions. The target operation and maintenance agent uses the strategy determination models of different execution time dimensions to provide the energy router, which is the core control hub of the energy internet, with action strategies represented by conversion actions that can ensure the smooth and robust operation of the entire regional system. Figure 4

[0070] For transient faults, the target operation and maintenance agent can use the strategy determination model trained based on the transient action expert knowledge base to give conversion actions for transient faults or make a pre-judgment of transient faults, so that the reaction capability of the energy internet to transient faults is effectively enhanced, and the related operation fault indicators can be greatly improved.

[0071] For the medium and long-term stable operation of the energy internet, the target operation and maintenance agent can use the strategy determination model trained based on the medium and long-term operation expert knowledge base to give conversion actions for energy supply and demand matching, ensure the long-term stable operation of the energy internet, reduce the low-frequency overall fluctuation, and effectively improve the performance of the energy internet.

[0072] For the benefit optimization of the energy internet, the target operation and maintenance agent can use the strategy determination model trained based on the medium and long-term management knowledge base to give conversion actions for the medium and long-term configuration and operation of the energy internet, realize the timely optimization configuration of the energy internet, reduce the energy router supply and demand cost, and ensure the optimal system benefit.

[0073] Figure 2 FIG. 1 is a flowchart of the method for establishing the routing strategy knowledge base provided by the present application, as shown in FIG. 1, the method comprises the following steps: Figure 2 ​​Step 201, obtaining historical state data of the target energy router and corresponding conversion actions.

[0074] For detailed description of the historical state data and the conversion actions, refer to the description of the state data and the conversion actions in step 101, which will not be repeated here.

[0075] In the specific implementation process, it is necessary to collect the historical state data of the target energy router. For example, the system running parameters and running samples related to each aspect of the target energy router are collected, which include but are not limited to static configuration parameters, dynamic running parameters, system topology and device threshold values, etc.

[0076] The static configuration parameters can include the hardware configuration of the router, the software version, the network interface information, etc.

[0077] The dynamic running parameters include real-time state data of the router, such as input and output power, voltage and current, temperature, load condition, etc.

[0078] Step 202, extracting parameter characteristic values from the historical state data.

[0079] In the specific implementation process, the parameter characteristic values can be extracted from the historical state data in the following ways.

[0080] For the collected state data with continuous value and unstructured parameter value, the corresponding parameter characteristic values can be obtained by discretization and parameterization method.

[0081] For example, based on the continuous value state data, the corresponding parameter characteristic values can be obtained by uniform quantization, and the quantization step (for example, 1) and the quantization result range are set based on experience.

[0082] For unstructured parameters, the corresponding parameter characteristic values can be obtained by integer one-to-one mapping, and the maximum value corresponds to the number of parameter categories.

[0083] Step 203, using big data analysis algorithm to obtain frequent item data set of state data and conversion actions according to the parameter characteristic values of the historical state data and the corresponding conversion actions.

[0084] The big data analysis algorithm can use Apriori or FP-growth algorithm. The latter is simple and efficient in calculation but needs large memory to save FP-tree, so it becomes the recommended algorithm. The frequent item calculation threshold can be set to 0.005-0.05 (the smaller the threshold, the more the number of frequent items, and the more the number of generated knowledge), and the data set of state data and conversion actions with an appearance frequency greater than the frequent item calculation threshold can be determined as the frequent item data set.

[0085] For example, the frequent item calculation threshold is set to >0.6 to >0.8, and the conditional probability of the parameter characteristic value of the historical state data and the data group composed of the conversion action B Within the above range, it is determined that the state data A and the conversion action B frequently occur or have a certain causal relationship, and the two can be used as a frequent item data group.

[0086] Step 204, according to the execution time dimension corresponding to the frequent item data group, the frequent item data group is stored in the routing strategy knowledge base corresponding to the execution time dimension.

[0087] In the specific implementation process, since there are complex and variable features or simultaneous support for multiple execution time dimensions of business, such as Figure 5 As shown, artificial intelligence classification technology can be used to consider factors such as action execution time span of conversion action, state sampling time / frequency before action, state sampling time / frequency after action, business type preliminary division result, knowledge update time / frequency, etc. The frequent item data group is stored in the routing strategy knowledge base corresponding to the execution time dimension.

[0088] In the specific implementation process, the characteristics of the knowledge stored in the instantaneous action expert knowledge base are single-point or continuous action sequence exceeding (threshold); the typical knowledge characteristics stored in the medium and long-term operation expert knowledge base are that the variance of the conversion action sequence is lower than the stability threshold; and the typical knowledge characteristics stored in the medium and long-term management knowledge base are that the system benefit is maximized or meets the system robustness requirement. Related thresholds can be obtained through big data analysis or generated through deep learning neural network modeling.

[0089] The energy router state control device based on the operation and maintenance agent provided by the application will be described below. The energy router state control device based on the operation and maintenance agent described below can be correspondingly referred to the energy router state control method based on the operation and maintenance agent described above.

[0090] Figure 6 FIG. 1 is a structural schematic diagram of the energy router state control device based on the operation and maintenance agent provided by the application. As shown in Figure 6 The energy router state control device based on the operation and maintenance agent includes the following modules.

[0091] The first acquisition module 610 is configured to acquire state data by a target operation and maintenance agent associated with a target energy router when a transition condition of a state machine of the target energy router is met; wherein the state data includes state data of the target energy router itself and / or interaction data between an energy internet accessed by the target energy router and the target energy router; and the state machine of the target energy router is used to describe different states in the running process of the target energy router and conversion actions between the states.

[0092] The second acquisition module 620 is used to determine the transition action corresponding to the transition condition by using a strategy that matches the type of the state data based on the state data; wherein the strategy determination model is a machine learning model trained using historical state data of the same type as the state data.

[0093] The execution module 630 is used to instruct the target energy router to perform a state transition according to the conversion action.

[0094] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute an energy router state control method based on an operation and maintenance intelligent agent. This method includes: when the transition conditions of the state machine of the target energy router are met, the associated target operation and maintenance intelligent agent acquires state data; wherein the state data includes the target energy router's own state data, and / or the interaction data between the energy internet accessed by the target energy router and the target energy router; the target energy router's state machine is used to describe different states during its operation and the transition actions between states; based on the state data, a strategy determination model matching the type of the state data is used to obtain the transition action corresponding to the transition condition; wherein the strategy determination model is a machine learning model trained using historical state data of the same type as the state data; and instructing the target energy router to perform a state transition according to the transition action.

[0095] In addition, the logic instructions in the memory 730 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0096] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the energy router state control method based on the operation and maintenance agent provided by the above-mentioned method. The method comprises: when the transition condition of the state machine of the target energy router is met, the target operation and maintenance agent associated with the target energy router acquires state data; wherein the state data comprises state data of the target energy router itself, and / or interaction data between the energy internet accessed by the target energy router and the target energy router; the state machine of the target energy router is used to describe different states in its running process and the conversion action between the states; according to the state data, a policy determination model matched with the type of the state data is used to obtain the conversion action corresponding to the transition condition; wherein the policy determination model is a machine learning model trained by using historical state data of the same type as the state data; the target energy router is instructed to perform state transition according to the conversion action.

[0097] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the energy router state control method based on an operation and maintenance agent provided by each of the above methods, the method comprising: when a transition condition of a state machine of a target energy router is met, a target operation and maintenance agent associated with the target energy router acquires state data; wherein the state data comprises state data of the target energy router itself and / or interaction data between an energy internet accessed by the target energy router and the target energy router; the state machine of the target energy router is used to describe different states in the running process of the target energy router and the conversion actions between the states; according to the state data, a conversion action corresponding to the transition condition is obtained by using a policy determination model matched with the type of the state data; wherein the policy determination model is a machine learning model trained using historical state data of the same type as the state data; and instructing the target energy router to perform state transition according to the conversion action.

[0098] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0099] From the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and the necessary general hardware platform, and of course, it can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0100] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An energy router state control method based on an operation and maintenance agent, characterized in that, Comprise: When the transition condition of the state machine of the target energy router is met, the target operation and maintenance agent associated therewith acquires state data; wherein the state data comprises state data of the target energy router itself, and / or interaction data between the target energy router and the energy internet accessed thereby; the state machine of the target energy router is used to describe different states in the running process thereof and the conversion actions between the states; According to the state data, a strategy determination model matched with the type of the state data is used to obtain the conversion action corresponding to the transition condition; wherein the strategy determination model is a machine learning model trained using historical state data of the same type as the state data; The target energy router is instructed to perform state transition according to the conversion action.

2. The energy router state control method based on operation and maintenance intelligent agents according to claim 1, characterized in that, The strategy determination model is trained in the following manner: According to the type of state data processed by the strategy determination model, a training data set is obtained from an established routing strategy knowledge base; The initial machine learning model is trained using the training data set to obtain the strategy determination model. 3.The energy router state control method based on operation and maintenance intelligent agent according to claim 2, characterized in that, The routing strategy knowledge base comprises a plurality of knowledge bases divided according to the execution time dimension of the conversion action of the state machine; According to the type of state data, the execution time dimension of the conversion action corresponding to the state data is determined; The training data set is obtained from the routing strategy knowledge base matched with the execution time dimension. The routing strategy knowledge base comprises an instantaneous action expert knowledge base, a medium and long term running expert knowledge base, and a medium and long term management knowledge base; the instantaneous action expert knowledge base corresponds to a first execution time dimension, the medium and long term running expert knowledge base corresponds to a second execution time dimension, and the medium and long term management knowledge base corresponds to a third execution time dimension; the first execution time dimension is greater than the second execution time dimension, and the second execution time dimension is greater than the third execution time dimension; 4. The energy router state control method based on operation and maintenance intelligent agent according to claim 3, characterized in that, The training data set is obtained from the routing strategy knowledge base matched with the execution time dimension, comprising: If the execution time dimension is the first execution time dimension, the training data set is obtained from the instantaneous action expert knowledge base; If the execution time dimension is the second execution time dimension, the training data set is obtained from the medium and long term running expert knowledge base; If the execution time dimension is the third execution time dimension, the training data set is obtained from the medium and long term management knowledge base. The routing strategy knowledge base is established in the following manner:

5. The energy router state control method based on the operation and maintenance intelligent agent according to any one of claims 2-4, characterized in that, Historical state data of the target energy router and the conversion action corresponding thereto are obtained; Parameter characteristic values are extracted from the historical state data; Using a big data analysis algorithm, the parameter characteristic values of the historical state data and the conversion action corresponding thereto are used to obtain frequent item data groups of state data and conversion actions; ​ According to an execution time dimension corresponding to the frequent item data set, the frequent item data set is stored in a routing strategy knowledge base corresponding to the execution time dimension.

6. The energy router state control method based on the operation and maintenance intelligent agent according to claim 5, characterized in that, The method further comprises: Using a reinforcement learning algorithm, based on accumulated state data in the running of the target energy router, optimizing its corresponding conversion action; Using accumulated state data and its corresponding conversion action, expanding the routing strategy knowledge base; When a retraining condition is reached, using data in the routing strategy knowledge base to retrain the strategy determination model to obtain an optimized strategy determination model. 7.The method of claim 1, wherein, In the energy internet, an operation and maintenance agent is associated with each energy router, and the carriers of all operation and maintenance agents are connected through optical fibers as transmission media according to the connection topology of each energy router in the energy internet, forming an operation and maintenance agent network. The method further comprises: The target operation and maintenance agent shares the conversion action corresponding to the transfer condition with other operation and maintenance agents in the energy internet through the operation and maintenance agent network.

8. An energy router state control device based on an operation and maintenance agent, characterized in that, Comprise: A first acquisition module is configured to acquire state data by a target operation and maintenance agent associated with a target energy router when a transfer condition of a state machine of the target energy router is met; wherein the state data includes state data of the target energy router itself and / or interaction data between an energy internet accessed by the target energy router and the target energy router; the state machine of the target energy router is used to describe different states in its running process and conversion actions between states; A second acquisition module is configured to obtain a conversion action corresponding to the transfer condition by using a strategy determination model matched with the type of the state data according to the state data; wherein the strategy determination model is a machine learning model trained using historical state data of the same type as the state data; An execution module is configured to instruct the target energy router to perform state transfer according to the conversion action.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to realize the energy router state control method based on an operation and maintenance agent according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the energy router state control method based on an operation and maintenance agent according to any one of claims 1 to 7.

Images