Task allocation method, system, device and equipment, storage medium and program product

Abstract

The invention relates to a task allocation method, device and system, equipment, a storage medium and a program product. The method is used for a cloud center server in a power network system, the power network system comprises power equipment, edge nodes and the cloud center server, and the method comprises the steps of obtaining operation states of the power equipment in the power network system; and if the operation state of the power equipment is abnormal, task allocation is performed based on a digital twin model and a preset deep reinforcement learning model, and the task allocation is used for migrating tasks in the power equipment to edge nodes or other power equipment for processing. By adopting the method, the task processing efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of power network technology, and in particular to a task allocation method, device, equipment, storage medium and program product. Background Art

[0002] With the rapid development of the economy and society, the demand for electricity has grown exponentially, the large-scale access of smart power equipment and the rapid increase in the number of power network nodes have formed a complex and huge power Internet of Things.

[0003] In the prior art, data processing or caching is performed on smart power devices, that is, the smart power devices have certain computing and caching capabilities to perform real-time processing and transmission of data in tasks.

[0004] However, the amount of data in the power network system is huge, but the communication and computing resources of smart power equipment are limited. Frequent data processing requests cause great pressure on data processing transmission and storage resources. It is difficult to meet the task processing needs with huge data volumes based on the existing methods, which makes the task processing efficiency low. Summary of the invention

[0005] Based on this, it is necessary to provide a task allocation method, device, equipment, storage medium and program product that can improve task processing efficiency in response to the above technical problems.

[0006] In a first aspect, the present application provides a task allocation method for a cloud center server in a power network system, the power network system including power equipment, edge nodes and a cloud center server, the method comprising:

[0007] Obtain the operating status of each power device in the power network system;

[0008] If the operating status of the power equipment is abnormal, task allocation is performed based on the digital twin model and the preset deep reinforcement learning model. Task allocation is used to migrate tasks in the power equipment to edge nodes or other power equipment for processing.

[0009] In one embodiment, task allocation is performed based on a digital twin model and a preset reinforcement learning model, including:

[0010] The cost minimization problem of the power network system is determined based on the digital twin model. Taking the minimization of the cost of the power network system as the optimization goal, the task allocation strategy is determined based on the preset deep reinforcement learning model, so as to migrate the tasks in the power network system to edge nodes or other power equipment for processing according to the task allocation strategy.

[0011] In one of the embodiments, the preset reinforcement learning model is implemented based on the QMIX reinforcement learning algorithm. The preset reinforcement learning model includes an agent network, a hybrid network and a super network. The preset reinforcement learning model decomposes the overall action-state value function. Each agent network is used to determine the local action-state value function of each power device. The hybrid network is used to generate an overall action-state value function in a monotonic form from the local action-state value function of each power device. The super network is used to generate weights and biases in the hybrid network based on the global state information of the power network system.

[0012] In one embodiment, the operating state of each power device is determined by a preset abnormality recognition model in each power device according to the operating parameters of each power device, and the training process of the preset abnormality recognition model includes:

[0013] Receive model parameters of the abnormality recognition model in each power device; aggregate and update each model parameter, and send the updated model parameters to the power device, so that each power device trains the abnormality recognition model based on the updated model parameters.

[0014] In one embodiment, the method further comprises:

[0015] Obtain the recognition accuracy of the abnormal recognition model in each power device; determine the incentive mechanism corresponding to each power device based on the recognition accuracy, and the incentive mechanism is used to represent the weight of the model parameters corresponding to each power device when the model parameters are aggregated and updated.

[0016] In a second aspect, the present application also provides a task allocation system, which includes a cloud center server, power equipment and edge nodes;

[0017] The cloud center server is used to obtain the operating status of each power device. If the operating status of the power device is abnormal, task allocation is performed based on the digital twin model and the preset deep reinforcement learning model. Task allocation is used to migrate tasks in the power device to edge nodes or other power devices for processing.

[0018] The electric power equipment is used to receive tasks from the electric power equipment in abnormal operation state and process the tasks;

[0019] The edge node is used to receive tasks from power equipment with abnormal operating status and process the tasks.

[0020] In a third aspect, the present application further provides a task allocation device for a cloud center server in a power network system, the power network system including power equipment, edge nodes and a cloud center server, the device including:

[0021] An acquisition module, used to acquire the operation of each power device in the power network system;

[0022] The allocation module is used to allocate tasks based on the digital twin model and the preset deep reinforcement learning model if the operating status of the power equipment is abnormal. The task allocation is used to migrate the tasks in the power equipment to the edge nodes or other power equipment for processing.

[0023] In one of the embodiments, the allocation module is specifically used to determine the cost minimization problem of the power network system based on the digital twin model; taking the minimization of the cost of the power network system as the optimization goal, determining the task allocation strategy based on the preset deep reinforcement learning model, so as to migrate the tasks in the power network system to edge nodes or other power equipment for processing according to the task allocation strategy.

[0024] In one of the embodiments, the preset reinforcement learning model is implemented based on the QMIX reinforcement learning algorithm. The preset reinforcement learning model includes an agent network, a hybrid network and a super network. The preset reinforcement learning model decomposes the overall action-state value function. Each agent network is used to determine the local action-state value function of each power device. The hybrid network is used to generate an overall action-state value function in a monotonic form from the local action-state value function of each power device. The super network is used to generate weights and biases in the hybrid network based on the global state information of the power network system.

[0025] In one of the embodiments, the operating status of each power device is determined by a preset abnormality recognition model in each power device according to the operating parameters of each power device. The task allocation device also includes a training module, which is used to receive model parameters of the abnormality recognition model in each power device; aggregate and update each model parameter, and send the updated model parameters to the power device, so that each power device trains the abnormality recognition model based on the updated model parameters.

[0026] In one of the embodiments, the training module is also used to obtain the recognition accuracy of the abnormal recognition model in each power device; the incentive mechanism corresponding to each power device is determined based on the recognition accuracy, and the incentive mechanism is used to represent the weight of the model parameters corresponding to each power device when the model parameters are aggregated and updated.

[0027] In a fourth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements any of the methods described in the first aspect when executing the computer program.

[0028] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the methods described in the first aspect above.

[0029] In a sixth aspect, the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements any of the methods described in the first aspect above.

[0030] The above-mentioned task allocation method, system, device, equipment, storage medium and program product obtain the operating status of each power equipment in the power network system; if the operating status of the power equipment is abnormal, task allocation is performed based on the digital twin model and the preset deep reinforcement learning model. The task allocation is used to migrate the tasks in the power equipment to the edge nodes or other power equipment for processing. In this way, the operating status of each power equipment can be detected. When the operating status of the power equipment is found to be abnormal, task allocation is performed through the digital twin model and the preset deep reinforcement learning model, and part of the processing tasks in the power equipment are migrated to the edge nodes or other power equipment. These processing tasks are processed by the edge nodes or other power equipment, thereby alleviating the processing pressure of the power equipment and improving the efficiency of task processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0032] FIG1 is a schematic diagram of a process flow of a task allocation method in one embodiment;

[0033] FIG2 is a schematic diagram of a process of performing task allocation steps based on a digital twin model and a preset reinforcement learning model in one embodiment;

[0034] FIG3 is a flow chart of a training process of a preset abnormality recognition model in one embodiment;

[0035] FIG4 is a schematic diagram of a flow chart of a task allocation method in another embodiment;

[0036] FIG5 is a diagram of a digital twin power network system architecture in one embodiment;

[0037] FIG6 is a block diagram of a task allocation device in one embodiment;

[0038] FIG. 7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0040] With the rapid development of the economy and society, the demand for electricity has grown exponentially, the large-scale access of smart power equipment and the rapid increase in the number of power network nodes have formed a complex and huge power Internet of Things.

[0041] In the prior art, data is processed or cached in the smart power device, that is, the smart power device has a certain computing and caching capability to process and transmit data in real time.

[0042] However, the amount of data in the power network system is huge, but the communication and computing resources of smart power equipment are limited. Frequent data processing requests cause great pressure on data processing transmission and storage resources. It is difficult to meet the task processing needs with huge data volumes based on the existing methods, which makes the task processing efficiency low.

[0043] In view of this, the present application provides a task allocation method that can improve the efficiency of task processing. The task allocation method provided in the embodiment of the present application, its execution subject can be a task allocation device, and the task allocation device can be implemented by software, hardware, or a combination of software and hardware. It can be embedded in or independent of the processor in the computer device in the form of hardware, or it can be stored in the memory of the computer device in the form of software. The computer device can be a server, and the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. In the following method embodiments, the execution subject is an example of a cloud center server for explanation.

[0044] In an exemplary embodiment, as shown in FIG1 , a task allocation method is provided for a cloud center server in a power network system, the power network system includes power equipment, edge nodes and a cloud center server, and the method includes the following steps 101 to 102. Among them:

[0045] S101, obtaining the operating status of each power device in the power network system.

[0046] Optionally, the power network system is based on a highly flexible cloud-edge-end collaborative architecture. Optionally, cloud-edge-end collaboration can achieve seamless data processing and resource sharing between cloud center servers, edge nodes and power equipment to improve computing efficiency and response speed to meet the needs of different application scenarios.

[0047] Optionally, smart grid applications include automated dispatching, real-time monitoring, and emergency response, which put forward communication requirements and challenges such as low latency, high reliability, and high mobility. The traditional power network architecture cannot meet the networking needs of various power equipment, distributed energy, and heterogeneous network edge devices due to its singleness, low flexibility, and poor real-time performance. Power equipment needs to migrate its tasks to the cloud and multi-access edge computing servers through the network for collaborative computing to support large-scale services and applications.

[0048] In one possible implementation, the operating status of each power device can be determined by obtaining the operating parameters and unique identification information of each power device and analyzing the operating parameters. The operating parameters may include the real-time voltage, current, and temperature collected by each power device, as well as the computing resources, network delay, and equipment energy consumption of each power device.

[0049] In another possible implementation method, each power device may analyze its own operating parameters, and after determining its own operating status, the operating status of the power device and the unique identification information of the power device may be sent to a cloud center server, which then obtains the operating status of each power device.

[0050] Optionally, the operating status of each electrical device may be a normal status or an abnormal status.

[0051] S102: If the operating status of the power equipment is abnormal, task allocation is performed based on the digital twin model and the preset deep reinforcement learning model.

[0052] Among them, task allocation is used to migrate tasks in power equipment to edge nodes or other power equipment for processing.

[0053] Optionally, when the operating state of the power equipment is abnormal, it may be caused by an abnormality in the hardware of the power equipment, or it may be caused by insufficient computing resources of the power equipment, etc., and the embodiments of the present application are not limited to this.

[0054] Optionally, when the operating status of the power equipment is abnormal, it indicates that the current power equipment is temporarily unable to effectively process the task data, and the tasks in the power equipment need to be migrated to other power equipment for processing, or migrated to an edge node for processing, or migrated to a cloud center server for processing.

[0055] Optionally, the digital twin model can be a sensor and virtual simulation technology that is used to accurately map and simulate the state of the physical power network. It relies on a large number of sensors to collect data about physical entities, such as temperature, pressure, operating speed, etc., and then uses these data to build a complete digital model in a virtual environment. This model is not only similar to the actual object in appearance, but also can reflect the state changes of the object in real time, and can be used for simulation and analysis to optimize the performance of the physical entity.

[0056] Optionally, the digital twin model can be obtained by simulating the actual prototype state data of the existing power grid system in the state space. It is a twin simulation body that is the same as the actual power grid system. Multi-agent information sharing between power equipment, between power equipment and the digital twin model, and within the digital twin model requires the consumption of system computing resources and communication resources. The unloading strategy, that is, the task allocation strategy, is reasonably selected according to the deep reinforcement learning model to perform dynamic, efficient, and intelligent resource scheduling.

[0057] Optionally, the deep reinforcement learning model can be a technology that combines deep learning and reinforcement learning, learns to make decisions based on trial and error, uses neural networks to represent strategies and value functions to solve decision-making problems, and enables the agent to achieve optimal performance in the power network system, wherein the agent can be an entity that can learn and make decisions, multiple agents make decisions simultaneously, and the decision-making processes of the agents influence each other, and the agent can be a device of the power network system.

[0058] The above-mentioned task allocation method obtains the operating status of each power equipment in the power network system; if the operating status of the power equipment is abnormal, task allocation is performed based on the digital twin model and the preset deep reinforcement learning model. The task allocation is used to migrate the tasks in the power equipment to the edge nodes or other power equipment for processing. In this way, the operating status of each power equipment can be detected. When the operating status of the power equipment is found to be abnormal, task allocation is performed through the digital twin model and the preset deep reinforcement learning model, and part of the processing tasks in the power equipment are migrated to the edge nodes or other power equipment. These processing tasks are processed by the edge nodes or other power equipment, thereby alleviating the processing pressure of the power equipment and improving the efficiency of task processing.

[0059] In an exemplary embodiment, as shown in FIG2 , optionally, task allocation is performed based on a digital twin model and a preset reinforcement learning model, including the following steps 201 to 202 . Among them:

[0060] S201, determining the power network system cost minimization problem based on the digital twin model.

[0061] Optionally, a system cost minimization problem is constructed based on the digital twin model, aiming to jointly optimize the service delay and energy consumption of the power network, focusing on determining the offloading strategy and resource allocation of computationally intensive tasks of power equipment, thereby reducing processing costs and improving overall benefits.

[0062] S202, taking minimizing the cost of the power network system as the optimization goal, determining a task allocation strategy based on a preset deep reinforcement learning model, so as to migrate tasks in the power network system to edge nodes or other power equipment for processing according to the task allocation strategy.

[0063] In one possible implementation, the preset deep reinforcement learning model can be implemented based on a deep Q-network, which uses a neural network to approximate the Q-value function, which is a special value function that estimates the expected cumulative reward starting from a given state, taking a specific action, and then following a specific strategy.

[0064] In another possible implementation method, the preset reinforcement learning model is implemented based on the QMIX reinforcement learning algorithm. The preset reinforcement learning model includes an agent network, a hybrid network and a super network. The preset reinforcement learning model decomposes the overall action-state value function. Each agent network is used to determine the local action-state value function of each power device. The hybrid network is used to generate an overall action-state value function in a monotonic form from the local action-state value function of each power device. The super network is used to generate weights and biases in the hybrid network based on the global state information of the power network system.

[0065] Optionally, the QMIX reinforcement learning algorithm seeks a balance between agents. It works by expressing the joint action-state value function as a nonlinear combination of the local action-state value functions of each agent. The agent network adopts a deep recurrent Q network to cope with the Markov decision process of the agents in the power grid under partially observable conditions and better handle missing information.

[0066] Optionally, during the training process of the preset deep reinforcement learning model, a target Q network is set to generate a target Q value, the mean square error is used as the loss function, the predicted overall action-state value function is compared with the target Q value, the loss function is optimized by the back propagation algorithm, the parameters of the local Q network and the hybrid network are adjusted to more accurately predict the overall action-state value function, and the loss function is minimized through end-to-end training to optimize task allocation and improve the overall benefits of the power network and task processing efficiency.

[0067] Optionally, after determining the allocation strategy, the digital twin model can be used to simulate the allocation strategy in a virtual environment to verify and apply the allocation strategy to achieve collaborative optimization of the virtual and the real.

[0068] Optionally, when verifying the allocation strategy, the status of each power device and edge node in the power network system is monitored in real time. Through a closed-loop feedback mechanism, after the task is migrated according to the allocation strategy, if the status of each power device and edge node does not reach the preset state, the allocation strategy is adjusted in time to determine the reliability of the task allocation method in a complex and changing environment.

[0069] In an exemplary embodiment, as shown in FIG3 , optionally, the operating state of each power device is determined by a preset abnormality recognition model in each power device according to the operating parameters of each power device, and the training process of the preset abnormality recognition model includes the following steps 301 to 302. Among them:

[0070] S301, receiving model parameters of an abnormality identification model in each electric power device.

[0071] Optionally, personalized federated learning is used to train anomaly recognition models on decentralized power devices instead of centralizing data in a single data center to protect data privacy. Each power device uses historical data to train the model locally and uploads the model parameters to the cloud center server.

[0072] Optionally, when training the local model of the power equipment, the initial abnormality recognition model distributed by the cloud center server can be obtained, and the historical operation data of the power equipment can be obtained. After preprocessing the historical operation data such as data cleaning and normalization, the preprocessed historical data can be used as training samples to train the initial abnormality recognition model.

[0073] Optionally, the power equipment may send the model parameters to the cloud center server after a specific round of training, and the cloud center server will receive the model parameters corresponding to each power device; or the model parameters may be sent to the cloud center server after each training, and the cloud center server will receive the model parameters corresponding to each power device.

[0074] S302, aggregate and update each model parameter, and send the updated model parameters to the power equipment, so that each power equipment trains the abnormality recognition model based on the updated model parameters.

[0075] Optionally, an aggregation algorithm may be used to merge updates of model parameters to generate a new global model, and then the model parameters corresponding to the new global model are sent to each power device.

[0076] In a possible implementation, the same weight may be assigned to the model update of each power device, and then the updates of the model parameters may be combined through a simple averaging method.

[0077] In another possible implementation, different weights may be assigned to the model updates of the various power devices according to the model quality or data volume of the various power devices, and then the updates of the various model parameters may be combined by weighted averaging.

[0078] The above-mentioned method receives the model parameters of the abnormality recognition model in each power device, aggregates and updates each model parameter, and sends the updated model parameters to the power equipment, so that each power equipment can train the abnormality recognition model based on the updated model parameters. The model parameters of its own can be shared and integrated through the federated learning mechanism, which can effectively cope with the challenges brought by data diversity and heterogeneity, ensure data privacy and security, and improve the reliability of model training.

[0079] Optionally, the deep reinforcement learning model mentioned above can be trained through personalized federated learning. In this way, efficient data processing and intelligent decision-making are achieved in the power network, so that the power system can better cope with complex and changing environmental requirements; or the model can be customized according to the needs of the power equipment itself for model training, while ensuring the contribution of each power equipment to the global model, so as to achieve a balance between global optimization and personalized needs.

[0080] In an exemplary embodiment, as shown in FIG4 , optionally, the method further includes the following steps 401 to 402. Among them:

[0081] S401, obtaining the recognition accuracy of the abnormality recognition model in each power device.

[0082] Optionally, in order to achieve a long-term stable federated learning environment and attract high-quality data providers, a fair user reward mechanism can be established, in which users can reward each power device to encourage each power device to actively participate and share data.

[0083] Optionally, under the digital twin model, each power device is mapped as a digital twin, which can monitor the training status of the abnormality recognition model in the power network in real time.

[0084] Optionally, during the global model training process, the recognition accuracy of the abnormality recognition model in each power device during each local training may be obtained.

[0085] S402, determining the incentive mechanism corresponding to each power device according to the recognition accuracy.

[0086] Among them, the incentive mechanism is used to represent the weight of the model parameters corresponding to each power equipment when the model parameters are aggregated and updated.

[0087] Optionally, when performing aggregate update on the model parameters, the weight of the model parameters corresponding to the model whose recognition accuracy is greater than the preset accuracy threshold may be higher, and the weight of the model parameters corresponding to the model whose recognition accuracy is lower than the preset accuracy threshold may be lower.

[0088] Optionally, by real-time monitoring of the participation and data quality of each power device, and dynamically adjusting the reward mechanism according to the changing network environment and demand, the fairness and effectiveness of the reward mechanism can be ensured through an intelligent feedback mechanism, further improving user enthusiasm and satisfaction.

[0089] Optionally, the incentive mechanism may also include data sharing rewards, resource utilization rewards, and continuous participation rewards, which are intended to encourage power devices to provide high-quality data and participate in the collaboration of edge computing resources, thereby maintaining the stability of the federated learning environment.

[0090] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0091] In an exemplary embodiment, the present application embodiment also provides a task allocation system, which includes a cloud center server, power equipment and edge nodes;

[0092] The cloud center server is used to obtain the operating status of each power device. If the operating status of the power device is abnormal, task allocation is performed based on the digital twin model and the preset deep reinforcement learning model. Task allocation is used to migrate tasks in the power device to edge nodes or other power devices for processing.

[0093] The electric power equipment is used to receive tasks from the electric power equipment in abnormal operation state and process the tasks;

[0094] The edge node is used to receive tasks from power equipment with abnormal operating status and process the tasks.

[0095] Optionally, the cloud center server is specifically used to implement the steps described in any of the above method embodiments.

[0096] In an exemplary embodiment, as shown in Figure 5, an exemplary digital twin power network system architecture diagram is proposed. In this power network system architecture diagram, a power network 501 and a digital twin network 502 are included. The digital twin network 502 is a simulation of the power network 501, which is structurally consistent with the power network 501 and can be used for simulation and analysis to optimize the performance of the power network 501.

[0097] Exemplarily, the power network 501 may include a data cache unit 5011, various types of power equipment 5012, 5013 and 5014, a roadside unit 5015 and an edge service unit 5016, as well as a local model 5017 deployed in each power device and a global model 5018 deployed in an edge node or a cloud center server; correspondingly, the digital twin network 502 may include a data cache unit 5021, various types of power equipment 5022, 5023 and 5024, a roadside unit 5025 and an edge service unit 5026.

[0098] Exemplarily, when performing task processing, communication resources, cache resources, and computing resources in the power network 501 may be analyzed and predicted, and optimized scheduling may be performed.

[0099] Those skilled in the art will understand that the digital twin power network system architecture diagram shown in Figure 5 is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the digital twin power network system architecture diagram to which the scheme of the present application is applied. The specific digital twin power network system architecture may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0100] Based on the same inventive concept, the embodiment of the present application also provides a task allocation device for implementing the task allocation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more task allocation device embodiments provided below can refer to the limitations on the task allocation method above, and will not be repeated here.

[0101] In an exemplary embodiment, as shown in FIG6 , a task allocation device 600 is provided, which is used for a cloud center server in a power network system. The power network system includes power equipment, edge nodes, and a cloud center server. The task allocation device includes:

[0102] An acquisition module 601 is used to acquire the operating status of each power device in the power network system;

[0103] The allocation module 602 is used to allocate tasks based on the digital twin model and the preset deep reinforcement learning model if the operating status of the power equipment is abnormal. The task allocation is used to migrate the tasks in the power equipment to the edge node or other power equipment for processing.

[0104] In an exemplary embodiment, the allocation module 602 is specifically used to determine the cost minimization problem of the power network system based on the digital twin model; taking the minimization of the cost of the power network system as the optimization goal, determining the task allocation strategy based on a preset deep reinforcement learning model, so as to migrate the tasks in the power network system to edge nodes or other power equipment for processing according to the task allocation strategy.

[0105] In an exemplary embodiment, the preset reinforcement learning model is implemented based on the QMIX reinforcement learning algorithm. The preset reinforcement learning model includes an agent network, a hybrid network and a super network. The preset reinforcement learning model decomposes the overall action-state value function. Each agent network is used to determine the local action-state value function of each power device. The hybrid network is used to generate an overall action-state value function in a monotonic form from the local action-state value function of each power device. The super network is used to generate weights and biases in the hybrid network based on the global state information of the power network system.

[0106] In an exemplary embodiment, the operating status of each power device is determined by a preset abnormality recognition model in each power device according to the operating parameters of each power device. The task allocation device 600 also includes a training module, which is used to receive model parameters of the abnormality recognition model in each power device; aggregate and update each model parameter, and send the updated model parameters to the power device, so that each power device trains the abnormality recognition model based on the updated model parameters.

[0107] In an exemplary embodiment, the training module is also used to obtain the recognition accuracy of the abnormal recognition model in each power device; determine the incentive mechanism corresponding to each power device based on the recognition accuracy, and the incentive mechanism is used to represent the weight of the model parameters corresponding to each power device when the model parameters are aggregated and updated.

[0108] Each module in the above task allocation device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0109] In an exemplary embodiment, a computer device is provided, which may be a cloud center server, and its internal structure diagram may be shown in FIG7. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a task allocation method is implemented.

[0110] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0111] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps described in any of the above method embodiments when executing the computer program.

[0112] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps described in any of the above method embodiments are implemented.

[0113] In one embodiment, a computer program product is provided, including a computer program, which implements the steps described in any of the above method embodiments when executed by a processor.

[0114] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0115] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0116] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A task allocation method, characterized in that: A cloud center server in a power network system, the power network system comprising power equipment, edge nodes and a cloud center server, the method comprising: Acquiring the operating status of each of the electric power devices in the electric power network system; If the operating state of the power equipment is abnormal, task allocation is performed based on the digital twin model and the preset deep reinforcement learning model, and the task allocation is used to migrate the tasks in the power equipment to the edge node or other power equipment for processing.

2. The method according to claim 1, characterized in that The task allocation based on the digital twin model and the preset reinforcement learning model includes: Determining the power network system cost minimization problem based on the digital twin model; Taking minimizing the cost of the power network system as the optimization goal, a task allocation strategy is determined based on the preset deep reinforcement learning model, so as to migrate the tasks in the power network system to the edge nodes or other power devices for processing according to the task allocation strategy.

3. The method according to claim 2, characterized in that The preset reinforcement learning model is implemented based on the QMIX reinforcement learning algorithm. The preset reinforcement learning model includes an agent network, a hybrid network and a super network. The preset reinforcement learning model decomposes the overall action-state value function. Each of the agent networks is used to determine the local action-state value function of each of the power devices. The hybrid network is used to generate an overall action-state value function in a monotonic form from the local action-state value function of each of the power devices. The super network is used to generate weights and biases in the hybrid network according to the global state information of the power network system.

4. The method according to claim 1, characterized in that: The operating state of each of the electric devices is determined by a preset abnormality recognition model in each of the electric devices according to the operating parameters of each of the electric devices; The training process of the preset anomaly recognition model includes: Receiving model parameters of the abnormality identification model in each of the electric power devices; The model parameters are aggregated and updated, and the updated model parameters are sent to the power devices, so that each power device trains the abnormality recognition model based on the updated model parameters.

5. The method according to claim 4, characterized in that The method further comprises: Obtaining the recognition accuracy of the abnormality recognition model in each of the electric power devices; An incentive mechanism corresponding to each of the electric power devices is determined according to the recognition accuracy, and the incentive mechanism is used to represent the weight of the model parameters corresponding to each of the electric power devices when the model parameters are aggregated and updated.

6. A task allocation system, characterized in that: The task allocation system includes a cloud center server, power equipment and edge nodes; The cloud center server is used to obtain the operating status of each of the power devices; if the operating status of the power device is abnormal, task allocation is performed based on the digital twin model and the preset deep reinforcement learning model, and the task allocation is used to migrate the tasks in the power device to the edge node or other power devices for processing; The electric power equipment is used to receive tasks from the electric power equipment in abnormal operation state and process the tasks; The edge node is used to receive the tasks in the power equipment with abnormal operating status and process the tasks.

7. A task allocation device, characterized in that: A cloud center server used in a power network system, the power network system comprising power equipment, edge nodes and a cloud center server, the device comprising: An acquisition module, used for acquiring the operating status of each of the electric power devices in the electric power network system; An allocation module is used to allocate tasks based on a digital twin model and a preset deep reinforcement learning model if the operating state of the power equipment is abnormal. The task allocation is used to migrate the tasks in the power equipment to the edge node or other power equipment for processing.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images