Power robot hot-line work task planning method and system

Abstract

The invention belongs to the field of power robots, and provides a power robot hot-line work task planning method and system, which realizes real-time perception of a work environment and dynamic optimization of a task execution strategy by introducing a deep reinforcement learning algorithm and combining with multi-modal sensor data, and utilizes a deep deterministic strategy gradient algorithm to realize real-time task planning of a power robot. The task priority is dynamically adjusted based on a graph attention mechanism according to environment data and task states collected in real time, real-time environment information and the current task completion condition of the power robot, and then the task execution sequence and path planning are dynamically adjusted. And efficient and safe operation of the electric robot in a complex operation environment is realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of electric power robots, and particularly relates to an electric power robot live working task planning method and system. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] With the intelligent development of the power system, electric power robots are increasingly widely used in live working. However, the existing task planning methods mostly rely on preset rules or static paths, lack the adaptability to dynamic environments, and are difficult to cope with complex and variable working scenarios. In addition, the existing planning methods often ignore real-time feedback during task execution, resulting in difficulty in guaranteeing working efficiency and safety.

[0004] On the other hand, electric power robots often face challenges such as concurrent tasks, complex and variable working environments, dynamic adjustment of task priorities, etc. The existing planning methods are difficult to dynamically adjust task execution strategies according to environmental changes, resulting in low working efficiency; difficult to respond to emergencies in a timely manner, unable to guarantee the continuity and safety of the work; unable to achieve optimal task allocation and path planning in the case of concurrent tasks. SUMMARY

[0005] In order to solve the above problems, the present application proposes an electric power robot live working task planning method and system. By introducing a deep reinforcement learning algorithm and combining multi-modal sensor data, the present application realizes real-time perception of the working environment and dynamic optimization of the task execution strategy. Using the deep deterministic policy gradient (DDPG) algorithm, the present application dynamically adjusts the task execution order and path planning according to the real-time collected environmental data and task state, and realizes efficient and safe working of electric power robots in complex working environments.

[0006] According to some embodiments, the present application adopts the following technical solutions: An electric power robot live working task planning method, comprising the following steps: constructing a state space according to the working state of the electric power robot, environmental information and task priority; determining a reward function by comprehensively considering task completion, working efficiency and safety; obtaining real-time environmental information and working state of the electric power robot; based on a graph attention mechanism, dynamically adjusting the task priority according to the real-time environmental information and the current task completion of the electric power robot; According to the real-time environmental information, the working state of the power robot, and the adjusted task priority, the state space is updated, and the task execution order and path planning are dynamically adjusted based on the reward function using a deep deterministic policy gradient algorithm.

[0007] As an alternative embodiment, the process of constructing the state space according to the working state of the power robot, the environmental information, and the task priority includes: constructing the state space according to the working state of the power robot, the environmental information, and the task priority, and forming a state vector as:

[0008] wherein, is the current position of the robot, is the attitude angle, represents the current task load or urgency or task importance, is the environmental code.

[0009] As an alternative embodiment, the process of constructing the state space according to the working state of the power robot, the environmental information, and the task priority includes defining a set of actions that the robot can perform to form an action space .

[0010] As an alternative embodiment, the process of determining the reward function considering the task completion, the working efficiency, and the safety includes: the reward function is:

[0011] wherein , , is a weight factor, respectively representing the weighting factor for the task completion , the safety level , and the task timeliness .

[0012] As an alternative embodiment, the process of dynamically adjusting the task priority based on the real-time environmental information and the current task completion of the power robot based on the graph attention mechanism includes: constructing a task graph, each node in the task graph representing an independent task, and the edges representing the association between tasks; using a graph attention network to perform information propagation and weight evaluation on the task graph; determining the priority of each task according to the estimated weight.

[0013] As a further defined embodiment, the process of constructing the task graph, in which each node represents an independent task and the edges represent the association between tasks, comprises: constructing a directed task graph G=(V, E), where each node represents an independent task, and the edges represent the dependency or physical association between tasks, and each node has a feature vector , where is the task location, is the estimated duration required by the task, is the remaining time tolerance, is the security level code.

[0014] As a further defined embodiment, the process of information propagation and weight evaluation of the task graph using the graph attention network comprises: Calculating the scheduling priority weight of each task node i in the current state using the attention mechanism, specifically:

[0015]

[0016]

[0017] where, represents the attention coefficient of node i to node j, i.e. the weight given by task i when evaluating the priority of task j; LeakyReLU represents an activation function for introducing nonlinearity; a is a trainable weight vector; W is a trainable weight matrix for linear transformation of the input feature vector; and are the feature vectors of tasks i and j, respectively; = i.e. the normalized attention weight, reflecting the relative importance of task j relative to the priority of task i; the final output represents the current priority value of task i , which is fused with the current robot operation state vector and input into the strategy network.

[0018] As an optional embodiment, the process of dynamically adjusting the task priority further comprises: when the robot completes the current task or detects a high-priority task update during task execution, performing task switching according to the scheduling priority weight calculation result, and introducing a task switching penalty function to prevent energy consumption and efficiency loss caused by frequent switching;

[0019] where, a penalty value representing task switching; is a weight factor for controlling the intensity of the penalty; This norm is used to quantify the distance between the two task execution actions, i.e. the last action and the current action, the greater the distance, the greater the penalty.

[0020] As an optional implementation, using a deep deterministic policy gradient algorithm, the process of dynamically adjusting the task execution order and path planning includes: The deep deterministic policy gradient algorithm includes a policy function and an action-value function, wherein the policy network, whose input is a state vector, outputs a continuous action a; the action-value network is used to evaluate the expected return of the current state-action pair; in actual work, the robot performs the corresponding operation according to the action output by the policy network and the action output by the policy network , and updates the policy according to the environmental information feedback to realize dynamic optimization.

[0021] A power robot live working task planning system, comprising: A state space construction module configured to construct a state space according to the working state of the power robot, environmental information and task priority; A reward function design module configured to determine a reward function by comprehensively considering task completion, working efficiency and safety; A data acquisition module configured to acquire real-time environmental information and working state of the power robot; A task priority evaluation module configured to dynamically adjust the task priority based on a graph attention mechanism according to real-time environmental information and current task completion of the power robot; A dynamic planning module configured to update the state space according to real-time environmental information, working state of the power robot and adjusted task priority, and dynamically adjust the task execution order and path planning based on the reward function and using a deep deterministic policy gradient algorithm.

[0022] A power robot applying the above method or provided with the above system.

[0023] A computer readable storage medium for storing computer instructions, which, when executed by a processor, completes the steps in the above method.

[0024] Compared with the prior art, the beneficial effects of the present application are: The application innovatively provides a power robot hot-line work task planning method, a Markov decision model is constructed, and a DDPG algorithm is introduced, task strategy learning in a continuous action space is realized, autonomy and adaptability of the power robot in a complex environment are improved, and a task graph modeling method based on a graph attention mechanism is introduced in task scheduling, dynamic modeling and weight calculation of a dependency relationship between multiple tasks are realized, and real-time adjustment and intelligent switching of a task priority are realized.

[0025] The application innovatively provides a power robot hot-line work task planning method, a Markov decision model is constructed, and a DDPG algorithm is introduced, task strategy learning in a continuous action space is realized, autonomy and adaptability of the power robot in a complex environment are improved, and a task graph modeling method based on a graph attention mechanism is introduced in task scheduling, dynamic modeling and weight calculation of a dependency relationship between multiple tasks are realized, and real-time adjustment and intelligent switching of a task priority are realized.

[0026] The application innovatively provides a power robot hot-line work task planning method, a Markov decision model is constructed, and a DDPG algorithm is introduced, task strategy learning in a continuous action space is realized, autonomy and adaptability of the power robot in a complex environment are improved, and a task graph modeling method based on a graph attention mechanism is introduced in task scheduling, dynamic modeling and weight calculation of a dependency relationship between multiple tasks are realized, and real-time adjustment and intelligent switching of a task priority are realized.

[0027] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0028] The drawings accompanying the specification of the present application form a part thereof, serve to provide further understanding of the present application, and together with the exemplary embodiments of the present application and their description serve to explain the present application, and do not constitute an improper limitation of the present application.

[0029] Figure 1 It is a method flowchart of an embodiment. DETAILED DESCRIPTION

[0030] The present application will be further described below in combination with the drawings and embodiments.

[0031] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0032] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0033] Where there is no conflict, the embodiments and features described in this application may be combined with each other.

[0034] Example 1 A method for planning live-line work tasks for electric robots, such as Figure 1 As shown, it includes the following steps: A state space is constructed based on the operating status, environmental information, and task priority of the electric robot. The reward function is determined by comprehensively considering task completion, operational efficiency, and safety. To obtain real-time environmental information and the operating status of the electric robot; Based on real-time environmental information and the current task completion status of the electric robot, the task priority is dynamically adjusted using a graph attention mechanism. Based on real-time environmental information, the operating status of the electric robot, and the adjusted task priorities, the state space is updated. Based on the reward function, the deep deterministic policy gradient algorithm is used to dynamically adjust the task execution order and path planning.

[0035] In this embodiment, the process of constructing the state space based on the electric robot's operating state, environmental information, and task priority includes: constructing the state space based on the electric robot's operating state, environmental information, and task priority, forming a state vector. for:

[0036] in, This is the robot's current position. For attitude angle, This indicates the current task load, urgency, or importance. Encode for the environment.

[0037] In this embodiment, the process of constructing the state space based on the electric robot's operating status, environmental information, and task priority also includes defining a set of executable actions for the robot, thus forming the action space. .

[0038] In this embodiment, a reward function is designed based on indicators such as task success rate (task_success), energy consumption (energy_cost), and safety (risk_factor). To guide the robot in learning the optimal policy, its expression is:

[0039] in , , , where are weighting factors, and represent the preference coefficients for success rate, energy consumption, and risk indicators, respectively.

[0040] Then, the DDPG algorithm is employed, and through the collaborative training of the policy network and value network, the mapping relationship from state to action is learned to achieve optimal task planning. DDPG is a reinforcement learning algorithm that handles continuous action spaces, and this invention is based on DDPG for task policy training. Its structure includes: Actor Network: Policy Function Input state s, output continuous action a.

[0041] Critic Network: Action-Value Function Assess the expected return of the current state-action pair.

[0042] Store the robot's state-action-reward-next state quadruple for each execution to break temporal dependencies. Use a target network. To improve training stability, the update rule is as follows:

[0043] Finally, in actual operation, the robot adjusts its operation based on the current state. Actions output by the policy network It executes the corresponding operations and updates the strategy based on environmental feedback to achieve dynamic optimization.

[0044] This invention, based on the dynamic changes in task priority and safety level stratification of power operations, adds a multi-task reward fusion mechanism to DDPG for concurrent multi-task scenarios. Specifically, task importance, task timeliness, and safety level are introduced into the reward function as weighted factors to form an adaptive reward calculation function, enabling the model to dynamically adjust in the face of multi-objective trade-offs.

[0045] This function is specifically designed for concurrent multitasking in power operations and introduces a multitasking reward fusion mechanism.

[0046] In actual live-line work on power distribution networks, power robots face situations where multiple tasks coexist (such as inspection, fastening, resetting, and decontamination). These tasks differ in priority, timeliness, and safety level. Current task scheduling mechanisms are mostly based on fixed rules or simple priority queues, making it difficult to implement dynamic, context-sensitive task switching and scheduling strategies. Therefore, this invention introduces a decision-making method based on Graph Attention Network (GAT) from Graph Neural Networks (GNNs) to construct a task relationship graph. Combining environmental conditions and the robot's current execution status, it dynamically judges task priorities and switches scheduling tasks, achieving a truly intelligent task orchestration mechanism in power operation scenarios.

[0047] Specifically, it includes: Construct a task graph, where each node represents an independent task and edges represent the relationships between tasks; Use graph attention networks to perform information propagation and weight evaluation on the task graph; The priority of each task is determined based on the estimated weights.

[0048] As a further defined implementation, the process of constructing a task graph, where each node in the task graph represents an independent task and edges represent the relationships between tasks, includes: constructing a directed task graph G = (V, E), where each node... This represents an independent task (such as inspecting nodes, capturing target points, repositioning insulating rods, etc.), with edges... This represents the dependencies or physical relationships between tasks, with each node possessing a feature vector. ,in For the task location, Estimate the time required for the task. For the remaining time grace period, This is a security level code.

[0049] The process of using graph attention networks to perform information propagation and weight evaluation on task graphs includes: The scheduling priority weight of each task node i in its current state is calculated using an attention mechanism, specifically as follows:

[0050]

[0051]

[0052] in, denoted as the attention coefficient of node i to node j, i.e., the weight given by task i when evaluating the priority of task j; LeakyReLU represents an activation function used to introduce non-linearity; a is a trainable weight vector; W is a trainable weight matrix used to perform a linear transformation on the input feature vector. and These are the feature vectors of tasks i and j, respectively; = This refers to the normalized attention weights, which reflect the relative importance of task j compared to task i; the final output... Indicates task i The current priority value is obtained, and this priority value is fused with the current robot operation state vector and input into the policy network.

[0053] When the robot completes its current task or detects a higher-priority task update during task execution, task switching is performed based on the scheduling priority weight calculation result to avoid "deadlock" or "priority inversion" problems. A task switching penalty function is also introduced. Used to prevent energy consumption and efficiency loss caused by frequent switching;

[0054] in, This represents the penalty value for switching tasks; It is a weighting factor used to control the intensity of the penalty; This norm is used to quantify the "distance" between the execution of two tasks (the previous action and the current action); the greater the distance, the greater the penalty.

[0055] As an alternative implementation method, the process of dynamically adjusting the task execution order and path planning using the deep deterministic policy gradient algorithm includes: The deep deterministic policy gradient algorithm includes a policy function and an action-value function. The policy network takes a state vector as input and outputs a continuous action 'a'. The action-value network evaluates the expected reward of the current state-action pair. In actual operation, the robot calculates the reward based on the current state vector. Actions output by the policy network It executes the corresponding operations and updates the strategy based on environmental information feedback to achieve dynamic optimization.

[0056] Example 2 A power robot live-line operation task planning system includes: The state space construction module is configured to construct the state space based on the working state of the electric robot, environmental information, and task priority. The reward function design module is configured to comprehensively consider task completion, work efficiency, and safety to determine the reward function. The data acquisition module is configured to acquire real-time environmental information and the operating status of the electric robot; The task priority evaluation module is configured to dynamically adjust task priorities based on graph attention mechanism, according to real-time environmental information and the current task completion status of the electric robot. The dynamic programming module is configured to update the state space based on real-time environmental information, the working status of the electric robot, and the adjusted task priority, and dynamically adjust the task execution order and path planning based on the reward function and using a deep deterministic policy gradient algorithm.

[0057] Example 3 An electric robot that applies the above method or is equipped with the above system.

[0058] Example 4 A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the steps in the above method.

[0059] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of one or more computer-usable storage media (including, but not limited to, disk storage, etc.) containing computer-usable program code. CD - ROM It takes the form of a computer program product implemented on (such as optical memory, etc.).

[0060] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0061] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0062] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made by those skilled in the art without creative effort within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A power robot live working task planning method, characterized in that, The method comprises the following steps: According to the working state of the electric power robot, environmental information and task priority, a state space is constructed; Considering the task completion, work efficiency and safety, a reward function is determined; Real-time environmental information and the working state of the electric power robot are obtained; According to the real-time environmental information and the current task completion of the electric power robot, the task priority is dynamically adjusted based on the graph attention mechanism; According to the real-time environmental information, the working state of the electric power robot and the adjusted task priority, the state space is updated, and the task execution order and path planning are dynamically adjusted based on the reward function and using the deep deterministic policy gradient algorithm.

2. The power robot live working task planning method of claim 1, wherein, According to the working state of the electric power robot, the environmental information and the task priority, the process of constructing the state space comprises: constructing the state space according to the working state of the electric power robot, the environmental information and the task priority, forming a state vector is: ; wherein, is the current position of the robot, is the angle of the pose, denotes the current task load or urgency or task importance, is the environment encoding; According to the working state of the electric robot, environmental information and task priority, the process of constructing the state space further includes defining a set of actions executable by the robot, constituting an action space .

3. The power robot live working task planning method of claim 1, wherein, The process of determining the reward function considering the task completion, work efficiency and safety includes: determining the reward function f: wherein , , are weight factors, respectively representing the weighting factors for the task completion , security level and task timeliness .

4. The power robot live working task planning method of claim 1, wherein, The process of dynamically adjusting the task priority based on the real-time environmental information and the current task completion of the electric power robot includes: A task graph is constructed, each node in the task graph represents an independent task, and the edge represents the association between tasks; The graph attention network is used to perform information propagation and weight evaluation on the task graph; According to the estimated weight, the priority of each task is determined.

5. The power robot live working task planning method of claim 4, wherein, The process of constructing the task graph includes: constructing a directed task graph G=(V, E), wherein each node represents an independent task, and an edge represents a dependency relationship or a physical association between tasks, and each node has a feature vector , wherein is a task position, is a required duration estimation of the task, is a remaining time tolerance, and is a security level code.

6. The power robot live working task planning method of claim 4, wherein, The process of using the graph attention network to perform information propagation and weight evaluation on the task graph includes: The scheduling priority weight of each task node i in the current state is calculated using the attention mechanism, specifically: wherein, represents the attention coefficient of node i to node j, i.e., the weight given by task i when evaluating the priority of task j; LeakyReLU represents an activation function for introducing nonlinearity; a is a trainable weight vector; W is a trainable weight matrix for linear transformation of the input feature vector; and are the feature vectors of tasks i and j, respectively; is the normalized attention weight, reflecting the relative importance of task j relative to task i priority; the final output represents the current priority value of task i , which is fused with the current robot job state vector and input into the policy network.​ 7. The power robot live working task planning method of claim 1, wherein, The process of dynamically adjusting the task priority further includes: when the robot completes the current task or detects a high-priority task update in task execution, switching tasks according to the scheduling priority weight calculation result, and introducing a task switching penalty function for preventing frequent switching from causing energy consumption and efficiency loss; wherein, represents a penalty value for task switching; is a weight factor to control the strength of the penalty; This norm is used to quantify the distance between two task execution actions, the greater the distance, the greater the penalty.

8. The power robot live working task planning method of claim 1, wherein, The process of dynamically adjusting the task execution order and path planning using the deep deterministic policy gradient algorithm includes: The deep deterministic policy gradient algorithm comprises a policy function and an action-value function, wherein the policy network, inputting a state vector, outputs a continuous action a; the action-value network is used for evaluating the expected return of a current state-action pair; in actual operation, the robot executes a corresponding operation according to the action output by the policy network and the state vector and the action output by the policy network , and updates the policy according to the environment information feedback to realize dynamic optimization.

9. A power robot live working task planning system characterized by, It includes: The state space construction module is configured to construct a state space according to the working state of the electric power robot, environmental information and task priority; The reward function design module is configured to determine a reward function by considering the task completion, work efficiency and safety; The data acquisition module is configured to obtain real-time environmental information and the working state of the electric power robot; The task priority evaluation module is configured to dynamically adjust the task priority based on the graph attention mechanism according to the real-time environmental information and the current task completion of the electric power robot; The dynamic planning module is configured to update the state space according to the real-time environmental information, the working state of the electric power robot and the adjusted task priority, and dynamically adjust the task execution order and path planning based on the reward function and using the deep deterministic policy gradient algorithm.

10. An electric robot, characterized by The method of any one of claims 1-8 or the system of claim 9 is applied.

11. A computer readable storage medium characterized by, A computer program product for storing computer instructions, which are executed by a processor to complete the steps in the method of any one of claims 1-8.

Images