Trajectory planning method and trajectory planning device for collision avoidance of isopotential working machine arm

Abstract

The application provides a trajectory planning method and device for live-line working mechanical arm collision avoidance, the method comprising: determining a state space, an action space and a reward function of the mechanical arm, the reward function being a function considering a safe working space of the mechanical arm under live-line work and a demand for the mechanical arm to avoid a dangerous space on the basis of ensuring that a terminal end of the mechanical arm moves to a target point position; iteratively training, based on the state space, the action space and the reward function, a deep deterministic policy gradient algorithm to obtain a target policy model; obtaining the target point position of the mechanical arm, inputting the target point position into the target policy model to obtain an optimal moving path of the mechanical arm; and controlling the mechanical arm to move to the target point position according to the optimal moving path. The application solves the problem that a conventional method is difficult to control the mechanical arm to avoid a dangerous space.

Description

Technical Field

[0001] This invention relates to the field of robotic arm trajectory planning technology, and more specifically, to a trajectory planning method, trajectory planning device, computer-readable storage medium, and computer program product for collision avoidance of an equipotential robotic arm. Background Technology

[0002] The safe and stable operation of the power system is crucial to the national economy and people's livelihood. The increasing autonomy and intelligence of power robots have played a vital role in ensuring the safe and stable operation of power equipment. In recent years, breakthroughs have been achieved in several key technologies, and a series of prototypes have been developed. These have been piloted in power generation, transmission, substation, distribution, and consumption sectors. Substation inspection robots and transmission line inspection drones have been widely deployed. The increasing autonomy and intelligence of power robots, particularly equipotential bonding robots, have played a significant role in ensuring the safe and stable operation of power equipment.

[0003] As a widely used automated mechanical device, the robotic arm has assisted humans in performing many dangerous tasks. However, in most cases, the robotic arm is operated by humans, that is, the movement trajectory of the robotic arm is planned by humans, which plays a teaching and supervision role [1]. This limits the distance between humans and robotic arms to a certain range. Therefore, traditional robotic arms either rely on humans and cooperate with them, or their trajectory planning follows a fixed and structured process. Therefore, in some dangerous, complex and changeable environments, the trajectory planning of traditional robotic arms is easily affected by the changing environment, and its trajectory may deviate, which may lead to serious consequences. Therefore, it is urgent to carry out research on autonomous robotic arms.

[0004] In recent years, scholars both domestically and internationally have conducted extensive research in the field of obstacle avoidance motion planning for robotic arms. Existing technologies include using a superior learning generalization mechanism for obstacle avoidance motion planning, improving the intelligence level and autonomous operation capability of the robotic arm through a teaching learning system. However, this method is overly reliant on the quality and method of collecting samples. Another approach uses an artificial potential field method for obstacle avoidance path planning, adding virtual obstacles near the nearest point to solve local extremum problems. However, this method lacks generalization and struggles to adapt to complex environments. A further approach employs an improved fast expanding random tree algorithm for motion planning of dual robotic arms, considering not only static obstacles but also the motion pose of the master robotic arm at each moment as dynamic obstacles, using this as a basis for planning feasible paths for the slave robotic arm. However, this method suffers from high computational cost and slow convergence speed. A further approach combines sampling-based path planning methods with flexible obstacle avoidance methods for motion planning, modifying the posture of obstacles during the planning process and ultimately finding a successful obstacle avoidance path. However, this method involves contact between the robotic arm and obstacles during the planning process, posing a safety hazard. The aforementioned algorithm relies on real-time modeling of the robotic arm and its environment, and its computational complexity increases exponentially with the increase of the robotic arm's degrees of freedom. Therefore, in some dangerous, complex, and variable environments, the trajectory planning of traditional robotic arms is easily affected by the changing environment, and its trajectory may deviate, leading to serious consequences. Summary of the Invention

[0005] The main objective of this application is to provide a trajectory planning method, trajectory planning device, computer-readable storage medium, and computer program product for collision avoidance of an equipotential working robot, so as to at least solve the problem in the prior art that it is difficult to control a robot to avoid dangerous spaces.

[0006] To achieve the above objectives, according to one aspect of this application, a trajectory planning method for collision avoidance of an equipotential working robotic arm is provided, comprising: determining the state space, motion space, and reward function of the robotic arm, wherein the state space is a set of states of the environment and the robotic arm itself in the working space where the robotic arm is located, the motion space is a set of motions of the robotic arm moving in the space, and the reward function is a function that considers the safe working space of the robotic arm under energized conditions and the need for the robotic arm to avoid dangerous spaces, while ensuring that the end effector of the robotic arm moves to a target point; iteratively training the model using a deep deterministic policy gradient algorithm based on the state space, the motion space, and the reward function to obtain a target policy model; obtaining the target point position of the robotic arm and inputting the target point position into the target policy model to obtain the optimal movement path of the robotic arm; and controlling the robotic arm to move to the target point position according to the optimal movement path.

[0007] Optionally, determining the state space, motion space, and reward function of the robotic arm includes: determining the robotic arm's own state and the robotic arm's environmental state, and determining the state space based on the robotic arm's own state and the robotic arm's environmental state, wherein the state space includes at least the robotic arm's own state and the environmental state, the robotic arm's own state includes at least the joint angles of the robotic arm and the spatial position of the robotic arm's end effector, and the environmental state includes at least the spatial position of the target point and the relative distance between the robotic arm's end effector and the target point; determining the motion space based on the robotic arm's joint angles; calculating the distance between the robotic arm's end effector and the target point to obtain the relative distance; performing simulation calculations on the spatial electric field of the working environment of the robotic arm to obtain the safe working space and the hazardous space; and determining the reward function based on the position of the robotic arm's end effector combined with the relative distance, the safe working space, and the hazardous space.

[0008] Optionally, the spatial electric field of the working environment of the robotic arm is simulated to obtain the safe working space and the hazardous space, including: using the finite element method to simulate and model the electric field distribution of the robotic arm within a set range to obtain an electric field model, wherein the set range is the set range of the hardware and wires before the robotic arm enters the live working position; using the method of images to calculate the electric field of the image charges in the electric field model to obtain the conductor electric field of the robotic arm, and calculating the spatial electric field generated by the image charges; adding the conductor electric field and the spatial electric field to obtain the substation spatial electric field; based on the substation space The electric field calculation measures the maximum electric field strength on the surface of the flexible conductor at different gap distances between the robotic arm and the adjacent flexible conductor. The distance between the joint center point of the robotic arm and the key point of the insulation safety boundary is calculated using the key point method, and the smallest distance is taken as the minimum distance. The area satisfying the first and second conditions is defined as the safe working space. The first condition is that the maximum electric field strength is less than or equal to the set maximum working electric field strength, and the second condition is that the minimum distance is greater than the set required distance. The area formed by an irregularly swept body with the flexible conductor as its axis and a specific length as its radius is defined as the hazardous space.

[0009] Optionally, the electric field of the image charge in the electric field model is calculated using the method of images to obtain the conductor electric field of the robotic arm, and the spatial electric field generated by the image charge is calculated, including: calculating the conductor electric field of the robotic arm according to a first formula, wherein the first formula is... V represents the potential of the conductor in the electric field, E d The electric field of the conductor is represented; the spatial electric field generated by the mirror charge is calculated according to the second formula, which is: Eq Let q represent the electric field in space, k be the Coulomb constant, and q be the electric field in space. i r is the magnitude of the i-th mirror charge in the electric field of space. i It is the distance from the i-th mirror charge to the conductor. It is a unit vector pointing towards the conductor.

[0010] Optionally, the reward function is determined based on the position of the robotic arm's end effector, the relative distance, the safe working space, and the danger zone. This includes: determining a first function, where the first function is a piecewise constant function where the penalty value is a first preset value when the robotic arm enters the danger zone or exceeds the safe working space, and the penalty value is 0 when the robotic arm does not enter the danger zone or exceeds the safe working space, with the first preset value being negative; determining a second function, where the second function is a piecewise constant function where the reward value is a second preset value when the robotic arm reaches the target point's position, and the reward value is 0 when the robotic arm does not reach the target point's position, with the second preset value being positive; and adding the relative distance, the first function, and the second function to obtain the reward function.

[0011] Optionally, a target policy model is obtained by iteratively training based on the state space, the action space, and the reward function using a deep deterministic policy gradient algorithm, including: initializing an evaluation network, an action network, a target evaluation network, and a target action network; a generation step, inputting the current sample state into the action network so that the action network generates the optimal action based on the current sample state, wherein the current sample state space is any sample state in the training sample set; an execution step, wherein the evaluation network executes the optimal action and evaluates it based on the reward function to obtain an evaluation score; an update step, updating the parameters in the evaluation network, the action network, the target evaluation network, and the target action network using gradient descent; a repetition step, repeating the generation step, the execution step, and the update step at least once in sequence until the end effector of the robotic arm reaches a set position, and using the latest evaluation score as the cumulative reward value of the current iteration; repeating the generation step, the execution step, the update step, and the repetition step at least once in sequence until the number of iterations reaches a set maximum number of iterations or the cumulative reward value reaches a maximum value, thereby obtaining the target policy model.

[0012] Optionally, the gradient descent method is used to update the parameters in the evaluation network, the action network, the target evaluation network, and the target action network, including: minimizing a third formula to update the evaluation network parameters in the evaluation network, wherein the third formula is... N represents the total number of samples selected for learning from the training sample set, R i Q(S) represents the reward score calculated by the reward function, where γ represents the discount parameter. i A i |θ Q ) indicates that in the current sample state S i Execute the optimal action A i The evaluation score obtained subsequently, Q′((S) i+1 |θ u′ )|θ Q′ ) indicates that in the current sample state S i The target score after performing the action, θ Q For the evaluation network parameters, θ Q′ To evaluate the network parameters, θ u′ S represents the target action network parameters. i For the current sample state, A i The optimal action is determined; the action network parameters in the action network are updated according to the fourth formula, which is: θ u Represents the action network parameters, u(S|θ) u ) represents the action network, Q(S, A|θ) Q ) represents the evaluation network; the target evaluation network parameters in the target evaluation network are updated according to the fifth formula, where the fifth formula is θ. Q′ ←τθ Q +(1-τ)θ Q′ τ represents the parameters for soft updates; the target action network parameters in the target action network are updated according to the sixth formula, where the sixth formula is θ. u′ ←τθ u +(1-τ)θ u′ .

[0013] According to another aspect of this application, a trajectory planning device for collision avoidance of an equipotential working robotic arm is provided. The device includes: a determining unit, used to determine the state space, motion space, and reward function of the robotic arm; the state space is a set of states of the environment and the robotic arm itself in the working space where the robotic arm is located; the motion space is a set of motions of the robotic arm moving in the space; and the reward function is a function that considers the safe working space of the robotic arm under energized conditions and the need for the robotic arm to avoid dangerous spaces, while ensuring that the end effector of the robotic arm moves to the target point position; a training unit, used to iteratively train the target strategy model using a deep deterministic policy gradient algorithm based on the state space, the motion space, and the reward function; an acquisition unit, used to acquire the target point position of the robotic arm and input the target point position into the target strategy model to obtain the optimal movement path of the robotic arm; and a control unit, used to control the robotic arm to move to the target point position according to the optimal movement path.

[0014] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform any of the methods described.

[0015] According to another aspect of this application, a computer program product is provided, including computer instructions that, when executed by a processor, implement any of the methods described.

[0016] Applying the technical solution of this application, in the trajectory planning method for collision avoidance of a robotic arm in equipotential work, firstly, the state space, motion space, and reward function of the robotic arm are determined. The state space is the set of states of the environment and the robotic arm itself within the work space where the robotic arm is located. The motion space is the set of actions the robotic arm performs in the space. The reward function is a function that considers the safe working space of the robotic arm under live-line work and the need for the robotic arm to avoid dangerous spaces, while ensuring that the end effector of the robotic arm moves to the target point. Then, based on the state space, motion space, and reward function, a deep deterministic policy gradient algorithm is used for iterative training to obtain a target policy model. Next, the target point position of the robotic arm is obtained and input into the target policy model to obtain the optimal movement path of the robotic arm. Finally, the robotic arm is controlled to move to the target point position according to the optimal movement path. This application designs a work space for the robotic arm, which includes a safe working space and a threat space. When the robotic arm works in the safe working space, it can ensure the safety of the equipment in the substation and itself, while effectively completing the task. By training the robotic arm using reinforcement learning and employing deep reinforcement learning algorithms for path planning, this application fully considers the safe working environment of live-line operations and the obstacle avoidance requirements of the robotic arm, ensuring that the robotic arm can reach the target point in an orderly manner without collisions. This application solves the problem that traditional methods of controlling robotic arms struggle to avoid hazardous spaces. Attached Figure Description

[0017] Figure 1 A hardware structure block diagram of a mobile terminal for performing a trajectory planning method for collision avoidance of an equipotential working robot arm, according to an embodiment of this application, is shown.

[0018] Figure 2 A schematic flowchart of a trajectory planning method for collision avoidance of an equipotential working robot arm according to an embodiment of this application is shown.

[0019] Figure 3 A schematic diagram of simulation results for an equipotential working robot provided according to an embodiment of this application is shown;

[0020] Figure 4 A schematic diagram of an equipotential working robot arm structure provided according to an embodiment of this application is shown;

[0021] Figure 5 A schematic diagram of a safe working space for a robotic arm in a 220kV electric field, provided according to an embodiment of this application, is shown.

[0022] Figure 6 A structural diagram of a depth-deterministic strategy gradient algorithm provided according to an embodiment of this application is shown;

[0023] Figure 7 A structural block diagram of a trajectory planning device for collision avoidance of an equipotential working robot arm according to an embodiment of this application is shown.

[0024] The above figures include the following reference numerals:

[0025] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0026] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0030] Equipotential work is a safety measure that involves adjusting the body potential of a worker to the same level as the potential of surrounding voltage conductors. This is done to reduce the risk of electric shock because, in an equipotential state, there is no potential difference between the human body and the charged conductor, and therefore no current flows through the body.

[0031] Equipotential working robotic arm: This type of robotic arm replaces the human body and adjusts the potential of the robotic arm to the same level as the potential of the surrounding conductors.

[0032] As described in the background section, existing robotic arm trajectory planning follows a fixed and structured process. Therefore, in some dangerous, complex, and variable environments, the trajectory planning of traditional robotic arms is easily affected by the changing environment, and its trajectory may deviate, leading to serious consequences. To solve the problem that traditional methods of controlling robotic arms are difficult to avoid dangerous spaces, embodiments of this application provide a trajectory planning method, trajectory planning device, computer-readable storage medium, and computer program product for collision avoidance of equipotential working robotic arms.

[0033] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0034] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a trajectory planning method for collision avoidance of an equipotential robotic arm, according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0035] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the trajectory planning method for collision avoidance of the equipotential robotic arm in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0036] This embodiment provides a trajectory planning method for collision avoidance of an equipotential working robot arm running on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0037] Figure 2 This is a flowchart of a trajectory planning method for collision avoidance by an equipotential robotic arm according to an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0038] Step S201: Determine the state space, motion space, and reward function of the robotic arm. The state space is the set of states of the environment in the work space where the robotic arm is located and the state of the robotic arm itself. The motion space is the set of motions of the robotic arm moving in the space. The reward function is a function that takes into account the safe working space of the robotic arm under live operation and the requirement of the robotic arm to avoid dangerous spaces, while ensuring that the end effector of the robotic arm moves to the target point.

[0039] Specifically, deep reinforcement learning algorithms are used to implement path planning for the equipotential robotic arm. This involves training a path planning model using a deep deterministic policy gradient algorithm to generate the optimal path. Therefore, the parameters of the deep deterministic policy gradient algorithm, namely the state space, action space, and reward function, are first required. The action space defines all possible actions the robotic arm can perform in space. These actions can be: increments or absolute values ​​of joint angles; velocity or acceleration of the end effector; specific motion commands, such as forward, backward, left turn, right turn, etc. The reward function includes positive rewards: reduction in the distance between the robotic arm's end effector and the target point, completion of a specific task, or reaching a specific milestone; and negative rewards: approaching obstacles, entering unsafe working areas, or unstable states of the robotic arm, such as joint angles exceeding the normal operating range.

[0040] Step S202: Based on the above state space, action space and reward function, the target policy model is obtained by iterative training using the deep deterministic policy gradient algorithm.

[0041] Specifically, the Actor Network and Critic Network are initialized, as are the Target Action Network and Target Critic Network, and the ReplayBuffer is initialized. Initial experience is collected, such as randomly selecting actions, executing them, and collecting interaction data with the environment (state, action, reward, new state). This interaction data is stored in the ReplayBuffer. A batch of data is randomly selected from the ReplayBuffer, and this data is used to update the parameters of the Critic Network and Actor Network for training, resulting in the final target policy model. After training, the obtained target policy model can be used to guide the robotic arm to perform tasks in a real-world environment. Utilizing deep reinforcement learning enables the equipotential robotic arm to have automatic obstacle avoidance capabilities, ensuring it does not enter dangerous areas in high-voltage electrical environments, playing a crucial role in ensuring the safe and stable operation of power equipment.

[0042] Step S203: Obtain the target point position of the robotic arm, input the target point position into the target strategy model, and obtain the optimal movement path of the robotic arm.

[0043] Specifically, a vision system is used for coarse localization to determine the pose that the robotic arm's end effector needs to reach. Determining the robotic arm's end effector means determining the target point position that the robotic arm needs to reach, with the aim of ensuring that the robotic arm operates within a safe space and reaches the target point to perform subsequent tasks. After determining the target point position, the target point position is input into the aforementioned target strategy model to obtain the optimal movement path of the robotic arm.

[0044] Step S204: Control the robotic arm to move to the target point position according to the optimal movement path.

[0045] Specifically, after obtaining the optimal movement path of the robotic arm, it can be directly controlled to move to the target point according to the optimal movement path. Furthermore, to verify the effectiveness of this application, an equipotential robotic arm simulation experiment was conducted. For example... Figure 3 As shown, Figure 3 The system includes two points (point 1 and point 2), which are the target points that the robotic arm needs to reach. By using reinforcement learning to train the robotic arm, it can eventually reach the target points in an orderly manner without collisions.

[0046] In this embodiment, firstly, the state space, motion space, and reward function of the robotic arm are determined. The state space is the set of states of the environment in which the robotic arm is located and the state of the robotic arm itself. The motion space is the set of actions the robotic arm takes to move within the space. The reward function is a function that considers the safe working space of the robotic arm under live-line conditions and the need for the robotic arm to avoid dangerous spaces, while ensuring that the end effector of the robotic arm moves to the target point. Then, based on the state space, motion space, and reward function, a deep deterministic policy gradient algorithm is used for iterative training to obtain a target policy model. Afterward, the target point position of the robotic arm is obtained and input into the target policy model to obtain the optimal movement path of the robotic arm. Finally, the robotic arm is controlled to move to the target point position according to the optimal movement path. This application designs a working space for the robotic arm, which includes a safe working space and a threat space. When the robotic arm works in the safe working space, it can ensure the safety of the equipment in the substation and itself, while effectively completing the task. By training the robotic arm using reinforcement learning and employing deep reinforcement learning algorithms for path planning, this application fully considers the safe working environment of live-line operations and the obstacle avoidance requirements of the robotic arm, ensuring that the robotic arm can reach the target point in an orderly manner without collisions. This application solves the problem that traditional methods of controlling robotic arms struggle to avoid hazardous spaces.

[0047] To enable those skilled in the art to better understand the technical solution of this application, the implementation process of the trajectory planning method for collision avoidance of the equipotential working robot arm of this application will be described in detail below with reference to specific embodiments.

[0048] To achieve path planning for the equipotential robotic arm, ensuring both the safety of equipment within the substation and the arm's own operation while effectively completing the task, in one optional implementation, step S201 includes:

[0049] Step S2011: Determine the self-state of the robotic arm and the environmental state of the robotic arm, and determine the state space based on the self-state and the environmental state of the robotic arm. The state space includes at least the self-state of the robotic arm and the environmental state. The self-state includes at least the joint angle of the robotic arm and the spatial position of the end of the robotic arm. The environmental state includes at least the spatial position of the target point and the relative distance between the end of the robotic arm and the target point.

[0050] Step S2012: Determine the motion space based on the joint angles of the robotic arm.

[0051] Step S2013: Calculate the spatial distance between the end of the robotic arm and the target point to obtain the relative distance;

[0052] Step S2014: Simulate and calculate the spatial electric field of the working environment of the robotic arm to obtain the safe working space and the dangerous space.

[0053] Step S20 1 5: Determine the reward function based on the location of the robotic arm end effector, the relative distance, the safe working space, and the dangerous space.

[0054] In the above embodiments, the state space of the robotic arm is defined, such as... Figure 4As shown, this invention uses a six-degree-of-freedom robotic arm for operation. The state space of the robotic arm refers to the state of the environment and the state of the robotic arm itself. The state space of the robotic arm mainly includes the angles of each joint, the position of the target point, and the position of the robotic arm's end effector. These serve as input variables, allowing the robotic arm to acquire relevant information and perform actions based on this information to obtain reward points. The state of the robotic arm itself refers to the changes in its structure, including the magnitude of the joint angles, which can be represented as A = (θ1, θ2, θ3, θ4, θ5, θ6). Simultaneously, the spatial coordinates of the robotic arm's end effector are defined as (x, y, z). The environmental state refers to the state acquired when the robotic arm interacts with the environment, mainly including the spatial coordinates of the target point (a, b, c) and the relative distance r1 between the robotic arm's end effector and the target point. The robotic arm's motion space is defined. To plan the trajectory of the robotic arm's end effector from the initial point to the target point, various environmental state information is first acquired, including the spatial coordinates of the target point and the robotic arm's end effector. Then, through inverse kinematics calculations, the joint angles are obtained. These joint angles are continuously adjusted to shorten the distance between the end effector and the target point. Therefore, the motion space is defined as A = (θ1, θ2, θ3, θ4, θ5, θ6), and the joint angles of the robotic arm constitute the motion space. The reward function is defined: the key to autonomous trajectory planning of the robotic arm lies in the method of obtaining the reward value. The design of the reward function is mainly based on the relative coordinates between the end effector and the target point. The relative distance is calculated by substituting the spatial positions of the end effector and the target point into the seventh formula. The seventh formula is... r1 represents the aforementioned relative distance, (x, y, z) represents the spatial coordinates of the robotic arm's end effector, and (a, b, c) represents the spatial coordinates of the target point. Secondly, the equipotential robotic arm should operate within a safe working area and must not touch other objects. The safe working space for live-line working robots is limited not only by the confined space between equipment in the substation but also by the electrical safety distances between phases and between phases and ground. Therefore, a safe working space must be designed for the robot, ensuring the safety of the equipment within the substation and the robot itself while effectively completing its task. The safe working space and hazardous space are planned through electromagnetic simulation of the environment. The reward function is determined based on the aforementioned relative distance and the positional relationship between the robotic arm's end effector and the safe working space and hazardous space.

[0055] In order to improve the safety of substation equipment and itself by planning a safe working space through electromagnetic simulation of the robot's working environment, in an optional embodiment, the above step S2014 includes:

[0056] Step S20141: The electric field distribution of the above-mentioned robotic arm within the set range is simulated and modeled using the finite element analysis method to obtain the electric field model. The set range is the set range of the hardware and wires before the above-mentioned robotic arm enters the live working position.

[0057] Step S20142: The electric field of the image charge in the electric field model is calculated using the image method to obtain the conductor electric field of the robotic arm and the spatial electric field generated by the image charge.

[0058] Step S20143: Add the above conductor electric field and the above space electric field to calculate the substation space electric field;

[0059] Step S20144: Calculate the maximum electric field intensity on the surface of the flexible conductor at different gap distances between the robotic arm and the adjacent flexible conductor based on the electric field of the substation space.

[0060] Step S20145: The distance between the joint center point of the above-mentioned robotic arm and the key point of the insulation safety boundary is calculated using the key point method, and the smallest of the above-mentioned distances is taken as the minimum distance.

[0061] Step S20146: The area that meets the first condition and the second condition is determined as the above-mentioned safe working space. The first condition is that the maximum electric field strength is less than or equal to the set maximum working electric field strength, and the second condition is that the minimum distance is greater than the set required distance.

[0062] Step S20147: The area formed by the irregular sweeping body with the aforementioned soft wire as the axis and a specific length as the radius is determined as the aforementioned hazardous space.

[0063] In the above embodiment, the image charge method is used to establish a model and calculate the surrounding spatial electric field. Specifically, finite element analysis software is used to simulate and model the electric field distribution around the hardware and conductors before the robot enters the target point. The image charge method is characterized by its simplicity, practicality, and high accuracy. This method does not require solving the Laplace equation with complex boundary conditions; it only needs to use image charges to replace various charged bodies and grounded bodies to obtain the potential and electric field strength at any point in the electric field. When calculating the spatial electric field around the busbar connecting conductor, the conductor can be regarded as a charged line segment with uniformly distributed charge, and the image charge is located at the center of the conductor. When the influence of the ground is taken into account, the image method is used to calculate the electric field, which can calculate the conductor electric field and the spatial electric field generated by the image charge. Adding the conductor electric field and the spatial electric field generated by the image charge gives the spatial electric field between the equipment in the substation where the robot is located, i.e., the aforementioned substation spatial electric field. Based on the electric field solution results, the maximum working electric field strength E of the robotic arm is set. m Then the electric field strength E in the robotic arm's workspace r E should be satisfied r≤E m The robotic arm's working space includes a safe working space and a hazardous space. Based on the calculation of the maximum electric field strength on the surface of the flexible conductor under different gap distances between the robotic arm's elbow joint and the adjacent flexible conductor, the safe working space outside the hazardous space formed by an irregular sweeping body with the flexible conductor as its axis and a specific length as its radius can be derived. In simpler terms, this is a three-dimensional space extending along the length of the flexible conductor with the flexible conductor as its centerline. The boundary of this three-dimensional space is formed by extending the surface of the flexible conductor outward by a specific safe distance (i.e., a specific radius). Since the flexible conductor may not be perfectly straight and may have different bends and twists in space, the resulting spatial region is irregular. In safe operation of power systems, the area formed by this irregular sweeping body is defined as the hazardous space. A hazardous boundary plane (also known as a safe boundary; areas not exceeding this plane are considered safe areas, and areas exceeding it are considered hazardous areas) is constructed. This plane is perpendicular to the ground and located between the flexible conductor and the gripping robotic arm. Let the distance between this hazardous boundary plane and the adjacent phase flexible conductor be Dissafe. Figure 5 As shown, the hazardous boundary is represented by a dashed line. The standard for safe live-line work is that the robotic arm should not contact the hazardous boundary plane. This plane is a hypothetical construct and does not actually exist. In the analysis, the key point method is used to calculate the minimum distance between the robot and the insulated safety boundary. This involves selecting the position of the robotic arm's end effector and the center point (x) of each joint. i y i , z i ), and the key points of the security boundary (x l y l , z l The distances between all key points of the robotic arm and all key points of the insulation safety boundary are calculated using the distance formula. The smallest of all calculated distances is selected as the minimum distance d between the robotic arm and the insulation safety boundary. min The distance formula is: The required distance is set to d. m Then the minimum distance between the robot and the insulation safety boundary should satisfy d. min >d m .

[0064] In an optional implementation, for the electric field distribution of the robotic arm, step S20142 includes:

[0065] Step S201421: Calculate the electric field of the conductor in the robotic arm according to the first formula, whereby the first formula is... V represents the potential of the conductor in the aforementioned electric field, E d This represents the electric field of the aforementioned conductor;

[0066] Step S201422: Calculate the spatial electric field generated by the aforementioned mirror charge according to the second formula, which is: E q Let q represent the electric field in the space described above, where k is the Coulomb constant and q is the electric field in the space i r is the magnitude of the i-th mirror charge in the aforementioned spatial electric field. i It is the distance from the i-th mirrored charge to the conductor. It is a unit vector pointing towards the conductor.

[0067] In the above embodiments, the simulated charge method uses simulated charges to replace various charged bodies and grounded bodies, thereby obtaining the potential and electric field strength at any point in the field. When considering the influence of the ground, the image method is used to calculate the electric field E at the conductor based on the known potential V of the conductor (already obtained using the simulated charge method) and the first formula. d The spatial electric field generated by the mirror charge is calculated using the second formula.

[0068] The method of images is a classic electromagnetic analysis method used to solve for the electric field or potential distribution around a charged object. Based on the principle of images, it simplifies the calculation of electromagnetic problems on complex boundaries by mirroring charges or currents where boundary conditions change. In electromagnetism, the method of images is often used to solve electromagnetic field problems near the surface of a conductor or a charged object. Specifically, the basic principle of the method of images is: if a charge is located near a grounded conductor, it can be assumed that there is an equal but opposite charge (i.e., a mirror charge) on the other side of the conductor. This makes the electric field on the conductor surface zero (because the electric field inside the conductor is always zero). By calculating the electric field produced by this mirror charge, the electric field distribution in the original problem can be approximated. By calculating the electric field under the combined action of the original charge and the mirror charge, the electric field strength and direction in the space near the charged object can be determined.

[0069] To fully consider the safe working environment of live-line work and the obstacle avoidance requirements of the robotic arm, in one optional implementation, step S2015 includes:

[0070] Step S20151: Determine a first function, wherein the first function is a piecewise constant function in which the penalty value is a first set value when the robotic arm enters the dangerous space or exceeds the safe working space, and the penalty value is 0 when the robotic arm does not enter the dangerous space or exceeds the safe working space, wherein the first set value is a negative number.

[0071] Step S20152: Determine the second function, which is a piecewise constant function in which the reward value is a second set value when the robotic arm reaches the position of the target point, and the reward value is 0 when the robotic arm does not reach the position of the target point, wherein the second set value is a positive number.

[0072] Step S20153: Add the above relative distance, the above first function, and the above second function to obtain the above reward function.

[0073] In the above embodiments, the relative distance is calculated according to the seventh formula, which is: r1 represents the aforementioned relative distance, (x, y, z) represents the spatial coordinates of the robotic arm's end effector, and (a, b, c) represents the spatial coordinates of the target point. r2 is set for cases where the robotic arm touches hazardous space or exceeds the safe working space; this is the first function mentioned above. For the case where the robotic arm reaches the target point, set r3, which is the second function mentioned above. Finally, r1, r2, and r3 are added together to obtain the reward function R, R = r1 + r2 + r3. r1 represents the relative distance between the end of the robotic arm and the target point, r3 represents the penalty points deducted if the robotic arm exceeds the safe range, and r3 represents the reward points added if the robotic arm reaches the target point. The reward function fully considers the safe working environment of live-line work and the obstacle avoidance requirements of the robotic arm.

[0074] To enhance the robustness and adaptability of the target policy model, in an optional implementation, step S202 includes:

[0075] Step S2021: Initialize the evaluation network, action network, target evaluation network, and target action network;

[0076] Step S2022, generation step, input the current sample state into the above action network so that the above action network generates the optimal action according to the above current sample state, the above current sample state space is any sample state in the training sample set;

[0077] Step S2023, execution step: The evaluation network performs the above-mentioned optimal action and evaluates based on the above-mentioned reward function to obtain an evaluation score;

[0078] Step S2024, update step, using gradient descent to update the parameters in the above evaluation network, the above action network, the above target evaluation network and the above target action network;

[0079] Step S2025: Repeat the above steps, including the generation step, execution step, and update step, at least once, until the end of the robotic arm reaches the set position, and use the latest evaluation score as the cumulative reward value for the current iteration.

[0080] Step S2026: Repeat the above generation step, execution step, update step and repetition step at least once in sequence until the number of iterations reaches the set maximum number of iterations or the cumulative return value reaches the maximum value, to obtain the above target strategy model.

[0081] In the above embodiments, the purpose is to construct the DDPG algorithm model, i.e., the aforementioned target policy model. Deep Deterministic Policy Gradient (DDPG) is one of the classic algorithms in DRL, which is a product of the combination of reinforcement learning and neural network technology. The reinforcement learning process can be represented as a Markov decision process, defined as a tuple (S, A, P, R, γ), where each element represents a finite set of states S, a set of actions A, a state transition probability P, a reward R, and a discount factor γ ∈ [0, 1). This process can be described as follows: within each unit time step t, the agent acquires the current state information s. t ∈S, s t Input strategy π, output new action a t ∈A executes and obtains a new state s t+1 ∈S, and return s. t+1 Reward r t+1 The ultimate goal of the process is to obtain an optimal strategy π that maximizes the cumulative reward. Figure 6 This is a diagram of the deep deterministic policy gradient algorithm structure. It is based on specific network parameters (the network parameters are evaluated as θ). Q The action network parameters are θμ), and the evaluation network Q(S, A|θ) is initialized. Q The initial action network u(S, A|θ) is initialized. μ The training consists of a target evaluation network Q′ and a target action network u′. The goal of the learning phase is to update the weights of each node in both the evaluation and action networks. The current state is input into the action network, which outputs the next optimal action based on a probability distribution. The evaluation network scores the optimal action generated by the action network, and the action network then adjusts the probability of selecting each action based on the evaluation network's score. This process is repeated until the robotic arm reaches the target position, ending the current training. However, to ensure the accuracy of the target policy model, the target position to be reached by the robotic arm can be adjusted for retraining, or only the parameters of the policy model can be adjusted to select the model with the highest cumulative reward as the target policy model.

[0082] To improve the accuracy of the trained target policy model, in one optional implementation, step S2024 includes:

[0083] Step S20241: Minimize the third formula and update it using the evaluation network parameters in the evaluation network described above. The third formula is... N represents the total number of samples selected for learning from the training sample set, R i Q(S) represents the reward score calculated by the reward function, where γ represents the discount parameter. i A i |θ Q ) indicates the current sample state S mentioned above. i Perform the above optimal action A i The evaluation score obtained afterward, Q′((S) i+1 |θ u′ )|θ Q′ ) indicates the current sample state S mentioned above. i The target score after performing the action, θ Q For the above evaluation network parameters, θ Q′ To evaluate the network parameters, θ u′ S represents the target action network parameters. i Given the current sample state described above, A i This is the optimal action described above;

[0084] Step S20242: Update the action network parameters in the above action network according to the fourth formula, whereby the fourth formula is... θ u Denotes the above action network parameters, u(S|θ) u ) represents the above action network, Q(S, A|θ) Q ) represents the aforementioned evaluation network;

[0085] Step S20243: Update the target evaluation network parameters in the target evaluation network according to the fifth formula, where the fifth formula is θ. Q′ ←τθ Q +(1-τ)θ Q′ τ represents the parameter for soft updates;

[0086] Step S20244: Update the target action network parameters in the target action network according to the sixth formula, where the sixth formula is θ. u′ ←τθ u +(1-τ)θ u′ .

[0087] In the above embodiment, after the action network predicts action A, the evaluation network will give the value Q(S, A) as the evaluation of the current state S after performing action A. In order to enable the evaluation network to give the corresponding value accurately, the third formula is minimized to update the evaluation network parameters: In other words, the evaluation network parameters are updated by minimizing L, that is, by updating the evaluation network parameters θ in L. Q This minimizes the value of the third formula, L. L does not represent the evaluation network value; it has no actual meaning and is merely used to represent the formula. In action networks, to achieve a larger Q-value, the parameters of the action network are updated using the fourth formula. The Q-value is the sum of the cumulative future rewards expected to be obtained by taking a specific action A = u(S) in a specific state S. In other words, the Q-value reflects the "goodness" or "badness" of an action in a specific state. In reinforcement learning algorithms such as Q-learning, the Q-value is approximated by continuously learning and updating the optimal policy. In the fourth formula, Representing the gradient, the fifth and sixth formulas mentioned above are used to update the target evaluation network parameters and the target action network parameters, respectively.

[0088] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0089] This application also provides a trajectory planning device for collision avoidance of an equipotential working robot. It should be noted that this trajectory planning device can be used to execute the trajectory planning method for collision avoidance of an equipotential working robot provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0090] The following describes the trajectory planning device for collision avoidance of the equipotential working robot provided in the embodiments of this application.

[0091] Figure 7 This is a structural block diagram of a trajectory planning device for collision avoidance of an equipotential working robot arm according to an embodiment of this application.

[0092] like Figure 7 As shown, the device includes:

[0093] The determining unit 10 is used to determine the state space, motion space and reward function of the robotic arm. The state space is the set of states of the environment of the work space where the robotic arm is located and the state of the robotic arm itself. The motion space is the set of motions of the robotic arm moving in the space. The reward function is a function that takes into account the safe working space of the robotic arm under live operation and the requirement of the robotic arm to avoid dangerous spaces, while ensuring that the end of the robotic arm moves to the target point.

[0094] Specifically, deep reinforcement learning algorithms are used to implement path planning for the equipotential robotic arm. This involves training a path planning model using a deep deterministic policy gradient algorithm to generate the optimal path. Therefore, the parameters of the deep deterministic policy gradient algorithm, namely the state space, action space, and reward function, are first required. The action space defines all possible actions the robotic arm can perform in space. These actions can be: increments or absolute values ​​of joint angles; velocity or acceleration of the end effector; specific motion commands, such as forward, backward, left turn, right turn, etc. The reward function includes positive rewards: reduction in the distance between the robotic arm's end effector and the target point, completion of a specific task, or reaching a specific milestone; and negative rewards: approaching obstacles, entering unsafe working areas, or unstable states of the robotic arm, such as joint angles exceeding the normal operating range.

[0095] Training unit 20 is used to iteratively train the target policy model based on the above state space, action space and reward function using the deep deterministic policy gradient algorithm.

[0096] Specifically, the Actor Network and Critic Network are initialized, as are the Target Action Network and Target Critic Network, and the ReplayBuffer is initialized. Initial experience is collected, such as randomly selecting actions, executing them, and collecting interaction data with the environment (state, action, reward, new state). This interaction data is stored in the ReplayBuffer. A batch of data is randomly selected from the ReplayBuffer, and this data is used to update the parameters of the Critic Network and Actor Network for training, resulting in the final target policy model. After training, the obtained target policy model can be used to guide the robotic arm to perform tasks in a real-world environment. Utilizing deep reinforcement learning enables the equipotential robotic arm to have automatic obstacle avoidance capabilities, ensuring it does not enter dangerous areas in high-voltage electrical environments, playing a crucial role in ensuring the safe and stable operation of power equipment.

[0097] The acquisition unit 30 is used to acquire the target point position of the robotic arm, input the target point position into the target strategy model, and obtain the optimal movement path of the robotic arm.

[0098] Specifically, a vision system is used for coarse localization to determine the pose that the robotic arm's end effector needs to reach. Determining the robotic arm's end effector means determining the target point position that the robotic arm needs to reach, with the aim of ensuring that the robotic arm operates within a safe space and reaches the target point to perform subsequent tasks. After determining the target point position, the target point position is input into the aforementioned target strategy model to obtain the optimal movement path of the robotic arm.

[0099] Control unit 40 is used to control the robotic arm to move to the target point position according to the optimal movement path.

[0100] Specifically, after obtaining the optimal movement path of the robotic arm, it can be directly controlled to move to the target point according to the optimal movement path. Furthermore, to verify the effectiveness of this application, an equipotential robotic arm simulation experiment was conducted. For example... Figure 3 As shown, Figure 3 The system includes two points (point 1 and point 2), which are the target points that the robotic arm needs to reach. By using reinforcement learning to train the robotic arm, it can eventually reach the target points in an orderly manner without collisions.

[0101] In this embodiment, a working space is designed for the robotic arm, including a safe working space and a threat space. When the robotic arm works within the safe working space, the safety of the equipment in the substation and its own safety are ensured, while the task can be completed effectively. The robotic arm is trained using reinforcement learning, and a deep reinforcement learning algorithm is used to implement path planning for the equipotential robotic arm. This fully considers the safe working environment of live-line work and the obstacle avoidance requirements of the robotic arm, ensuring that the robotic arm can reach the target point in an orderly manner without collisions. This application solves the problem that traditional methods of controlling robotic arms are difficult to avoid hazardous spaces.

[0102] To achieve path planning for the equipotential robotic arm, ensuring both the safety of equipment within the substation and the arm's own operation while effectively completing the task, in one optional implementation, the determining unit includes:

[0103] The first determining subunit determines the self-state of the robotic arm and the environmental state of the robotic arm, and determines the state space based on the self-state and the environmental state of the robotic arm. The state space includes at least the self-state of the robotic arm and the environmental state. The self-state includes at least the joint angle of the robotic arm and the spatial position of the end effector of the robotic arm. The environmental state includes at least the spatial position of the target point and the relative distance between the end effector of the robotic arm and the target point.

[0104] The second determining subunit determines the aforementioned motion space based on the joint angles of the aforementioned robotic arm;

[0105] The first calculation subunit calculates the spatial distance between the end effector of the robotic arm and the target point to obtain the relative distance.

[0106] The second calculation subunit performs simulation calculations on the spatial electric field of the working environment of the robotic arm to obtain the safe working space and the dangerous space.

[0107] The third determining sub-unit determines the reward function based on the location of the robotic arm end effector, the relative distance, the safe working space, and the dangerous space.

[0108] In the above embodiments, the state space of the robotic arm is defined, such as... Figure 4 As shown, this invention uses a six-degree-of-freedom robotic arm for operation. The state space of the robotic arm refers to the state of the environment and the state of the robotic arm itself. The state space of the robotic arm mainly includes the angles of each joint, the position of the target point, and the position of the robotic arm's end effector. These serve as input variables, allowing the robotic arm to acquire relevant information and perform actions based on this information to obtain reward points. The state of the robotic arm itself refers to the changes in its structure, including the magnitude of the joint angles, which can be represented as A = (θ1, θ2, θ3, θ4, θ5, θ6). Simultaneously, the spatial coordinates of the robotic arm's end effector are defined as (x, y, z). The environmental state refers to the state acquired when the robotic arm interacts with the environment, mainly including the spatial coordinates of the target point (a, b, c) and the relative distance r1 between the robotic arm's end effector and the target point. The robotic arm's motion space is defined. To plan the trajectory of the robotic arm's end effector from the initial point to the target point, various environmental state information is first acquired, including the spatial coordinates of the target point and the robotic arm's end effector. Then, through inverse kinematics calculations, the joint angles are obtained. These joint angles are continuously adjusted to shorten the distance between the end effector and the target point. Therefore, the motion space is defined as A = (θ1, θ2, θ3, θ4, θ5, θ6), and the joint angles of the robotic arm constitute the motion space. The reward function is defined: the key to autonomous trajectory planning of the robotic arm lies in the method of obtaining the reward value. The design of the reward function is mainly based on the relative coordinates between the end effector and the target point. The relative distance is calculated by substituting the spatial positions of the end effector and the target point into the seventh formula. The seventh formula is... r1 represents the aforementioned relative distance, (x, y, z) represents the spatial coordinates of the robotic arm's end effector, and (a, b, c) represents the spatial coordinates of the target point. Secondly, the equipotential robotic arm should operate within a safe working area and must not touch other objects. The safe working space for live-line working robots is limited not only by the confined space between equipment in the substation but also by the electrical safety distances between phases and between phases and ground. Therefore, a safe working space must be designed for the robot, ensuring the safety of the equipment within the substation and the robot itself while effectively completing its task. The safe working space and hazardous space are planned through electromagnetic simulation of the environment. The reward function is determined based on the aforementioned relative distance and the positional relationship between the robotic arm's end effector and the safe working space and hazardous space.

[0109] To improve the safety of substation equipment and the robot itself by planning a safe working space through electromagnetic simulation of the robot's working environment, in one optional embodiment, the aforementioned second calculation subunit includes:

[0110] The simulation module uses the finite element analysis method to simulate and model the electric field distribution of the above-mentioned robotic arm within a set range, and obtains the electric field model. The set range is the set range of the hardware and wires before the above-mentioned robotic arm enters the live working position.

[0111] The first calculation module uses the mirror method to calculate the electric field of the mirror charge in the above electric field model, obtains the conductor electric field of the above robotic arm, and calculates the spatial electric field generated by the above mirror charge.

[0112] The second calculation module adds the above conductor electric field and the above space electric field to calculate the substation space electric field;

[0113] The third calculation module calculates the maximum electric field intensity on the surface of the flexible wire at different gap distances between the robotic arm and the adjacent flexible wire, based on the electric field of the substation space.

[0114] The fourth calculation module uses the key point method to calculate the distance between the joint center point of the above-mentioned robotic arm and the key point of the insulation safety boundary, and takes the smallest of the above-mentioned distances as the minimum distance.

[0115] The first determining module determines the area that meets the first condition and the second condition as the above-mentioned safe working space. The first condition is that the maximum electric field strength is less than or equal to the set maximum working electric field strength, and the second condition is that the minimum distance is greater than the set required distance.

[0116] The second determining module defines the area formed by an irregular sweeping body with the aforementioned soft wire as its axis and a specific length as its radius as the aforementioned dangerous space.

[0117] In the above embodiment, the image charge method is used to establish a model and calculate the surrounding spatial electric field. Specifically, finite element analysis software is used to simulate and model the electric field distribution around the hardware and conductors before the robot enters the target point. The image charge method is characterized by its simplicity, practicality, and high accuracy. This method does not require solving the Laplace equation with complex boundary conditions; it only needs to use image charges to replace various charged bodies and grounded bodies to obtain the potential and electric field strength at any point in the electric field. When calculating the spatial electric field around the busbar connecting conductor, the conductor can be regarded as a charged line segment with uniformly distributed charge, and the image charge is located at the center of the conductor. When the influence of the ground is taken into account, the image method is used to calculate the electric field, which can calculate the conductor electric field and the spatial electric field generated by the image charge. Adding the conductor electric field and the spatial electric field generated by the image charge gives the spatial electric field between the equipment in the substation where the robot is located, i.e., the aforementioned substation spatial electric field. Based on the electric field solution results, the maximum working electric field strength E of the robotic arm is set. m Then the electric field strength E in the robotic arm's workspace r E should be satisfied r ≤E m The robotic arm's working space includes a safe working space and a hazardous space. Based on the calculation of the maximum electric field strength on the surface of the flexible conductor under different gap distances between the robotic arm's elbow joint and the adjacent flexible conductor, the safe working space outside the hazardous space formed by an irregular sweeping body with the flexible conductor as its axis and a specific length as its radius can be derived. In simpler terms, it is a three-dimensional space extending along the length of the flexible conductor with the flexible conductor as its centerline. The boundary of this three-dimensional space is formed by extending the surface of the flexible conductor outward by a specific safe distance (i.e., a specific radius). Since the flexible conductor may not be perfectly straight and may have different bends and twists in space, the resulting spatial region is irregular. A hazardous boundary plane (also known as a safe boundary; areas not exceeding this plane are considered safe, and areas exceeding it are considered hazardous) is constructed. This plane is perpendicular to the ground and located between the flexible conductor and the gripping robotic arm. Let the distance between this hazardous boundary plane and the adjacent flexible conductor be Dissafe. Figure 5 As shown, the hazardous boundary is represented by a dashed line. The standard for safe live-line work is that the robotic arm should not contact the hazardous boundary plane. This plane is a hypothetical construct and does not actually exist. In the analysis, the key point method is used to calculate the minimum distance between the robot and the insulated safety boundary. This involves selecting the position of the robotic arm's end effector and the center point (x) of each joint. i y i , z i ), and the key points of the security boundary (x l y l , z lThe distances between all key points of the robotic arm and all key points of the insulation safety boundary are calculated using the distance formula. The smallest of all calculated distances is selected as the minimum distance d between the robotic arm and the insulation safety boundary. min The distance formula is: The required distance is set to d. m Then the minimum distance between the robot and the insulation safety boundary should satisfy d. min >d m .

[0118] In order to calculate the electric field distribution of the robotic arm, in one optional embodiment, the second calculation module includes:

[0119] The first calculation submodule calculates the electric field of the conductor in the robotic arm according to a first formula, wherein the first formula is: V represents the potential of the conductor in the aforementioned electric field, E d This represents the electric field of the aforementioned conductor;

[0120] The second calculation submodule calculates the spatial electric field generated by the aforementioned mirror charge according to the second formula, which is: E q Let q represent the electric field in the space described above, where k is the Coulomb constant and q is the electric field in the space i r is the magnitude of the i-th mirror charge in the aforementioned spatial electric field. i It is the distance from the i-th mirrored charge to the conductor. It is a unit vector pointing towards the conductor.

[0121] In the above embodiments, the simulated charge method uses simulated charges to replace various charged bodies and grounded bodies, thereby obtaining the potential and electric field strength at any point in the field. When considering the influence of the ground, the image method is used to calculate the electric field E at the conductor based on the known potential V of the conductor (already obtained using the simulated charge method) and the first formula. d The spatial electric field generated by the mirror charge is calculated using the second formula.

[0122] The method of images is a classic electromagnetic analysis method used to solve for the electric field or potential distribution around a charged object. Based on the principle of images, it simplifies the calculation of electromagnetic problems on complex boundaries by mirroring charges or currents where boundary conditions change. In electromagnetism, the method of images is often used to solve electromagnetic field problems near the surface of a conductor or a charged object. Specifically, the basic principle of the method of images is: if a charge is located near a grounded conductor, it can be assumed that there is an equal but opposite charge (i.e., a mirror charge) on the other side of the conductor. This makes the electric field on the conductor surface zero (because the electric field inside the conductor is always zero). By calculating the electric field produced by this mirror charge, the electric field distribution in the original problem can be approximated. By calculating the electric field under the combined action of the original charge and the mirror charge, the electric field strength and direction in the space near the charged object can be determined.

[0123] To fully consider the safe working environment of live-line work and the obstacle avoidance requirements of the robotic arm, in one optional embodiment, the aforementioned third determining subunit includes:

[0124] The third determining module determines the first function, which is a piecewise constant function in which the penalty value is a first set value when the robotic arm enters the dangerous space or exceeds the safe working space, and the penalty value is 0 when the robotic arm does not enter the dangerous space or exceeds the safe working space, and the first set value is a negative number.

[0125] The fourth determining module determines the second function, which is a piecewise constant function in which the reward value is a second set value when the robotic arm reaches the position of the target point, and the reward value is 0 when the robotic arm does not reach the position of the target point. The second set value is a positive number.

[0126] The addition module adds the aforementioned relative distance, the aforementioned first function, and the aforementioned second function to obtain the aforementioned reward function.

[0127] In the above embodiments, the relative distance is calculated according to the seventh formula, which is: r1 represents the aforementioned relative distance, (x, y, z) represents the spatial coordinates of the robotic arm's end effector, and (a, b, c) represents the spatial coordinates of the target point. r2 is set for cases where the robotic arm touches hazardous space or exceeds the safe working space; this is the first function mentioned above. For the case where the robotic arm reaches the target point, set r3, which is the second function mentioned above. Finally, the sum of r1, r3, and r3 gives the reward function R, where R = r i+r2+r3, where r1 represents the relative distance between the end of the robotic arm and the target point, r3 represents the penalty points deducted if the robotic arm exceeds the safe range, and r3 represents the reward points added if the robotic arm reaches the target point. The reward function fully considers the safe working environment of live-line work and the obstacle avoidance requirements of the robotic arm.

[0128] To enhance the robustness and adaptability of the target policy model, in one optional implementation, the training unit includes:

[0129] Initialize the sub-units, and initialize the evaluation network, action network, target evaluation network, and target action network;

[0130] The generation subunit is used to perform the generation step, inputting the current sample state into the action network so that the action network generates the optimal action based on the current sample state. The current sample state space is any sample state in the training sample set.

[0131] The execution subunit is used to execute the execution steps. The evaluation network executes the above-mentioned optimal action and obtains an evaluation score based on the above-mentioned reward function.

[0132] The update sub-unit is used to perform the update step, and the gradient descent method is used to update the parameters in the above evaluation network, the above action network, the above target evaluation network and the above target action network.

[0133] The first repeating subunit is used to perform the repeating steps, repeating the above generation steps, the above execution steps and the above update steps at least once in sequence until the end of the robotic arm reaches the set position, and using the latest evaluation score as the cumulative reward value of the current iteration.

[0134] The second repeating subunit is used to repeat the above generation step, the above execution step, the above update step and the above repeating step at least once, until the number of iterations reaches the set maximum number of iterations or the above cumulative reward value reaches the maximum value, so as to obtain the above target strategy model.

[0135] In the above embodiments, the purpose is to construct the DDPG algorithm model, i.e., the aforementioned target policy model. Deep Deterministic Policy Gradient (DDPG) is one of the classic algorithms in DRL, which is a product of the combination of reinforcement learning and neural network technology. The reinforcement learning process can be represented as a Markov decision process, defined as a tuple (S, A, P, R, γ), where each element represents a finite set of states S, a set of actions A, a state transition probability P, a reward R, and a discount factor γ ∈ [0, 1). This process can be described as follows: within each unit time step t, the agent acquires the current state information s.t ∈S, s t Input strategy π, output new action a t ∈A executes and obtains a new state s t+1 ∈S, and return s. t+1 Reward r t+1 The ultimate goal of the process is to obtain an optimal strategy π that maximizes the cumulative reward. Figure 6 This is a diagram of the deep deterministic policy gradient algorithm structure. It is based on specific network parameters (the network parameters are evaluated as θ). Q The action network parameters are θμ), and the evaluation network Q(S, A|θ) is initialized. Q The initial action network u(S, A|θ) is initialized. μ The training consists of a target evaluation network Q′ and a target action network u′. The goal of the learning phase is to update the weights of each node in both the evaluation and action networks. The current state is input into the action network, which outputs the next optimal action based on a probability distribution. The evaluation network scores the optimal action generated by the action network, and the action network then adjusts the probability of selecting each action based on the evaluation network's score. This process is repeated until the robotic arm reaches the target position, ending the current training. However, to ensure the accuracy of the target policy model, the target position to be reached by the robotic arm can be adjusted for retraining, or only the parameters of the policy model can be adjusted to select the model with the highest cumulative reward as the target policy model.

[0136] To improve the accuracy of the trained target policy model, in one optional implementation, the above step of updating the sub-unit includes:

[0137] The first update module minimizes the third formula and updates it using the evaluation network parameters from the aforementioned evaluation network. The third formula is... N represents the total number of samples selected for learning from the training sample set, R i Q(S) represents the reward score calculated by the reward function, where γ represents the discount parameter. i A i |θ Q ) indicates the current sample state S mentioned above. i Perform the above optimal action A i The evaluation score obtained afterward, Q′((S) i+1 |θ u′ )|θ Q ) indicates the current sample state S mentioned above. i The target score after performing the action, θ Q For the above evaluation network parameters, θ Q′ To evaluate the network parameters, θ u′ S represents the target action network parameters. iGiven the current sample state described above, A i This is the optimal action described above;

[0138] The second update module updates the action network parameters in the action network according to the fourth formula, which is: θ u Denotes the above action network parameters, u(S|θ) u ) represents the above action network, Q(S, A|θ) Q ) represents the aforementioned evaluation network;

[0139] The third update module updates the target evaluation network parameters in the aforementioned target evaluation network according to the fifth formula, where the fifth formula is θ. Q′ ←τθ Q +(1-τ)θ Q′ τ represents the parameter for soft updates;

[0140] The fourth update module updates the target action network parameters in the target action network according to the sixth formula, where the sixth formula is θ. u′ ←τθ u +(1-τ)θ u′ .

[0141] In the above embodiment, after the action network predicts action A, the evaluation network will give the value Q(S, A) as the evaluation of the current state S after performing action A. In order to enable the evaluation network to give the corresponding value accurately, the third formula is minimized to update the evaluation network parameters: In other words, the evaluation network parameters are updated by minimizing L, that is, by updating the evaluation network parameters θ in L. Q This minimizes the value of the third formula, L. L does not represent the evaluation network value; it has no actual meaning and is merely used to represent the formula. In action networks, to achieve a larger Q-value, the parameters of the action network are updated using the fourth formula. The Q-value is the sum of the cumulative future rewards expected to be obtained by taking a specific action A = u(S) in a specific state S. In other words, the Q-value reflects the "goodness" or "badness" of an action in a specific state. In reinforcement learning algorithms such as Q-learning, the Q-value is approximated by continuously learning and updating the optimal policy. In the fourth formula, Representing the gradient, the fifth and sixth formulas mentioned above are used to update the target evaluation network parameters and the target action network parameters, respectively.

[0142] The aforementioned trajectory planning device for collision avoidance of the equipotential working robotic arm includes a processor and a memory. The determination unit, training unit, and acquisition unit are all stored as program units in the memory, and the processor executes these program units to achieve the corresponding functions. All of the above modules reside in the same processor; alternatively, the modules may be located in different processors in any combination.

[0143] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the problem of traditional methods of controlling robotic arms to avoid dangerous spaces.

[0144] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0145] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the trajectory planning method for collision avoidance of the equipotential working robot.

[0146] This invention provides a processor for running a program, wherein the program executes the trajectory planning method for collision avoidance of the equipotential working robot.

[0147] This invention provides an equipotential robotic arm collision avoidance trajectory planning system. The system includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs at least the following steps:

[0148] Step S201: Determine the state space, motion space, and reward function of the robotic arm. The state space is the set of states of the environment of the work space where the robotic arm is located and the state of the robotic arm itself. The motion space is the set of motions of the robotic arm moving in the space. The reward function is a function that takes into account the safe working space of the robotic arm under live operation and the requirement of the robotic arm to avoid dangerous spaces, while ensuring that the end of the robotic arm moves to the target point.

[0149] Step S202: Based on the above state space, action space and reward function, iterative training is performed using the deep deterministic policy gradient algorithm to obtain the target policy model;

[0150] Step S203: Obtain the target point position of the robotic arm, input the target point position into the target strategy model, and obtain the optimal movement path of the robotic arm.

[0151] Step S204: Control the robotic arm to move to the target point position according to the optimal movement path.

[0152] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having at least the following method steps:

[0153] Step S201: Determine the state space, motion space, and reward function of the robotic arm. The state space is the set of states of the environment of the work space where the robotic arm is located and the state of the robotic arm itself. The motion space is the set of motions of the robotic arm moving in the space. The reward function is a function that takes into account the safe working space of the robotic arm under live operation and the requirement of the robotic arm to avoid dangerous spaces, while ensuring that the end of the robotic arm moves to the target point.

[0154] Step S202: Based on the above state space, action space and reward function, iterative training is performed using the deep deterministic policy gradient algorithm to obtain the target policy model;

[0155] Step S203: Obtain the target point position of the robotic arm, input the target point position into the target strategy model, and obtain the optimal movement path of the robotic arm.

[0156] Step S204: Control the robotic arm to move to the target point position according to the optimal movement path.

[0157] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0158] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0159] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0160] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0161] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0162] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0163] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0164] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0165] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0166] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0167] 1) The trajectory planning method for collision avoidance of the equipotential working robotic arm in this application designs a working space for the robotic arm, which includes a safe working space and a threat space. When the robotic arm works within the safe working space, it can ensure the safety of the equipment in the substation and its own safety while effectively completing the task. By using reinforcement learning to train the robotic arm, and employing a deep reinforcement learning algorithm to achieve path planning for the equipotential working robotic arm, the method fully considers the safe working environment of live-line work and the obstacle avoidance requirements of the robotic arm, ensuring that the robotic arm can reach the target point in an orderly manner without collision. This application solves the problem that traditional methods of controlling robotic arms are difficult to avoid hazardous spaces.

[0168] 2) The trajectory planning device for collision avoidance of the equipotential working robotic arm in this application designs a working space for the robotic arm, which includes a safe working space and a threat space. When the robotic arm works in the safe working space, it can ensure the safety of the equipment in the substation and its own safety, while effectively completing the task. By using reinforcement learning to train the robotic arm, and using a deep reinforcement learning algorithm to realize the path planning of the equipotential working robotic arm, it fully considers the safe working environment of live-line work and the obstacle avoidance requirements of the robotic arm, ensuring that the robotic arm can reach the target point in an orderly manner without collision. This application solves the problem that traditional methods of controlling robotic arms are difficult to avoid dangerous spaces.

[0169] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A trajectory planning method for collision avoidance by an equipotential working robotic arm, characterized in that, include: The state space, motion space, and reward function of the robotic arm are determined. The state space is the set of states of the environment in the work space where the robotic arm is located and the state of the robotic arm itself. The motion space is the set of actions of the robotic arm moving in the space. The reward function is a function that takes into account the safe working space of the robotic arm under live operation and the need for the robotic arm to avoid dangerous spaces, while ensuring that the end effector of the robotic arm moves to the target point. Based on the state space, the action space, and the reward function, the target policy model is obtained through iterative training using the deep deterministic policy gradient algorithm. The target point position of the robotic arm is obtained, and the target point position is input into the target strategy model to obtain the optimal movement path of the robotic arm; The robotic arm is controlled to move to the target point position according to the optimal movement path. Determining the state space, motion space, and reward function of a robotic arm includes: determining the robotic arm's own state and the robotic arm's environmental state; determining the state space based on the robotic arm's own state and the robotic arm's environmental state, wherein the state space includes at least the robotic arm's own state and the environmental state; the robotic arm's own state includes at least the joint angles of the robotic arm and the spatial position of the robotic arm's end effector; the environmental state includes at least the spatial position of a target point and the relative distance between the robotic arm's end effector and the target point; determining the motion space based on the robotic arm's joint angles; and calculating the relative distance between the robotic arm's end effector and the target point. The spatial electric field of the working environment of the robotic arm is simulated and calculated to obtain the safe working space and the dangerous space; the reward function is determined based on the position of the robotic arm end effector, the relative distance, the safe working space, and the dangerous space. The electric field of the working environment of the robotic arm is simulated and calculated to obtain the safe working space and the hazardous space. This includes: using finite element analysis to simulate and model the electric field distribution of the robotic arm within a set range, obtaining an electric field model; the set range being the range of hardware and wires before the robotic arm enters the energized working position; using the method of images to calculate the electric field of the image charges in the electric field model, obtaining the conductor electric field of the robotic arm, and calculating the spatial electric field generated by the image charges; adding the conductor electric field and the spatial electric field to obtain the substation spatial electric field; and based on the substation spatial electric field... Calculate the maximum electric field strength on the surface of the flexible wire at different gap distances between the robotic arm and the adjacent flexible wire; calculate the distance between the joint center point of the robotic arm and the key point of the insulation safety boundary using the key point method, and take the minimum distance as the minimum distance; determine the area that satisfies the first condition and the second condition as the safe working space, wherein the first condition is that the maximum electric field strength is less than or equal to the set maximum working electric field strength, and the second condition is that the minimum distance is greater than the set required distance; determine the area formed by an irregular sweeping body with the flexible wire as the axis and a specific length as the radius as the danger space.

2. The method according to claim 1, characterized in that, The electric field of the image charge in the electric field model is calculated using the method of images to obtain the conductor electric field of the robotic arm, and the spatial electric field generated by the image charge is also calculated, including: The electric field of the conductor in the robotic arm is calculated according to the first formula, which is: , The electric potential of the conductor in the electric field is represented. This represents the electric field of the conductor; The spatial electric field generated by the mirror charge is calculated according to the second formula, which is: , Represents the electric field in space. It is the Coulomb constant. It is the magnitude of the i-th mirror charge in the aforementioned spatial electric field. It is the distance from the i-th mirror charge to the conductor. It is a unit vector pointing towards the conductor.

3. The method according to claim 1, characterized in that, The reward function is determined based on the position of the robotic arm's end effector, the relative distance, the safe working space, and the danger zone, including: A first function is defined, wherein the penalty value is a first set value when the robotic arm enters the dangerous space or the robotic arm exceeds the safe working space, and the penalty value is 0 when the robotic arm does not enter the dangerous space or the robotic arm does not exceed the safe working space, wherein the first set value is a negative number; A second function is determined, wherein the reward value is a second set value when the robotic arm reaches the position of the target point, and the reward value is 0 when the robotic arm does not reach the position of the target point, wherein the second set value is a positive number; The reward function is obtained by adding the relative distance, the first function, and the second function.

4. The method according to claim 1, characterized in that, Based on the state space, the action space, and the reward function, an iterative training process using a deep deterministic policy gradient algorithm is performed to obtain a target policy model, including: Initialize the evaluation network, action network, target evaluation network, and target action network; The generation step involves inputting the current sample state into the action network so that the action network generates the optimal action based on the current sample state. The current sample state space is any sample state in the training sample set. The execution steps involve the evaluation network performing the optimal action and evaluating the result based on the reward function to obtain an evaluation score. The update step involves using gradient descent to update the parameters in the evaluation network, the action network, the target evaluation network, and the target action network. Repeat the steps, sequentially repeating the generation step, the execution step, and the update step at least once, until the end of the robotic arm reaches the set position, and use the latest evaluation score as the cumulative reward value for the current iteration; The generation step, the execution step, the update step, and the repetition step are repeated at least once in sequence until the number of iterations reaches the set maximum number of iterations or the cumulative reward value reaches the maximum value, thereby obtaining the target strategy model.

5. The method according to claim 4, characterized in that, The parameters in the evaluation network, the action network, the target evaluation network, and the target action network are updated using the gradient descent method, including: The third formula is minimized to update the evaluation network parameters in the evaluation network, and the third formula is: N represents the total number of samples selected from the training set for learning. The reward score is calculated by the reward function. Represents the discount parameter. Indicates the current sample state Perform the optimal action The evaluation score obtained subsequently Indicates the current sample state The target score after performing the action. For the evaluation network parameters, To evaluate network parameters for the target, For the target action network parameters, The current sample state, This is the optimal action; The action network parameters in the action network are updated according to the fourth formula, which is: , Indicates the parameters of the action network. This represents the action network. This represents the evaluation network; The target evaluation network parameters are updated according to the fifth formula, which is: , Parameters representing soft updates; The target action network parameters in the target action network are updated according to the sixth formula, which is: .

6. A trajectory planning device for collision avoidance in an equipotential robotic arm, characterized in that, The device, applied to the trajectory planning method for collision avoidance of an equipotential working robot arm according to any one of claims 1 to 5, comprises: The determining unit is used to determine the state space, motion space and reward function of the robotic arm. The state space is a set of states of the environment of the work space in which the robotic arm is located and the state of the robotic arm itself. The motion space is a set of motions of the robotic arm moving in the space. The reward function is a function that takes into account the safe working space of the robotic arm under live operation and the need for the robotic arm to avoid dangerous spaces, while ensuring that the end effector of the robotic arm moves to the target point. The training unit is used to perform iterative training based on the state space, the action space, and the reward function using a deep deterministic policy gradient algorithm to obtain the target policy model. The acquisition unit is used to acquire the target point position of the robotic arm, input the target point position into the target strategy model, and obtain the optimal movement path of the robotic arm. The control unit is used to control the robotic arm to move to the target point position according to the optimal movement path.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 5.

8. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the method described in any one of claims 1 to 5.

Images