Robot obstacle avoidance path planning method and system, storage medium and computer equipment

Abstract

The invention discloses a robot obstacle avoidance path planning method and system, a storage medium and computer equipment, and the method comprises the steps: receiving a real working environment image sent by an entity robot, identifying the current position coordinate of an object to be grabbed, the coordinate of an obstacle and the target position coordinate of the object to be grabbed from the real working environment image; according to the current position coordinates of the to-be-grabbed object, the obstacle coordinates and the target position coordinates of the to-be-grabbed object, a virtual working environment of the twin robot corresponding to the entity robot is constructed; according to the virtual working environment, training the twin robot by using a reinforcement learning algorithm to obtain an obstacle avoidance path with a maximum reward value, and determining a target obstacle avoidance path of the entity robot based on the obstacle avoidance path with the maximum reward value; and the target obstacle avoidance path is sent to a control device of the entity robot, so that the entity robot is controlled by the control device to move the to-be-grabbed object from the current position to the target position based on the target obstacle avoidance path.

Description

Technical Field

[0001] The present application relates to the field of industrial intelligent manufacturing technology, and in particular to a robot obstacle avoidance path planning method and system, storage medium, and computer equipment. Background Art

[0002] With the rapid development of automation technology, robots are increasingly used in industrial production, warehousing and logistics, etc. In industrial production application scenarios, robots usually need to complete complex tasks, such as grabbing objects, moving objects, etc. At the same time, in the process of moving objects, there may be obstacles in the working environment, so robots need to autonomously navigate in an environment with obstacles, avoid collisions with obstacles, and move objects safely to the target location.

[0003] Traditional robot path planning methods often rely on predefined maps and fixed algorithms. However, the path accuracy ultimately obtained by this method is low, and when the robot moves along the planned path, the robot may face many unsafe factors due to path deviation, such as collisions with obstacles, which increases the robot's trial and error cost. Summary of the invention

[0004] In view of this, the present application provides a robot obstacle avoidance path planning method and system, storage medium, and computer equipment. Through the reinforcement learning algorithm, the twin robots are allowed to continuously trial and error in a virtual working environment, and finally obtain the target obstacle avoidance path, which can greatly improve the accuracy of the obstacle avoidance path. At the same time, it can reduce the trial and error cost of the physical robot, avoid the unsafe factors caused by directly operating the physical robot, and reduce the loss of the physical robot.

[0005] According to one aspect of the present application, a robot obstacle avoidance path planning method is provided, comprising:

[0006] Receiving a real working environment image sent by a physical robot, and identifying the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped from the real working environment image, wherein the real working environment image is acquired by a visual perception device carried by the physical robot;

[0007] Constructing a virtual working environment of the twin robot corresponding to the physical robot according to the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped;

[0008] According to the virtual working environment, the twin robot is trained using a reinforcement learning algorithm to obtain an obstacle avoidance path with a maximum reward value, and a target obstacle avoidance path of the physical robot is determined based on the obstacle avoidance path with the maximum reward value;

[0009] The target obstacle avoidance path is sent to the control device of the physical robot, so that the control device controls the physical robot to move the object to be grasped from the current position to the target position based on the target obstacle avoidance path.

[0010] According to another aspect of the present application, a robot obstacle avoidance path planning system is provided, comprising:

[0011] A coordinate recognition module, used to receive a real working environment image sent by the physical robot, and recognize the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped from the real working environment image, wherein the real working environment image is acquired by a visual perception device carried by the physical robot;

[0012] An environment construction module, used to construct a virtual working environment of the twin robot corresponding to the physical robot according to the current position coordinates of the object to be grasped, the obstacle coordinates and the target position coordinates of the object to be grasped;

[0013] A training module, configured to train the twin robot using a reinforcement learning algorithm according to the virtual working environment to obtain an obstacle avoidance path with a maximum reward value, and determine a target obstacle avoidance path of the physical robot based on the obstacle avoidance path with a maximum reward value;

[0014] The path sending module is used to send the target obstacle avoidance path to the control device of the physical robot, so as to control the physical robot through the control device to move the object to be grasped from the current position to the target position based on the target obstacle avoidance path.

[0015] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned robot obstacle avoidance path planning method is implemented.

[0016] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned robot obstacle avoidance path planning method when executing the program.

[0017] By means of the above technical scheme, the present application provides a robot obstacle avoidance path planning method and system, storage medium, and computer equipment. After receiving the real working environment image, the current position coordinates, obstacle coordinates, and target position coordinates of the object to be grasped can be identified from the real working environment image. Afterwards, based on the identified current position coordinates, obstacle coordinates, and target position coordinates of the object to be grasped, a virtual working environment of a twin robot corresponding to the physical robot can be constructed. Then, in the virtual working environment, the twin robot is trained using a reinforcement learning algorithm. Through continuous iterative learning, the twin robot can eventually find an obstacle avoidance path that obtains the maximum reward value, and then the target obstacle avoidance path of the physical robot, that is, the optimal path, can be determined based on the obstacle avoidance path. After obtaining the target obstacle avoidance path, the information of the target obstacle avoidance path is sent to the control device of the physical robot. The control device controls the physical robot to move along the planned path based on the received information of the target obstacle avoidance path, thereby moving the object to be grasped from the current position to the target position. The embodiment of the present application uses a reinforcement learning algorithm to allow the twin robot to continuously trial and error in a virtual working environment, and finally obtain a target obstacle avoidance path, which can greatly improve the accuracy of the obstacle avoidance path. At the same time, it can reduce the trial and error cost of the physical robot, avoid the unsafe factors caused by directly operating the physical robot, and reduce the loss of the physical robot.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A schematic diagram of a process flow of a robot obstacle avoidance path planning method provided in an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram showing a method for extracting the current position coordinates of an object to be grasped and the coordinates of obstacles provided in an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram showing the training results of a network with different strategies provided in an embodiment of the present application;

[0023] Figure 4A three-dimensional schematic diagram of an optimal obstacle avoidance path and obstacle position planned under a different strategy network provided in an embodiment of the present application is shown;

[0024] Figure 5 A schematic diagram of the structure of a robot obstacle avoidance path planning system provided in an embodiment of the present application is shown;

[0025] Figure 6 A schematic diagram of the device structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0027] In this embodiment, a robot obstacle avoidance path planning method is provided. Figure 1 As shown, the method includes:

[0028] Step 101, receiving a real working environment image sent by a physical robot, and identifying the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped from the real working environment image, wherein the real working environment image is collected by a visual perception device carried by the physical robot.

[0029] A robot obstacle avoidance path planning method provided in an embodiment of the present application combines visual perception, digital twin, and reinforcement learning technology to plan the obstacle avoidance path of the robot operation, aiming to help the physical robot efficiently and safely complete the task of grasping and carrying objects in a complex working environment, while avoiding collisions with obstacles. Among them, the physical robot is equipped with a visual perception device, and the real working environment image of the physical robot can be collected by the visual perception device (such as a camera). After the visual perception device collects the real working environment image, the real working environment image can be sent to the robot obstacle avoidance path planning system through the physical robot. After the robot obstacle avoidance path planning system receives the real working environment image, it can identify the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped from the real working environment image through image processing technology (such as computer vision algorithm). This information is the basis for subsequent obstacle avoidance path planning.

[0030] Step 102: construct a virtual working environment of the twin robot corresponding to the physical robot according to the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped.

[0031] In this embodiment, based on the identified current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped, a virtual working environment of a twin robot corresponding to the physical robot can be constructed. This virtual working environment corresponds to the real working environment of the physical robot, and is a digital replica of the real working environment of the physical robot, containing all key physical features, such as the position of the object to be grasped, the distribution of obstacles, etc. The application of digital twin technology allows simulation experiments and path planning in a virtual working environment without interfering with the actual operation of the physical robot, thereby improving the efficiency and safety of planning. It should be noted that the twin robot in the embodiment of the present application is completely constructed based on the physical robot, and its operation in the virtual working environment can simulate the operation of the physical robot in the real working environment.

[0032] Step 103: According to the virtual working environment, the twin robot is trained using a reinforcement learning algorithm to obtain an obstacle avoidance path with a maximum reward value, and a target obstacle avoidance path of the physical robot is determined based on the obstacle avoidance path with a maximum reward value.

[0033] In this embodiment, then, in a virtual working environment, the twin robot is trained using a reinforcement learning algorithm. Reinforcement learning is a machine learning method that allows the twin robot to learn how to achieve its goals while trying different behaviors, while considering rewards (positive feedback) and penalties (negative feedback) to optimize its behavior strategy. In this process, the twin robot can try a variety of possible paths, and after each attempt, it obtains a corresponding reward value based on whether it successfully avoids obstacles, path length, time efficiency and other factors. Through continuous iterative learning, the twin robot can eventually find an obstacle avoidance path that obtains the maximum reward value, and then the target obstacle avoidance path of the physical robot, that is, the optimal path, can be determined based on the obstacle avoidance path.

[0034] Step 104: Send the target obstacle avoidance path to the control device of the physical robot, so that the control device controls the physical robot to move the object to be grasped from the current position to the target position based on the target obstacle avoidance path.

[0035] In this embodiment, the information of the target obstacle avoidance path is sent to the control device of the physical robot. The control device controls the physical robot to move along the planned path based on the received information of the target obstacle avoidance path, thereby safely and efficiently moving the object to be grasped from the current position to the target position.

[0036] By applying the technical solution of this embodiment, after receiving the real working environment image, the current position coordinates, obstacle coordinates and target position coordinates of the object to be grasped can be identified from the real working environment image. Afterwards, based on the identified current position coordinates, obstacle coordinates and target position coordinates of the object to be grasped, a virtual working environment of a twin robot corresponding to the physical robot can be constructed. Then, in the virtual working environment, the twin robot is trained using a reinforcement learning algorithm. Through continuous iterative learning, the twin robot can eventually find an obstacle avoidance path that obtains the maximum reward value, and then the target obstacle avoidance path of the physical robot, that is, the optimal path, can be determined based on the obstacle avoidance path. After obtaining the target obstacle avoidance path, the information of the target obstacle avoidance path is sent to the control device of the physical robot. The control device controls the physical robot to move along the planned path based on the received information of the target obstacle avoidance path, thereby moving the object to be grasped from the current position to the target position. The embodiment of the present application uses a reinforcement learning algorithm to allow the twin robot to continuously trial and error in a virtual working environment, and finally obtain a target obstacle avoidance path, which can greatly improve the accuracy of the obstacle avoidance path. At the same time, it can reduce the trial and error cost of the physical robot, avoid the unsafe factors caused by directly operating the physical robot, and reduce the loss of the physical robot.

[0037] In an embodiment of the present application, optionally, the robot is a multi-axis parallel robot; before the step 103 of "training the twin robot using a reinforcement learning algorithm", the method also includes: defining an action space of the reinforcement learning algorithm, wherein each action in the action space is used to indicate a rotation angle change of each axis of the multi-axis parallel robot at a current position; defining a state space of the reinforcement learning algorithm, wherein the state representation in the state space is based on the current angles of each axis of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform, and the current three-dimensional coordinates of the center point of the moving platform are obtained based on the positive kinematic modeling of the twin robot; defining a reward function of the reinforcement learning algorithm, wherein the reward function is composed of The multi-axis parallel robot comprises a sparse reward, an intermediate reward and an action reward, wherein the sparse reward is determined based on the current position coordinates of the end of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform, the intermediate reward is determined based on the starting three-dimensional coordinates, the current three-dimensional coordinates and the end three-dimensional coordinates of the center point of the moving platform of the multi-axis parallel robot, and the action reward is determined based on the number of actions of the twin robot; accordingly, the step 103 of "training the twin robot according to the virtual working environment using a reinforcement learning algorithm to obtain an obstacle avoidance path with a maximum reward value" includes: training the twin robot according to the virtual working environment and the action space, state space and reward function of the defined reinforcement learning algorithm to obtain an obstacle avoidance path with a maximum reward value.

[0038] In this embodiment, the robot may be a multi-axis parallel robot. Before using the reinforcement learning algorithm to train the twin robot, the action space, state space, and reward function of the reinforcement learning algorithm may be defined first.

[0039] (1) Action space definition. The action space refers to the set of all possible actions that the multi-axis parallel robot can perform at each decision point. Each action represents a change in the rotation angle of each axis of the multi-axis parallel robot at the current position. That is, the multi-axis parallel robot can change its overall position and posture by adjusting the angle of each axis. The definition of the action space is the basis of the reinforcement learning algorithm because it determines what actions the multi-axis parallel robot can take to explore the environment. For example, the multi-axis parallel robot mentioned above is specifically a three-axis parallel robot with three degrees of freedom. Assuming that any axis of the three-axis parallel robot can rotate 1 degree, 0 degree, or -1 degree at the current position, then when an action is represented as [0, 1, 1], it means that the three-axis parallel robot does not move at the current position at axis 0, rotates 1 degree at axis 1, and rotates 1 degree at axis 2. Therefore, according to this rule, the action space is set, and each axis has three cases, and the dimension of the action space is 27.

[0040] (2) Definition of state space. The state representation in the state space is actually determined based on the current angles of the axes of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform. The definition of the state space enables the reinforcement learning algorithm to understand the current angles of the axes of the multi-axis parallel robot and the current position of the center point of the moving platform, so as to make decisions. For example, the multi-axis parallel robot is a three-axis parallel robot. The current angles of the three axes can be expressed as n0, n1, and n2. The state representation in the state space is constructed by the current angles of the axes and the current three-dimensional coordinates (x, y, z) of the center point of the moving platform obtained by solving the positive kinematics of the three-axis parallel robot. The state representation s is s = [n0, n1, n2, x, y, z]. Here, the current three-dimensional coordinates of the center point of the moving platform are obtained by solving the positive kinematics modeling of the twin robot.

[0041] Specifically, the kinematics forward solution of the multi-axis parallel robot is the process of solving the three-dimensional coordinates of the center point of the moving platform relative to the static platform coordinate system according to the kinematic equation when the opening angle of the active arm relative to the static platform is known. The current three-dimensional coordinates of the center point of the moving platform can be calculated based on the known data such as the opening angle of the active arm relative to the static platform.

[0042] (3) Reward function definition. The reward function is a core component in the reinforcement learning algorithm, which defines the immediate reward that the multi-axis parallel robot receives after performing a certain action. In the embodiment of the present application, the reward function consists of three parts: sparse reward, intermediate reward and action reward. Among them, the sparse reward is determined based on the current position coordinates of the end of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform. This reward is usually given when the robot reaches the target position (that is, the position where the object to be grasped finally wants to reach), so it is sparse (not every action has a reward). The intermediate reward is determined based on the starting three-dimensional coordinates, current three-dimensional coordinates and end three-dimensional coordinates of the center point of the moving platform. This reward encourages the robot to move towards the target position even if it does not reach the target position directly. Among them, the starting three-dimensional coordinates are the three-dimensional coordinates of the center point of the moving platform when the twin robot is at the starting position, which are known; the end three-dimensional coordinates are the three-dimensional coordinates of the center point of the moving platform when the end of the twin robot reaches the target position, which can be calculated based on the target position coordinates. The action reward is determined based on the number of actions of the twin robot, and is used to encourage the robot to complete the task with fewer actions, thereby improving efficiency.

[0043] After defining the action space, state space, and reward function, the twin robot is finally trained based on the virtual working environment and the defined action space, state space, and reward function. The goal of the training is to find an obstacle avoidance path that maximizes the cumulative reward value. The reinforcement learning algorithm continuously tries different action sequences and updates its strategy based on the rewards fed back by the reward function, and finally finds an obstacle avoidance path with the maximum reward value.

[0044] The embodiment of the present application defines the action space, state space and reward function, and uses the reinforcement learning algorithm to train the twin robot to find an optimal path to avoid obstacles in a virtual environment, which can greatly improve the autonomous navigation capability of the multi-axis parallel robot in a complex environment.

[0045] In the embodiment of the present application, optionally, the calculation formula of the sparse reward is as follows:

[0046]

[0047] Among them, R spRepresents a sparse reward for the current action; when the current position coordinates of the end of the multi-axis parallel robot coincide with the target position coordinates, it is determined that the end of the multi-axis parallel robot has reached the target position, and the current position coordinates of the end of the multi-axis parallel robot are determined based on the current position coordinates of the center point of the moving platform; when the current position coordinates of the center point of the moving platform of the multi-axis parallel robot intersect with the obstacle coordinate range, it is determined that the moving platform of the multi-axis parallel robot has collided with an obstacle; when the current position coordinates of the center point of the moving platform of the multi-axis parallel robot exceed the coordinate range corresponding to the preset workspace, it is determined that the moving platform of the multi-axis parallel robot exceeds the preset workspace;

[0048] The calculation formula of the intermediate reward is as follows:

[0049]

[0050] Among them, R t represents the intermediate reward of the current action; x, y, z represent the current three-dimensional coordinates of the center point of the moving platform, x0, y0, z0 represent the starting three-dimensional coordinates of the center point of the moving platform, x d ,y d ,z d represents the three-dimensional coordinates of the end point of the center point of the moving platform, and ω is the weight value;

[0051] The calculation formula of the reward function is as follows:

[0052] R=R sp +R t +R s ;

[0053] Among them, R represents the final reward of the current action, R s (R s <0) indicates the action reward of the current action.

[0054] In this embodiment, it is found in the experiment that the training efficiency of only setting sparse rewards is very low and far from meeting the requirements of practical applications. Therefore, an intermediate reward is added during the exploration process of the twin robot to guide the twin robot to approach the target position. a 、r b 、r cAnd ω can be determined based on empirical values. In the physical robot, since the position between the end of the multi-axis parallel robot and the center point of the moving platform is relatively fixed, after determining the current three-dimensional coordinates of the center point of the moving platform, the current position coordinates of the end of the multi-axis parallel robot can be calculated based on the current three-dimensional coordinates of the center point of the moving platform. Here, the end of the multi-axis parallel robot can be the grasping end of the object to be grasped. The preset workspace can be a space pre-set by the staff, which can be achieved by inputting the coordinate range of the workspace. In addition, in order to allow the twin robot to gradually learn a path with fewer steps, each step can also reduce a part of the score R for the twin robot. s The embodiment of the present application can greatly improve the overall learning efficiency by adding intermediate rewards and action rewards to the reward function.

[0055] In an embodiment of the present application, optionally, the "identifying the current position coordinates of the object to be grasped, the obstacle coordinates and the target position coordinates of the object to be grasped from the real working environment image" in step 101 includes: performing color space transformation processing on the real working environment image to obtain a target environment image, wherein the target environment image is represented based on a hue channel, a saturation channel and a lightness channel; extracting a channel sub-image corresponding to the saturation channel from the target environment image, and binarizing the channel sub-image based on a first preset saturation threshold to obtain a first sub-image containing only the object to be grasped, and binarizing the channel sub-image based on a second preset saturation threshold to obtain a second sub-image containing only the obstacle, and binarizing the channel sub-image based on a third preset saturation threshold. The channel sub-image is binarized to obtain a third sub-image that only contains the target position of the object to be grasped; the first sub-image, the second sub-image and the third sub-image are denoised respectively, and template matching is performed on the first sub-image, the second sub-image and the third sub-image after the denoising respectively, and according to the template matching result, the current position coordinates of the object to be grasped in the first sub-image, the obstacle coordinates of the obstacle in the second sub-image and the target position coordinates of the target position of the object to be grasped in the third sub-image are determined; the current position coordinates of the object to be grasped, the obstacle coordinates and the target position coordinates are converted by a preset coordinate conversion model to obtain the current position coordinates, obstacle coordinates and target position coordinates of the object to be grasped in the coordinate system of the multi-axis parallel robot.

[0056] In this embodiment, the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped can be identified from the real working environment image in the following manner. Specifically, first, the image of the real working environment is converted from the original color space (such as RGB) to another color space (such as HSV), so that the real working environment image becomes an image represented by a hue channel, a saturation channel, and a value channel, thereby obtaining a target environment image. This conversion is helpful for subsequent image analysis based on color characteristics. Then, the channel sub-image corresponding to the saturation channel is extracted from the converted target environment image. Afterwards, three different preset saturation thresholds (respectively, a first preset saturation threshold, a second preset saturation threshold, and a third preset saturation threshold) are used to binarize the above channel sub-image. These thresholds are determined based on the distinction between the objects to be grasped, obstacles, and target positions in saturation. Among them, the first sub-image obtained after processing according to the first preset saturation threshold only contains the items to be grasped; the second sub-image obtained after processing according to the second preset saturation threshold only contains obstacles; the third sub-image obtained after processing according to the third preset saturation threshold only contains the target position of the item to be grasped. Specifically, in order to be able to extract the items to be grasped, obstacles and target positions from the channel sub-image respectively, the items to be grasped, obstacles and target positions can be set in advance to colors that can be distinguished by saturation in the channel sub-image, so as to achieve rapid extraction. After obtaining the first sub-image, the second sub-image and the third sub-image, the three sub-images are denoised respectively to reduce noise interference in the image and improve the accuracy of subsequent analysis.

[0057] Next, template matching is performed on each denoised sub-image. Template matching is an image processing technique used to find parts in an image that are similar to a known template. Here, different templates can be used for different sub-images. For example, for the first sub-image, a template of a specific shape representing the object to be grasped can be used; for the second sub-image, a template of a specific shape representing the obstacle can be used; and for the third sub-image, a template of a specific shape representing the target position can be used. Furthermore, based on the results of template matching, the current position coordinates of the object to be grasped, the coordinates of the obstacles, and the target position coordinates of the object to be grasped can be determined in the real working environment image.

[0058] Finally, the position coordinates are converted from the image coordinate system (i.e., the camera coordinate system) to the coordinate system of the multi-axis parallel robot through a preset coordinate conversion model, because the robot needs to perform the grasping operation based on its own coordinate system. The converted coordinates can accurately guide the actions of the multi-axis parallel robot.

[0059] like Figure 2As shown in FIG. 1 , a method for extracting the current position coordinates and obstacle coordinates of the object to be grasped is given. It should be noted that Figure 2 This is just an example. In order to briefly explain the above coordinate extraction method, only the objects to be grasped and obstacles are shown in the real working environment image. Figure 2 In the figure, rectangles represent obstacles and stars represent objects to be grasped. First, obtain the real working environment image, that is, Figure 2 Original Image, then the real working environment image is converted into color space to obtain the target environment image represented by the HSV channel. Next, the channel sub-image under the S (saturation) channel is extracted from the target environment image, and then binarization is performed on it to obtain the first sub-image (a sub-image containing only star-shaped items to be grasped) and the second sub-image (a sub-image containing only rectangular obstacles). De-noising is performed on the first sub-image and the second sub-image respectively to obtain the first sub-image and the second sub-image after denoising. Then, template matching is performed on these two sub-images respectively, so that the current position coordinates of the object to be grasped in the camera coordinate system can be identified from the first sub-image after denoising, and the position coordinates of the obstacle in the camera coordinate system can be identified from the second sub-image after denoising. Subsequently, the preset coordinate transformation model is used for coordinate transformation (Transmission), and the current position coordinates of the object to be grasped in the camera coordinate system and the position coordinates of the obstacle in the camera coordinate system are converted into coordinates in the multi-axis parallel robot coordinate system, and input into the PLC system for subsequent calculations.

[0060] The embodiment of the present application can quickly and accurately identify the current position coordinates, obstacle coordinates and target position coordinates of the object to be grasped from the real working environment image, and convert these coordinates into coordinates in the coordinate system of the multi-axis parallel robot through a preset coordinate conversion model. The embodiment of the present application converts the coordinate information in the image into an operation instruction that the multi-axis parallel robot can understand and execute, which is crucial for achieving tasks such as automated grasping, avoiding obstacles and accurately placing objects.

[0061] In the embodiment of the present application, optionally, the method further includes: solving the rotation coefficient and the displacement coefficient in the initial coordinate transformation model by a nine-point calibration method, wherein the formula of the initial coordinate transformation model is as follows:

[0062]

[0063] in, represents the coordinates in the multi-axis parallel robot coordinate system, represents the rotation coefficient, represents the coordinates in the camera coordinate system, represents the displacement coefficient;

[0064] The solved rotation coefficient and displacement coefficient are substituted into the initial coordinate transformation model to obtain the preset coordinate transformation model.

[0065] In this embodiment, the nine-point calibration method is a method for determining the conversion relationship between two different coordinate systems, especially in a robot vision system, for converting the image coordinates captured by the camera into coordinates in the operating space of a multi-axis parallel robot. This method solves the conversion parameters, including the rotation coefficient and the displacement coefficient, through the corresponding positions of a series of known points in the two coordinate systems, so that the image coordinates can be converted into the coordinates in the multi-axis parallel robot coordinate system according to the determined rotation coefficient and displacement coefficient. The embodiment of the present application adopts the form of eye to hand, and the camera is placed on the upper part of the workbench.

[0066] First, the initial coordinate transformation model is established. Then, 9 known points are selected in the multi-axis parallel robot operation space. These points can be feature points that are easy to identify and locate in the camera image, and their positions in the multi-axis parallel robot coordinate system are also known. For each calibration point, its coordinates in the camera coordinate system P_camera are recorded, and at the same time, the coordinates of these points in the multi-axis parallel robot coordinate system P_robot are recorded.

[0067] For each known point, a set of equations is constructed using the initial coordinate transformation model formula. Since each point has two coordinate components (x, y), 9 points will generate 18 equations. After that, the least squares method or other optimization algorithm is used to solve the above set of equations to obtain the best estimates of the rotation coefficients and displacement coefficients. Next, the best estimates of the rotation coefficients and displacement coefficients are substituted into the initial coordinate transformation model, and the preset coordinate transformation model can be obtained. The preset coordinate transformation model can be used to convert the coordinates in the camera coordinate system to the coordinates in the multi-axis parallel robot coordinate system.

[0068] In fact, there are 6 unknown parameters in the initial coordinate transformation model, namely 4 unknowns a, b, a′, b′ in the rotation coefficients, and 2 unknowns c, c′ in the displacement coefficients. Therefore, 6 coordinate points are needed to complete the solution. However, in order to reduce errors, the embodiment of the present application chooses to use 9 points for camera calibration. This method is simple and effective, and can also improve the accuracy of the solution.

[0069] In an embodiment of the present application, optionally, after the "sending the target obstacle avoidance path to the control device of the physical robot" in step 104, the method also includes: receiving target data returned by the physical robot in real time, wherein the target data includes the current working environment image collected by the visual perception device and the working parameters collected by the sensor carried by the physical robot; based on the current working environment image and the working parameters, controlling the twin robot to follow the physical robot, and judging the path deviation between the following path of the twin robot and the target obstacle avoidance path during the following process; when the path deviation exceeds a preset deviation threshold, sending a return instruction to the physical robot, and retraining the twin robot using a reinforcement learning algorithm according to the virtual working environment and the path deviation to obtain a new target obstacle avoidance path; and resending the new target obstacle avoidance path to the control device of the physical robot.

[0070] In this embodiment, when the physical robot runs according to the target obstacle avoidance path, the physical robot can also send target data to the twin robot. Here, the physical robot can be equipped with a visual perception device and a sensor (such as a speed sensor, an acceleration sensor, an angle sensor, etc.), wherein the visual perception device can collect the current working environment image of the physical robot in real time, and the sensor can collect the working parameters of the physical robot in real time, such as the position of each axis, the robot speed, acceleration, the position of each joint axis, etc.

[0071] Next, the received target data is used to control the twin robot to follow the action of the physical robot in the virtual working environment, which is used to simulate the action of the physical robot in the real working environment. During the following process, the walking path of the twin robot in the virtual environment can be continuously compared with the preset target obstacle avoidance path (that is, the optimal path that the physical robot is expected to follow to avoid obstacles and complete the task efficiently), and the path offset between the walking path of the twin robot and the target obstacle avoidance path can be calculated and monitored to evaluate the difference between the actual execution and the expected planning. When the path offset is detected to exceed the preset offset threshold, it indicates that the actual movement of the physical robot has a large deviation from the expected path, which may be due to unforeseen obstacles or environmental changes. At this time, a return instruction is sent to the physical robot, requiring it to return to a safe position to avoid further deviation, and to start again from the origin later, so as to facilitate the verification of the effect of the new target obstacle avoidance path. At the same time, the current virtual working environment can be updated according to the received target data, and the twin robot can be retrained using the reinforcement learning algorithm according to the updated virtual working environment and path offset.

[0072] After retraining, the twin robot can learn a new obstacle avoidance path that is more suitable for the current working environment, that is, a new target obstacle avoidance path. Finally, the new target obstacle avoidance path is sent back to the control device of the physical robot so that the physical robot can navigate and avoid obstacles more accurately and effectively according to the updated path.

[0073] The embodiment of the present application forms a closed-loop feedback system, which continuously improves the obstacle avoidance capability of the physical robot by continuously collecting target data from the physical robot, simulating and optimizing in a virtual environment, and then applying the optimization results to the physical robot. This method not only improves the adaptability and intelligence level of the robot, but also reduces the number of expensive and time-consuming experiments in the actual environment.

[0074] In an embodiment of the present application, optionally, the reinforcement learning algorithm includes a DQN algorithm; before step 103, the method further includes: building multiple policy networks through the DQN algorithm, wherein the multiple policy networks are single-layer neural networks, and the number of neurons contained in each policy network is different; accordingly, step 103 includes: according to the virtual working environment, using the DQN algorithm to train each policy network, determine the obstacle avoidance path with the maximum reward value corresponding to each policy network, and record the number of actions and the maximum reward value corresponding to the obstacle avoidance path; according to the number of actions and the maximum reward value corresponding to each obstacle avoidance path, calculate the score corresponding to the obstacle avoidance path, and use the obstacle avoidance path with the highest score as the target obstacle avoidance path of the physical robot.

[0075] In this embodiment, the reinforcement learning algorithm may be a DQN (Deep Q-Network) algorithm, that is, a deep neural network is used to approximate the action function Q(s, a) based on the Q-learning algorithm. Specifically, the target obstacle avoidance path may be obtained through the DQN algorithm.

[0076] In one embodiment, a policy network can be built through the DQN algorithm, and then the policy network can be trained in a virtual working environment to obtain an obstacle avoidance path corresponding to the maximum reward value, and the obstacle avoidance path under the maximum reward value can be directly used as the above-mentioned target obstacle avoidance path.

[0077] In another embodiment, multiple policy networks are first built using the DQN algorithm. These policy networks are all single-layer neural networks, but each policy network contains a different number of neurons. In this way, the impact of different network structures (i.e., different numbers of neurons) on policy learning can be explored to find the network structure with the best performance.

[0078] Afterwards, in a virtual working environment, each policy network is trained using the DQN algorithm. During the training process, each policy network can try different actions (i.e., angle transformations of different axes) and find the obstacle avoidance path with the maximum reward value corresponding to it through training. After finding the obstacle avoidance path with the maximum reward value, the number of actions corresponding to the path (i.e., the minimum number of actions required to complete the path) and the maximum reward value are recorded. This information is used to subsequently evaluate the pros and cons of the obstacle avoidance paths obtained under different policy networks. Next, a score is calculated based on the number of actions and the maximum reward value corresponding to each obstacle avoidance path, aiming to measure the efficiency and effect of the obstacle avoidance path. Among them, it can be set that the more actions, the lower the score, and the larger the maximum reward value, the higher the score. Specifically, it can be calculated using a preset score calculation formula. The score calculation formula does not make special requirements, and only needs to meet the requirements that the more actions, the lower the score, and the larger the maximum reward value, the higher the score. After obtaining the score corresponding to the obstacle avoidance path under each policy network, the obstacle avoidance path with the highest score is used as the target obstacle avoidance path of the physical robot. At this time, the path is optimal in terms of obstacle avoidance effect, action efficiency, and reward value.

[0079] like Figure 3 As shown, the training results of a different strategy network provided by an embodiment of the present application are shown. Among them, when using the DQN reinforcement learning algorithm for training, it is found that the efficiency of structural learning using a single-layer neural network is higher. Therefore, in the present application, when using the DQN reinforcement learning algorithm to build a strategy network, a single-layer neuron architecture is adopted, and different strategy networks are built by changing the number of neurons. Assuming that the length and width of the current obstacle are detected through the real working environment image, the spatial coordinates occupied by the current rectangular obstacle can be obtained (the height is assumed to be known to be 30mm), and the target position coordinates and the current position coordinates of the item to be grasped are detected at the same time. After obtaining the above information, training begins, and the results are as follows Figure 3 As shown in the figure, it is found that when the number of neurons is 1024, the convergence speed is relatively fast, and when the number of neurons is 128, 256, and 512, there is also good convergence and training effect. When there are 2048 neurons, it is difficult to converge to the maximum reward when the number of training iterations is 2000. Figure 3 In the figure, the horizontal axis is the number of iterations (Epsiode), and the vertical axis is the training reward (Training Reward). Table 1 shows the number of actions contained in the obstacle avoidance paths trained by different strategy networks, as well as the maximum reward value of the optimal path. When there are 1024 and 128 neurons, the number of actions contained in the obstacle avoidance paths trained is at least 48.

[0080] Table 1 The optimal number of obstacle avoidance path actions and the maximum reward value obtained under different numbers of neurons

[0081] Number of neurons The number of optimal obstacle avoidance path actions Maximum reward value 128 48 99555.86 256 50 99491.85 512 51 99560.47 1024 48 99536.86

[0082] like Figure 4 As shown, the obstacle avoidance paths and three-dimensional schematic diagrams of obstacle positions planned under different strategy networks (strategy networks corresponding to 128, 256, 512, and 1024 neurons) are drawn at the same time. It can be found that the line represented by red is the optimal route trained under obstacle-free conditions. It has obvious collisions with obstacles, while the other four obstacle avoidance paths obtained based on the real working environment images all avoid obstacles and reach the target position. It can be seen that the above method of the embodiment of the present application is feasible.

[0083] The embodiment of the present application builds multiple policy networks with different structures and trains them in a virtual working environment to find the optimal obstacle avoidance path. The performance of different policy networks is compared based on the scores, and the obstacle avoidance path that best suits the current task is finally selected as the target obstacle avoidance path of the physical robot. This method greatly improves the accuracy and efficiency of the target obstacle avoidance path.

[0084] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides a robot obstacle avoidance path planning system, such as Figure 5 As shown, the system includes:

[0085] A coordinate recognition module, used to receive a real working environment image sent by the physical robot, and recognize the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped from the real working environment image, wherein the real working environment image is acquired by a visual perception device carried by the physical robot;

[0086] An environment construction module, used to construct a virtual working environment of the twin robot corresponding to the physical robot according to the current position coordinates of the object to be grasped, the obstacle coordinates and the target position coordinates of the object to be grasped;

[0087] A training module, configured to train the twin robot using a reinforcement learning algorithm according to the virtual working environment to obtain an obstacle avoidance path with a maximum reward value, and determine a target obstacle avoidance path of the physical robot based on the obstacle avoidance path with a maximum reward value;

[0088] The path sending module is used to send the target obstacle avoidance path to the control device of the physical robot, so as to control the physical robot through the control device to move the object to be grasped from the current position to the target position based on the target obstacle avoidance path.

[0089] Optionally, the robot is a multi-axis parallel robot; and the training module is further used for:

[0090] Before the twin robot is trained by the reinforcement learning algorithm, an action space of the reinforcement learning algorithm is defined, wherein each action in the action space is used to indicate a rotation angle change of each axis of the multi-axis parallel robot at a current position;

[0091] Define a state space of a reinforcement learning algorithm, wherein the state representation in the state space is obtained based on the current angles of each axis of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform, and the current three-dimensional coordinates of the center point of the moving platform are obtained based on the forward kinematics modeling of the twin robot;

[0092] Define a reward function of the reinforcement learning algorithm, wherein the reward function consists of a sparse reward, an intermediate reward and an action reward, the sparse reward is determined based on the current position coordinates of the end of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform, the intermediate reward is determined based on the starting three-dimensional coordinates, the current three-dimensional coordinates and the end three-dimensional coordinates of the center point of the moving platform of the multi-axis parallel robot, and the action reward is determined based on the number of actions of the twin robot;

[0093] Accordingly, the training module is also used for:

[0094] According to the virtual working environment, and the action space, state space and reward function of the defined reinforcement learning algorithm, the twin robot is trained to obtain an obstacle avoidance path with a maximum reward value.

[0095] Optionally, the calculation formula of the sparse reward is as follows:

[0096]

[0097] Among them, R sp Represents a sparse reward for the current action; when the current position coordinates of the end of the multi-axis parallel robot coincide with the target position coordinates, it is determined that the end of the multi-axis parallel robot has reached the target position, and the current position coordinates of the end of the multi-axis parallel robot are determined based on the current position coordinates of the center point of the moving platform; when the current position coordinates of the center point of the moving platform of the multi-axis parallel robot intersect with the obstacle coordinate range, it is determined that the moving platform of the multi-axis parallel robot has collided with an obstacle; when the current position coordinates of the center point of the moving platform of the multi-axis parallel robot exceed the coordinate range corresponding to the preset workspace, it is determined that the moving platform of the multi-axis parallel robot exceeds the preset workspace;

[0098] The calculation formula of the intermediate reward is as follows:

[0099]

[0100] Among them, Rt represents the intermediate reward of the current action; x, y, z represent the current three-dimensional coordinates of the center point of the moving platform, x0, y0, z0 represent the starting three-dimensional coordinates of the center point of the moving platform, x d ,y d ,z d represents the three-dimensional coordinates of the end point of the center point of the moving platform, and ω is the weight value;

[0101] The calculation formula of the reward function is as follows:

[0102] R=R sp +R t +R s ;

[0103] Among them, R represents the final reward of the current action, R s (R s <0) indicates the action reward of the current action.

[0104] Optionally, the coordinate identification module is used to:

[0105] Performing color space transformation processing on the real working environment image to obtain a target environment image, wherein the target environment image is represented based on a hue channel, a saturation channel, and a lightness channel;

[0106] Extracting a channel sub-image corresponding to the saturation channel from the target environment image, and binarizing the channel sub-image based on a first preset saturation threshold to obtain a first sub-image containing only the object to be grasped, and binarizing the channel sub-image based on a second preset saturation threshold to obtain a second sub-image containing only the obstacle, and binarizing the channel sub-image based on a third preset saturation threshold to obtain a third sub-image containing only the target position of the object to be grasped;

[0107] Performing denoising processing on the first sub-image, the second sub-image, and the third sub-image respectively, and performing template matching on the first sub-image, the second sub-image, and the third sub-image after the denoising processing respectively, and determining the current position coordinates of the object to be grasped in the first sub-image, the obstacle coordinates of the obstacle in the second sub-image, and the target position coordinates of the target position of the object to be grasped in the third sub-image according to the template matching results;

[0108] The current position coordinates of the object to be grasped, the obstacle coordinates and the target position coordinates are transformed by a preset coordinate transformation model to obtain the current position coordinates of the object to be grasped, the obstacle coordinates and the target position coordinates in the coordinate system of the multi-axis parallel robot.

[0109] Optionally, the device further comprises a calibration module, wherein the calibration module is used to:

[0110] The rotation coefficient and displacement coefficient in the initial coordinate transformation model are solved by the nine-point calibration method, wherein the formula of the initial coordinate transformation model is as follows:

[0111]

[0112] in, represents the coordinates in the multi-axis parallel robot coordinate system, represents the rotation coefficient, represents the coordinates in the camera coordinate system, represents the displacement coefficient;

[0113] The solved rotation coefficient and displacement coefficient are substituted into the initial coordinate transformation model to obtain the preset coordinate transformation model.

[0114] Optionally, the device further comprises:

[0115] A data receiving module, configured to receive target data returned by the physical robot in real time after the target obstacle avoidance path is sent to the control device of the physical robot, wherein the target data includes the current working environment image collected by the visual perception device and the working parameters collected by the sensor carried by the physical robot;

[0116] An offset determination module, used to control the twin robot to follow the physical robot based on the current working environment image and the working parameters, and to determine the path offset between the following path of the twin robot and the target obstacle avoidance path during the following process;

[0117] The training module is further configured to send a return instruction to the physical robot when the path deviation exceeds a preset deviation threshold, and to retrain the twin robot using a reinforcement learning algorithm according to the virtual working environment and the path deviation to obtain a new target obstacle avoidance path;

[0118] The path sending module is also used to resend the new target obstacle avoidance path to the control device of the physical robot.

[0119] Optionally, the reinforcement learning algorithm includes a DQN algorithm; and the device further includes:

[0120] A network building module is used for training the twin robot according to the virtual working environment using a reinforcement learning algorithm to obtain an obstacle avoidance path with a maximum reward value, and then building multiple strategy networks using a DQN algorithm, wherein the multiple strategy networks are all single-layer neural networks, and the number of neurons contained in each strategy network is different;

[0121] Accordingly, the training module is also used for:

[0122] According to the virtual working environment, each strategy network is trained using the DQN algorithm to determine the obstacle avoidance path with the maximum reward value corresponding to each strategy network, and the number of actions and the maximum reward value corresponding to the obstacle avoidance path are recorded;

[0123] According to the number of actions and the maximum reward value corresponding to each obstacle avoidance path, the score corresponding to the obstacle avoidance path is calculated, and the obstacle avoidance path with the highest score is used as the target obstacle avoidance path of the physical robot.

[0124] It should be noted that for other corresponding descriptions of the functional units involved in the robot obstacle avoidance path planning system provided in the embodiment of the present application, reference can be made to Figures 1 to 4 The corresponding description in the method will not be repeated here.

[0125] The present application also provides a computer device, which may be a personal computer, a server, a network device, etc. Figure 6 As shown, the computer device includes a bus, a processor, a memory and a communication interface, and may also include an input and output interface and a display device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in each method embodiment are implemented.

[0126] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0127] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0128] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0129] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0130] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0131] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A robot obstacle avoidance path planning method, characterized in that: include: Receiving a real working environment image sent by a physical robot, and identifying the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped from the real working environment image, wherein the real working environment image is acquired by a visual perception device carried by the physical robot; Constructing a virtual working environment of the twin robot corresponding to the physical robot according to the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped; According to the virtual working environment, the twin robot is trained using a reinforcement learning algorithm to obtain an obstacle avoidance path with a maximum reward value, and a target obstacle avoidance path of the physical robot is determined based on the obstacle avoidance path with the maximum reward value; The target obstacle avoidance path is sent to the control device of the physical robot, so that the control device controls the physical robot to move the object to be grasped from the current position to the target position based on the target obstacle avoidance path.

2. The method according to claim 1, characterized in that The robot is a multi-axis parallel robot; before the twin robot is trained by using a reinforcement learning algorithm, the method further includes: Defining an action space of a reinforcement learning algorithm, wherein each action in the action space is used to indicate a rotation angle change of each axis of the multi-axis parallel robot at a current position; Define a state space of a reinforcement learning algorithm, wherein the state representation in the state space is obtained based on the current angles of each axis of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform, and the current three-dimensional coordinates of the center point of the moving platform are obtained based on the forward kinematics modeling of the twin robot; Define a reward function of the reinforcement learning algorithm, wherein the reward function consists of a sparse reward, an intermediate reward and an action reward, the sparse reward is determined based on the current position coordinates of the end of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform, the intermediate reward is determined based on the starting three-dimensional coordinates, the current three-dimensional coordinates and the end three-dimensional coordinates of the center point of the moving platform of the multi-axis parallel robot, and the action reward is determined based on the number of actions of the twin robot; Accordingly, the method of training the twin robot using a reinforcement learning algorithm according to the virtual working environment to obtain an obstacle avoidance path with a maximum reward value includes: According to the virtual working environment, and the action space, state space and reward function of the defined reinforcement learning algorithm, the twin robot is trained to obtain an obstacle avoidance path with a maximum reward value.

3. The method according to claim 2, characterized in that The calculation formula of the sparse reward is as follows: Among them, R sp Represents a sparse reward for the current action; when the current position coordinates of the end of the multi-axis parallel robot coincide with the target position coordinates, it is determined that the end of the multi-axis parallel robot has reached the target position, and the current position coordinates of the end of the multi-axis parallel robot are determined based on the current position coordinates of the center point of the moving platform; when the current position coordinates of the center point of the moving platform of the multi-axis parallel robot intersect with the obstacle coordinate range, it is determined that the moving platform of the multi-axis parallel robot has collided with an obstacle; when the current position coordinates of the center point of the moving platform of the multi-axis parallel robot exceed the coordinate range corresponding to the preset workspace, it is determined that the moving platform of the multi-axis parallel robot exceeds the preset workspace; The calculation formula of the intermediate reward is as follows: Among them, R t represents the intermediate reward of the current action; x, y, z represent the current three-dimensional coordinates of the center point of the moving platform, x0, y0, z0 represent the starting three-dimensional coordinates of the center point of the moving platform, d ,y d ,z d represents the three-dimensional coordinates of the end point of the center point of the moving platform, and ω is the weight value; The calculation formula of the reward function is as follows: R=R sp +R t +R s ; Among them, R represents the final reward of the current action, R s (R s <0) indicates the action reward of the current action.

4. The method according to claim 1, characterized in that The identifying the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped from the real working environment image includes: Performing color space transformation processing on the real working environment image to obtain a target environment image, wherein the target environment image is represented based on a hue channel, a saturation channel, and a lightness channel; Extracting a channel sub-image corresponding to the saturation channel from the target environment image, and binarizing the channel sub-image based on a first preset saturation threshold to obtain a first sub-image containing only the object to be grasped, and binarizing the channel sub-image based on a second preset saturation threshold to obtain a second sub-image containing only the obstacle, and binarizing the channel sub-image based on a third preset saturation threshold to obtain a third sub-image containing only the target position of the object to be grasped; Performing denoising processing on the first sub-image, the second sub-image, and the third sub-image respectively, and performing template matching on the first sub-image, the second sub-image, and the third sub-image after the denoising processing respectively, and determining the current position coordinates of the object to be grasped in the first sub-image, the obstacle coordinates of the obstacle in the second sub-image, and the target position coordinates of the target position of the object to be grasped in the third sub-image according to the template matching results; The current position coordinates of the object to be grasped, the obstacle coordinates and the target position coordinates are transformed by a preset coordinate transformation model to obtain the current position coordinates of the object to be grasped, the obstacle coordinates and the target position coordinates in the coordinate system of the multi-axis parallel robot.

5. The method according to claim 4, characterized in that The method further comprises: The rotation coefficient and displacement coefficient in the initial coordinate transformation model are solved by the nine-point calibration method, wherein the formula of the initial coordinate transformation model is as follows: in, represents the coordinates in the multi-axis parallel robot coordinate system, represents the rotation coefficient, Represents the coordinates in the camera coordinate system, represents the displacement coefficient; The solved rotation coefficient and displacement coefficient are substituted into the initial coordinate transformation model to obtain the preset coordinate transformation model.

6. The method according to claim 1, characterized in that After sending the target obstacle avoidance path to the control device of the physical robot, the method further includes: Receiving target data returned by the physical robot in real time, wherein the target data includes the current working environment image collected by the visual perception device and the working parameters collected by the sensor carried by the physical robot; Based on the current working environment image and the working parameters, controlling the twin robot to follow the physical robot, and determining the path deviation between the following path of the twin robot and the target obstacle avoidance path during the following process; When the path deviation exceeds a preset deviation threshold, a return instruction is sent to the physical robot, and the twin robot is retrained using a reinforcement learning algorithm according to the virtual working environment and the path deviation to obtain a new target obstacle avoidance path; The new target obstacle avoidance path is resent to the control device of the physical robot.

7. The method according to claim 1, characterized in that The reinforcement learning algorithm includes a DQN algorithm; before the twin robot is trained using the reinforcement learning algorithm according to the virtual working environment to obtain an obstacle avoidance path with a maximum reward value, the method further includes: Using the DQN algorithm, multiple strategy networks are constructed, wherein each of the strategy networks is a single-layer neural network, and each strategy network contains a different number of neurons; Accordingly, the twin robot is trained using a reinforcement learning algorithm according to the virtual working environment to obtain an obstacle avoidance path with a maximum reward value, and a target obstacle avoidance path of the physical robot is determined based on the obstacle avoidance path with a maximum reward value, including: According to the virtual working environment, each strategy network is trained using the DQN algorithm to determine the obstacle avoidance path with the maximum reward value corresponding to each strategy network, and the number of actions and the maximum reward value corresponding to the obstacle avoidance path are recorded; According to the number of actions and the maximum reward value corresponding to each obstacle avoidance path, the score corresponding to the obstacle avoidance path is calculated, and the obstacle avoidance path with the highest score is used as the target obstacle avoidance path of the physical robot.

8. A robot obstacle avoidance path planning system, characterized in that: include: A coordinate recognition module, used to receive a real working environment image sent by the physical robot, and recognize the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped from the real working environment image, wherein the real working environment image is acquired by a visual perception device carried by the physical robot; An environment construction module, used to construct a virtual working environment of the twin robot corresponding to the physical robot according to the current position coordinates of the object to be grasped, the obstacle coordinates and the target position coordinates of the object to be grasped; A training module, configured to train the twin robot using a reinforcement learning algorithm according to the virtual working environment to obtain an obstacle avoidance path with a maximum reward value, and determine a target obstacle avoidance path of the physical robot based on the obstacle avoidance path with a maximum reward value; The path sending module is used to send the target obstacle avoidance path to the control device of the physical robot, so as to control the physical robot through the control device to move the object to be grasped from the current position to the target position based on the target obstacle avoidance path.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images