Robot obstacle avoidance path planning method and system, storage medium, and computer equipment

Abstract

The present application discloses a robot obstacle avoidance path planning method and system, storage medium, and computer equipment, the method comprising: receiving a real working environment image sent by a physical robot, and identifying the current position coordinates, obstacle coordinates, and target position coordinates of an item to be grasped from the real working environment image; constructing a virtual working environment of a twin robot corresponding to the physical robot according to the current position coordinates, obstacle coordinates, and target position coordinates of the item to be grasped; training the twin robot using a reinforcement learning algorithm based on the virtual working environment to obtain an obstacle avoidance path with a maximum reward value, and determining a target obstacle avoidance path of the physical robot based on the obstacle avoidance path with a maximum reward value; and sending the target obstacle avoidance path to a control device of the physical robot, so as to control the physical robot through the control device to move the item to be grasped from the current position to the target position based on the target obstacle avoidance path.

Description

Technical Field

[0001] The present application relates to the field of industrial intelligent manufacturing technology, and in particular to a robot obstacle avoidance path planning method and system, storage medium, and computer equipment. Background Art

[0002] With the rapid development of automation technology, robots are increasingly being used in industrial production, warehousing, and logistics. In industrial production scenarios, robots often need to complete complex tasks, such as grasping and moving objects. Furthermore, during this process, obstacles may exist in the working environment. Therefore, robots need to autonomously navigate these obstructions, avoid collisions, and safely move objects to their destinations.

[0003] Traditional robot path planning methods often rely on predefined maps and fixed algorithms. However, the path accuracy ultimately obtained by this method is low. When the robot moves along the planned path, path deviations may cause the robot to face many unsafe factors, such as collisions with obstacles, which increases the robot's trial and error costs. Summary of the Invention

[0004] In view of this, the present application provides a robot obstacle avoidance path planning method and system, storage medium, and computer equipment. Through the reinforcement learning algorithm, the twin robots are allowed to continuously trial and error in a virtual working environment, and finally obtain the target obstacle avoidance path, which can greatly improve the accuracy of the obstacle avoidance path. At the same time, it can reduce the trial and error cost of the physical robot, avoid the unsafe factors caused by directly operating the physical robot, and reduce the loss of the physical robot.

[0005] According to one aspect of the present application, a robot obstacle avoidance path planning method is provided, comprising:

[0006] Receiving a real working environment image transmitted by a physical robot, and identifying the current position coordinates of an object to be grasped, obstacle coordinates, and target position coordinates of the object to be grasped from the real working environment image, wherein the real working environment image is acquired by a visual perception device carried by the physical robot;

[0007] Constructing a virtual working environment of the twin robot corresponding to the physical robot according to the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped;

[0008] According to the virtual working environment, the twin robot is trained using a reinforcement learning algorithm to obtain an obstacle avoidance path with a maximum reward value, and a target obstacle avoidance path of the physical robot is determined based on the obstacle avoidance path with the maximum reward value;

[0009] The target obstacle avoidance path is sent to the control device of the physical robot, so that the control device controls the physical robot to move the object to be grasped from the current position to the target position based on the target obstacle avoidance path.

[0010] According to another aspect of the present application, a robot obstacle avoidance path planning system is provided, comprising:

[0011] a coordinate recognition module, configured to receive an image of a real working environment transmitted by the physical robot and identify, from the image, the current position coordinates of an object to be grasped, the coordinates of obstacles, and the coordinates of a target position of the object to be grasped, wherein the image of the real working environment is acquired by a visual perception device carried by the physical robot;

[0012] An environment construction module, configured to construct a virtual working environment of the twin robot corresponding to the physical robot according to the current position coordinates of the object to be grasped, the coordinates of the obstacles, and the target position coordinates of the object to be grasped;

[0013] A training module is configured to train the twin robot using a reinforcement learning algorithm according to the virtual working environment to obtain an obstacle avoidance path with a maximum reward value, and determine a target obstacle avoidance path for the physical robot based on the obstacle avoidance path with a maximum reward value;

[0014] A path sending module is used to send the target obstacle avoidance path to the control device of the physical robot, so that the physical robot is controlled by the control device to move the object to be grasped from the current position to the target position based on the target obstacle avoidance path.

[0015] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the above-mentioned robot obstacle avoidance path planning method is implemented.

[0016] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned robot obstacle avoidance path planning method when executing the program.

[0017] By means of the above technical solution, the present application provides a robot obstacle avoidance path planning method and system, storage medium, and computer equipment. After receiving an image of a real working environment, the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped can be identified from the real working environment image. Afterwards, based on the identified current position coordinates, obstacle coordinates, and target position coordinates of the object to be grasped, a virtual working environment of a twin robot corresponding to the physical robot can be constructed. Then, in the virtual working environment, the twin robot is trained using a reinforcement learning algorithm. Through continuous iterative learning, the twin robot can eventually find an obstacle avoidance path that obtains the maximum reward value. Then, based on the obstacle avoidance path, the target obstacle avoidance path of the physical robot, i.e., the optimal path, can be determined. After obtaining the target obstacle avoidance path, the information of the target obstacle avoidance path is sent to the control device of the physical robot. The control device controls the physical robot to move along the planned path based on the received information of the target obstacle avoidance path, thereby moving the object to be grasped from the current position to the target position. The embodiment of the present application uses a reinforcement learning algorithm to allow the twin robots to continuously trial and error in a virtual working environment, and finally obtain the target obstacle avoidance path, which can greatly improve the accuracy of the obstacle avoidance path. At the same time, it can reduce the trial and error cost of the physical robot, avoid the unsafe factors caused by directly operating the physical robot, and reduce the loss of the physical robot.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A schematic diagram of a process for a robot obstacle avoidance path planning method provided in an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram showing a method for extracting the current position coordinates of an object to be grasped and the coordinates of obstacles provided in an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram showing the training results of a different strategy network provided in an embodiment of the present application;

[0023] Figure 4A three-dimensional schematic diagram showing the optimal obstacle avoidance path and obstacle location planned under different strategy networks provided in an embodiment of the present application is shown;

[0024] Figure 5 A schematic structural diagram of a robot obstacle avoidance path planning system provided in an embodiment of the present application is shown;

[0025] Figure 6 A schematic diagram of the device structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0027] In this embodiment, a robot obstacle avoidance path planning method is provided. Figure 1 As shown, the method includes:

[0028] Step 101: Receive a real working environment image sent by a physical robot, and identify the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped from the real working environment image, wherein the real working environment image is collected by a visual perception device carried by the physical robot.

[0029] A robot obstacle avoidance path planning method provided in an embodiment of the present application combines visual perception, digital twin, and reinforcement learning technology to plan the obstacle avoidance path of the robot operation, aiming to help the physical robot complete the task of grasping and carrying objects efficiently and safely in a complex working environment, while avoiding collisions with obstacles. Among them, the physical robot is equipped with a visual perception device, and the real working environment image of the physical robot can be collected by the visual perception device (such as a camera). After the visual perception device collects the real working environment image, the real working environment image can be sent to the robot obstacle avoidance path planning system through the physical robot. After the robot obstacle avoidance path planning system receives the real working environment image, it can use image processing technology (such as computer vision algorithm) to identify the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped from the real working environment image. This information is the basis for subsequent obstacle avoidance path planning.

[0030] Step 102: construct a virtual working environment of the twin robot corresponding to the physical robot based on the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped.

[0031] In this embodiment, based on the identified current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped, a virtual working environment of a twin robot corresponding to the physical robot can be constructed. This virtual working environment corresponds to the real working environment of the physical robot and is a digital replica of the real working environment of the physical robot, containing all key physical features, such as the position of the object to be grasped, the distribution of obstacles, etc. The application of digital twin technology allows simulation experiments and path planning in a virtual working environment without interfering with the actual operation of the physical robot, thereby improving the efficiency and safety of planning. It should be noted that the twin robot in the embodiment of the present application is completely constructed based on the physical robot, and its operation in the virtual working environment can simulate the operation of the physical robot in the real working environment.

[0032] Step 103: According to the virtual working environment, the twin robot is trained using a reinforcement learning algorithm to obtain an obstacle avoidance path with a maximum reward value, and the target obstacle avoidance path of the physical robot is determined based on the obstacle avoidance path with a maximum reward value.

[0033] In this embodiment, the twin robots are then trained in a virtual working environment using a reinforcement learning algorithm. Reinforcement learning is a machine learning method that allows the twin robots to learn how to achieve goals while trying different behaviors, while taking into account rewards (positive feedback) and punishments (negative feedback) to optimize their behavioral strategies. In this process, the twin robots can try a variety of possible paths, and after each attempt, they will receive a corresponding reward value based on whether they successfully avoid obstacles, path length, time efficiency and other factors. Through continuous iterative learning, the twin robots will eventually be able to find an obstacle avoidance path that obtains the maximum reward value, and then the target obstacle avoidance path of the physical robot, that is, the optimal path, can be determined based on the obstacle avoidance path.

[0034] Step 104 : Send the target obstacle avoidance path to the control device of the physical robot, so that the control device controls the physical robot to move the object to be grasped from the current position to the target position based on the target obstacle avoidance path.

[0035] In this embodiment, the information of the target obstacle avoidance path is sent to the control device of the physical robot. The control device controls the physical robot to move along the planned path based on the received information of the target obstacle avoidance path, thereby safely and efficiently moving the object to be grasped from the current position to the target position.

[0036] By applying the technical solution of this embodiment, after receiving the real working environment image, the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped can be identified from the real working environment image. Afterwards, based on the identified current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped, a virtual working environment of a twin robot corresponding to the physical robot can be constructed. Then, in the virtual working environment, the twin robot is trained using a reinforcement learning algorithm. Through continuous iterative learning, the twin robot can eventually find an obstacle avoidance path that obtains the maximum reward value, and then the target obstacle avoidance path of the physical robot, that is, the optimal path, can be determined based on the obstacle avoidance path. After obtaining the target obstacle avoidance path, the information of the target obstacle avoidance path is sent to the control device of the physical robot. The control device controls the physical robot to move along the planned path based on the received information of the target obstacle avoidance path, thereby moving the object to be grasped from the current position to the target position. The embodiment of the present application uses a reinforcement learning algorithm to allow the twin robots to continuously trial and error in a virtual working environment, and finally obtain the target obstacle avoidance path, which can greatly improve the accuracy of the obstacle avoidance path. At the same time, it can reduce the trial and error cost of the physical robot, avoid the unsafe factors caused by directly operating the physical robot, and reduce the loss of the physical robot.

[0037] In an embodiment of the present application, optionally, the robot is a multi-axis parallel robot; before "using a reinforcement learning algorithm to train the twin robot" in step 103, the method further includes: defining an action space of the reinforcement learning algorithm, wherein each action in the action space is used to indicate a rotation angle change of each axis of the multi-axis parallel robot at the current position; defining a state space of the reinforcement learning algorithm, wherein the state representation in the state space is based on the current angles of each axis of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform, and the current three-dimensional coordinates of the center point of the moving platform are obtained based on the forward kinematic modeling solution of the twin robot; defining a reward function of the reinforcement learning algorithm, wherein the reward function is composed of The multi-axis parallel robot comprises a sparse reward, an intermediate reward and an action reward, wherein the sparse reward is determined based on the current position coordinates of the end of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform, the intermediate reward is determined based on the starting three-dimensional coordinates, the current three-dimensional coordinates and the end three-dimensional coordinates of the center point of the moving platform of the multi-axis parallel robot, and the action reward is determined based on the number of actions of the twin robot; accordingly, the step 103 of "training the twin robot using the reinforcement learning algorithm according to the virtual working environment to obtain an obstacle avoidance path with a maximum reward value" includes: training the twin robot according to the virtual working environment and the action space, state space and reward function of the defined reinforcement learning algorithm to obtain an obstacle avoidance path with a maximum reward value.

[0038] In this embodiment, the robot may be a multi-axis parallel robot. Before using the reinforcement learning algorithm to train the twin robots, the action space, state space, and reward function of the reinforcement learning algorithm may be defined first.

[0039] (1) Definition of action space. The action space refers to the set of all possible actions that the multi-axis parallel robot can perform at each decision point. Each action represents a change in the rotation angle of each axis of the multi-axis parallel robot at the current position. That is, the multi-axis parallel robot can change its overall position and posture by adjusting the angle of each axis. The definition of the action space is the basis of the reinforcement learning algorithm because it determines what actions the multi-axis parallel robot can take to explore the environment. For example, the multi-axis parallel robot mentioned above is specifically a three-axis parallel robot with three degrees of freedom. Assuming that any axis of the three-axis parallel robot can rotate 1 degree, 0 degree, or -1 degree at the current position, then when an action is represented as [0, 1, 1], it means that the three-axis parallel robot is at the current position with axis 0 remaining stationary, axis 1 rotating 1 degree, and axis 2 rotating 1 degree. Therefore, according to this rule, the action space is set up, and each axis has three cases, and the dimension of the action space is 27.

[0040] (2) Definition of state space. The state representation in the state space is actually determined by the current angles of the axes of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform. The definition of the state space enables the reinforcement learning algorithm to understand the current angles of the axes of the multi-axis parallel robot and the current position of the center point of the moving platform, so as to make decisions. For example, the multi-axis parallel robot is a three-axis parallel robot. The current angles of the three axes can be expressed as n0, n1, n2. The state representation in the state space is constructed by the current angles of the axes and the current three-dimensional coordinates (x, y, z) of the center point of the moving platform obtained by solving the positive kinematics of the three-axis parallel robot. The state representation s is obtained as s = [n0, n1, n2, x, y, z]. Here, the current three-dimensional coordinates of the center point of the moving platform are obtained by solving the positive kinematics model of the twin robot.

[0041] Specifically, the forward kinematic solution of a multi-axis parallel robot involves solving the kinematic equations for the three-dimensional coordinates of the moving platform's center point relative to the stationary platform's coordinate system, given the known angle of the active arm relative to the stationary platform. Using known data such as the angle of the active arm relative to the stationary platform, the current three-dimensional coordinates of the moving platform's center point can be calculated.

[0042] (3) Definition of reward function. The reward function is the core component of the reinforcement learning algorithm, which defines the immediate reward obtained by the multi-axis parallel robot after performing a certain action. In the embodiment of the present application, the reward function consists of three parts: sparse reward, intermediate reward and action reward. Among them, the sparse reward is determined based on the current position coordinates of the end of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform. This reward is usually given when the robot reaches the target position (that is, the position where the object to be grasped is finally intended to reach), so it is sparse (not every action has a reward). The intermediate reward is determined based on the starting three-dimensional coordinates, current three-dimensional coordinates and end three-dimensional coordinates of the center point of the moving platform. This reward encourages the robot to move towards the target position even if it does not reach the target position directly. Among them, the starting three-dimensional coordinates are the three-dimensional coordinates of the center point of the moving platform when the twin robot is at the starting position, which are known; the end three-dimensional coordinates are the three-dimensional coordinates of the center point of the moving platform when the end of the twin robot reaches the target position, which can be calculated based on the target position coordinates. The action reward is determined based on the number of actions of the twin robot, and is used to encourage the robot to complete the task with fewer actions, thereby improving efficiency.

[0043] After defining the action space, state space, and reward function, the twin robots are trained using the virtual work environment and the defined action space, state space, and reward function. The goal of training is to find an obstacle-avoiding path that maximizes the cumulative reward. The reinforcement learning algorithm continuously tries different action sequences and updates its strategy based on the rewards provided by the reward function, ultimately finding the obstacle-avoiding path that maximizes the reward.

[0044] The embodiment of the present application defines the action space, state space and reward function, and uses the reinforcement learning algorithm to train the twin robots to find an optimal path to avoid obstacles in a virtual environment, which can greatly improve the autonomous navigation capability of the multi-axis parallel robot in a complex environment.

[0045] In the embodiment of the present application, optionally, the calculation formula of the sparse reward is as follows:

[0046]

[0047] Among them, R spRepresents a sparse reward for the current action; when the current position coordinates of the end of the multi-axis parallel robot coincide with the target position coordinates, it is determined that the end of the multi-axis parallel robot has reached the target position, and the current position coordinates of the end of the multi-axis parallel robot are determined based on the current position coordinates of the center point of the moving platform; when the current position coordinates of the center point of the moving platform of the multi-axis parallel robot intersect with the obstacle coordinate range, it is determined that the moving platform of the multi-axis parallel robot has collided with an obstacle; when the current position coordinates of the center point of the moving platform of the multi-axis parallel robot exceed the coordinate range corresponding to the preset workspace, it is determined that the moving platform of the multi-axis parallel robot exceeds the preset workspace;

[0048] The calculation formula for the intermediate reward is as follows:

[0049]

[0050] Among them, R t represents the intermediate reward of the current action; x, y, z represent the current three-dimensional coordinates of the center point of the moving platform, x0, y0, z0 represent the starting three-dimensional coordinates of the center point of the moving platform, x d ,y d ,z d represents the three-dimensional coordinates of the end point of the center point of the moving platform, and ω is the weight value;

[0051] The calculation formula of the reward function is as follows:

[0052] R=R sp +R t +R s ;

[0053] Among them, R represents the final reward of the current action, R s (R s <0) indicates the action reward of the current action.

[0054] In this embodiment, it is found in the experiment that the training efficiency of only setting sparse rewards is very low and far from meeting the requirements of practical application. Therefore, intermediate rewards are added during the exploration process of the twin robots to guide the twin robots to approach the target position. a 、r b 、r cAnd ω can be determined based on empirical values. In the physical robot, since the position between the end of the multi-axis parallel robot and the center point of the moving platform is relatively fixed, after determining the current three-dimensional coordinates of the center point of the moving platform, the current position coordinates of the end of the multi-axis parallel robot can be calculated based on the current three-dimensional coordinates of the center point of the moving platform. Here, the end of the multi-axis parallel robot can be the grasping end of the object to be grasped. The preset workspace can be a space pre-set by the staff, which can be achieved by inputting the coordinate range of the workspace. In addition, in order to allow the twin robot to gradually learn a path with fewer steps, a part of the score R can be subtracted from the twin robot for each step. s The embodiment of the present application can greatly improve the overall learning efficiency by adding intermediate rewards and action rewards to the reward function.

[0055] In an embodiment of the present application, optionally, the "identifying the current position coordinates of the item to be grasped, the obstacle coordinates and the target position coordinates of the item to be grasped from the real working environment image" in step 101 includes: performing color space transformation processing on the real working environment image to obtain a target environment image, wherein the target environment image is represented based on a hue channel, a saturation channel and a lightness channel; extracting a channel sub-image corresponding to the saturation channel from the target environment image, and binarizing the channel sub-image based on a first preset saturation threshold to obtain a first sub-image containing only the item to be grasped, and binarizing the channel sub-image based on a second preset saturation threshold to obtain a second sub-image containing only the obstacle, and binarizing the channel sub-image based on a third preset saturation threshold to obtain a second sub-image containing only the obstacle. The channel sub-image is binarized to obtain a third sub-image that only contains the target position of the object to be grasped; the first sub-image, the second sub-image and the third sub-image are denoised respectively, and template matching is performed on the first sub-image, the second sub-image and the third sub-image after denoising respectively, and according to the template matching results, the current position coordinates of the object to be grasped in the first sub-image, the obstacle coordinates of the obstacle in the second sub-image and the target position coordinates of the target position of the object to be grasped in the third sub-image are determined; the current position coordinates of the object to be grasped, the obstacle coordinates and the target position coordinates are converted by a preset coordinate conversion model to obtain the current position coordinates, obstacle coordinates and target position coordinates of the object to be grasped in the coordinate system of the multi-axis parallel robot.

[0056] In this embodiment, the current position coordinates of the object to be grasped, the coordinates of the obstacles, and the target position coordinates of the object to be grasped can be identified from the real working environment image in the following manner. Specifically, first, the image of the real working environment is converted from the original color space (such as RGB) to another color space (such as HSV), so that the real working environment image becomes an image represented by the hue channel, the saturation channel, and the value channel, thereby obtaining the target environment image. This conversion facilitates subsequent image analysis based on color characteristics. Then, the channel sub-image corresponding to the saturation channel is extracted from the converted target environment image. Afterwards, the above-mentioned channel sub-image is binarized using three different preset saturation thresholds (respectively, a first preset saturation threshold, a second preset saturation threshold, and a third preset saturation threshold). These thresholds are determined based on the degree of distinction between the object to be grasped, the obstacle, and the target position in saturation. The first sub-image, obtained after processing according to the first preset saturation threshold, contains only the object to be grasped; the second sub-image, obtained after processing according to the second preset saturation threshold, contains only obstacles; and the third sub-image, obtained after processing according to the third preset saturation threshold, contains only the target location of the object to be grasped. Specifically, in order to be able to extract the object to be grasped, obstacles, and target location separately from the channel sub-image, the object to be grasped, obstacles, and target location can be pre-set to colors that can be distinguished by saturation in the channel sub-image, thereby achieving rapid extraction. After obtaining the first, second, and third sub-images, denoising processing is performed on each of these three sub-images to reduce noise interference in the image and improve the accuracy of subsequent analysis.

[0057] Next, template matching is performed on each denoised sub-image. Template matching is an image processing technique used to find parts of an image that are similar to a known template. Different templates can be used for different sub-images. For example, for the first sub-image, a template with a specific shape representing the object to be grasped can be used; for the second sub-image, a template with a specific shape representing an obstacle can be used; and for the third sub-image, a template with a specific shape representing the target location can be used. Furthermore, based on the template matching results, the current position coordinates of the object to be grasped, the coordinates of the obstacles, and the target position coordinates of the object to be grasped can be determined in the real working environment image.

[0058] Finally, a preset coordinate transformation model is used to convert these position coordinates from the image coordinate system (also known as the camera coordinate system) to the coordinate system of the multi-axis parallel robot. This is because the robot needs to perform grasping operations based on its own coordinate system. The converted coordinates can accurately guide the multi-axis parallel robot's movements.

[0059] like Figure 2As shown in the figure, a method for extracting the current position coordinates of the object to be grasped and the coordinates of the obstacles is given. It should be noted that Figure 2 This is just an example. To briefly illustrate the above coordinate extraction method, only the objects to be grasped and obstacles are shown in the real working environment image. Figure 2 In the figure, rectangles represent obstacles and stars represent objects to be grasped. First, obtain the real working environment image, i.e. Figure 2 The original image is obtained from the real working environment image, and then the color space is converted to obtain the target environment image under the HSV channel representation. Next, the channel sub-image under the S (saturation) channel is extracted from the target environment image, and then it is binarized to obtain the first sub-image (a sub-image containing only star-shaped objects to be grasped) and the second sub-image (a sub-image containing only rectangular obstacles). De-noising is performed on the first sub-image and the second sub-image respectively to obtain the denoised first sub-image and the second sub-image. Then, template matching is performed on these two sub-images respectively, so that the current position coordinates of the object to be grasped in the camera coordinate system can be identified from the denoised first sub-image, and the position coordinates of the obstacle in the camera coordinate system can be identified from the denoised second sub-image. Subsequently, the preset coordinate transformation model is used for coordinate transformation (Transmission), and the current position coordinates of the object to be grasped in the camera coordinate system and the position coordinates of the obstacle in the camera coordinate system are converted into coordinates in the multi-axis parallel robot coordinate system, and input into the PLC system for subsequent calculations.

[0060] The present embodiment can quickly and accurately identify the current position coordinates of the object to be grasped, the coordinates of obstacles, and the target position coordinates from an image of the real working environment. Using a preset coordinate conversion model, these coordinates are converted to coordinates in the coordinate system of the multi-axis parallel robot. This embodiment converts the coordinate information in the image into operational instructions that the multi-axis parallel robot can understand and execute, which is crucial for achieving tasks such as automated grasping, obstacle avoidance, and precise object placement.

[0061] In an embodiment of the present application, optionally, the method further includes: solving the rotation coefficient and the displacement coefficient in the initial coordinate transformation model by a nine-point calibration method, wherein the formula of the initial coordinate transformation model is as follows:

[0062]

[0063] in, Represents the coordinates in the multi-axis parallel robot coordinate system, represents the rotation coefficient, Represents the coordinates in the camera coordinate system, represents the displacement coefficient;

[0064] The obtained rotation coefficient and displacement coefficient are substituted into the initial coordinate transformation model to obtain the preset coordinate transformation model.

[0065] In this embodiment, the nine-point calibration method is a method for determining the conversion relationship between two different coordinate systems, especially in a robot vision system, for converting the image coordinates captured by a camera into coordinates in the operating space of a multi-axis parallel robot. This method solves the conversion parameters, including rotation coefficients and displacement coefficients, by the corresponding positions of a series of known points in the two coordinate systems, so that the image coordinates can be converted into coordinates in the multi-axis parallel robot coordinate system based on the determined rotation coefficients and displacement coefficients. The embodiment of the present application adopts the form of eye to hand, and the camera is placed on the upper part of the workbench.

[0066] First, establish an initial coordinate transformation model. Next, select nine known points within the multi-axis parallel robot's operating space. These points can be feature points that are easily identified and located in the camera image, and their positions in the multi-axis parallel robot's coordinate system are also known. For each calibration point, record its coordinates in the camera coordinate system, P_camera, and also record the coordinates of these points in the multi-axis parallel robot coordinate system, P_robot.

[0067] For each known point, a system of equations is constructed using the initial coordinate transformation model formula. Since each point has two coordinate components (x, y), 9 points will generate 18 equations. This system of equations is then solved using the least squares method or other optimization algorithm to obtain the best estimates of the rotation and displacement coefficients. These best estimates of the rotation and displacement coefficients are then substituted into the initial coordinate transformation model to obtain the preset coordinate transformation model. The preset coordinate transformation model can then be used to convert the camera coordinate system to the multi-axis parallel robot coordinate system.

[0068] In fact, there are 6 unknown parameters in the initial coordinate transformation model, namely 4 unknowns a, b, a′, b′ in the rotation coefficients, and 2 unknowns c, c′ in the displacement coefficients. Therefore, 6 coordinate points are required to complete the solution. However, in order to reduce errors, the embodiment of the present application chooses to use 9 points for camera calibration. This method is simple and effective, and can also improve the accuracy of the solution.

[0069] In an embodiment of the present application, optionally, after the "sending the target obstacle avoidance path to the control device of the physical robot" in step 104, the method further includes: receiving target data returned by the physical robot in real time, wherein the target data includes the current working environment image collected by the visual perception device and the working parameters collected by the sensor carried by the physical robot; based on the current working environment image and the working parameters, controlling the twin robot to follow the physical robot, and judging the path offset between the following path of the twin robot and the target obstacle avoidance path during the following process; when the path offset exceeds a preset offset threshold, sending a return instruction to the physical robot, and retraining the twin robot using a reinforcement learning algorithm based on the virtual working environment and the path offset to obtain a new target obstacle avoidance path; and resending the new target obstacle avoidance path to the control device of the physical robot.

[0070] In this embodiment, when the physical robot follows the target obstacle avoidance path, it can also send target data to the twin robot. Here, the physical robot can be equipped with a visual perception device and sensors (such as speed sensors, acceleration sensors, angle sensors, etc.). The visual perception device can collect real-time images of the current working environment of the physical robot during operation, while the sensors can collect real-time working parameters of the physical robot during operation, such as the position of each axis, robot speed, acceleration, and the position of each joint axis.

[0071] Next, the received target data is used to control the twin robot to follow the physical robot's movements in the virtual work environment, simulating the physical robot's movements in a real work environment. During the following process, the twin robot's walking path in the virtual environment is continuously compared with the preset target obstacle avoidance path (i.e., the optimal path that the physical robot is expected to follow to avoid obstacles and efficiently complete the task). The path offset between the twin robot's walking path and the target obstacle avoidance path is calculated and monitored to assess the difference between actual execution and the expected plan. When the detected path offset exceeds a preset offset threshold, it indicates that the physical robot's actual movement deviates significantly from the expected path, possibly due to encountering an unforeseen obstacle or environmental change. At this point, a return command is sent to the physical robot, instructing it to return to a safe position to avoid further deviation and subsequently to start again from the origin to facilitate verification of the effectiveness of the new target obstacle avoidance path. Simultaneously, the current virtual work environment can be updated based on the received target data, and the twin robot can be retrained using a reinforcement learning algorithm based on the updated virtual work environment and path offset.

[0072] After retraining, the twin robots learn a new obstacle avoidance path that is more suitable for their current working environment. This new target obstacle avoidance path is then sent back to the control unit of the real robot, allowing the real robot to navigate and avoid obstacles more accurately and effectively based on the updated path.

[0073] The present embodiment forms a closed-loop feedback system that continuously improves the physical robot's obstacle avoidance capabilities by continuously collecting target data from the physical robot, simulating and optimizing it in a virtual environment, and then applying the optimization results to the physical robot. This approach not only improves the robot's adaptability and intelligence, but also reduces the number of expensive and time-consuming experiments in a real environment.

[0074] In an embodiment of the present application, optionally, the reinforcement learning algorithm includes a DQN algorithm; before step 103, the method further includes: building multiple policy networks through the DQN algorithm, wherein the multiple policy networks are all single-layer neural networks, and the number of neurons contained in each policy network is different; accordingly, step 103 includes: according to the virtual working environment, using the DQN algorithm to train each policy network, determining the obstacle avoidance path with the maximum reward value corresponding to each policy network, and recording the number of actions and the maximum reward value corresponding to the obstacle avoidance path; according to the number of actions and the maximum reward value corresponding to each obstacle avoidance path, calculating the score corresponding to the obstacle avoidance path, and using the obstacle avoidance path with the highest score as the target obstacle avoidance path of the physical robot.

[0075] In this embodiment, the reinforcement learning algorithm can be a DQN (Deep Q-Network) algorithm, which uses a deep neural network based on the Q-learning algorithm to approximate the action function Q(s, a). Specifically, the DQN algorithm can be used to obtain the target obstacle avoidance path.

[0076] In one embodiment, a policy network can be built using the DQN algorithm, which is then trained in a virtual work environment to obtain an obstacle avoidance path corresponding to the maximum reward value. The obstacle avoidance path under the maximum reward value is directly used as the target obstacle avoidance path.

[0077] In another embodiment, multiple policy networks are first constructed using the DQN algorithm. These policy networks are all single-layer neural networks, but each contains a different number of neurons. This allows us to explore the impact of different network structures (i.e., different numbers of neurons) on policy learning, in order to find the network structure with the best performance.

[0078] Each policy network is then trained in a virtual work environment using the DQN algorithm. During training, each policy network is allowed to try different actions (i.e., angle transformations along different axes) to find the obstacle avoidance path with the maximum reward. After finding the obstacle avoidance path with the maximum reward, the number of actions corresponding to that path (i.e., the minimum number of actions required to complete the path) and the maximum reward are recorded. This information is used to subsequently evaluate the obstacle avoidance paths obtained by different policy networks. Next, a score is calculated based on the number of actions and maximum reward corresponding to each obstacle avoidance path, aiming to measure the efficiency and effectiveness of the obstacle avoidance path. A higher number of actions can be assigned a lower score, while a higher maximum reward can be assigned a higher score. This score can be calculated using a pre-defined score calculation formula. There are no specific requirements for the score calculation formula; it only needs to meet the following requirements: a higher number of actions results in a lower score, while a higher maximum reward results in a higher score. After obtaining the scores for each obstacle avoidance path from each policy network, the path with the highest score is selected as the target obstacle avoidance path for the physical robot. This path is then optimized in terms of obstacle avoidance effectiveness, action efficiency, and reward value.

[0079] like Figure 3 As shown, the training results of a different strategy network provided by an embodiment of the present application are shown. Among them, when using the DQN reinforcement learning algorithm for training, it is found that the efficiency of structural learning using a single-layer neural network is higher. Therefore, in this application, when using the DQN reinforcement learning algorithm to build a strategy network, a single-layer neuron architecture is adopted, and different strategy networks are built by changing the number of neurons. Assuming that the length and width of the current obstacle are detected through the real working environment image, the spatial coordinates occupied by the current rectangular obstacle can be obtained (the height is assumed to be known to be 30mm), and the target position coordinates and the current position coordinates of the item to be grasped are detected at the same time. After obtaining the above information, training begins, and the results are as follows Figure 3 As shown in the figure, it is found that the convergence speed is relatively fast when the number of neurons is 1024, and good convergence and training effects are also achieved when the number of neurons is 128, 256, and 512. However, when the number of neurons is 2048, it is difficult to converge to the maximum reward when the number of training iterations is 2000. Figure 3 In the table, the horizontal axis is the number of iterations (Epsiode), and the vertical axis is the training reward (Training Reward). Table 1 shows the number of actions included in the obstacle avoidance paths trained by different policy networks, as well as the maximum reward value of the optimal path. When the number of neurons is 1024 and 128, the number of actions included in the obstacle avoidance path is at least 48.

[0080] Table 1 The optimal number of obstacle avoidance path actions and the maximum reward value obtained under different numbers of neurons

[0081] Number of neurons The number of optimal obstacle avoidance path actions Maximum reward value 128 48 99555.86 256 50 99491.85 512 51 99560.47 1024 48 99536.86

[0082] like Figure 4 As shown, a three-dimensional schematic diagram of the obstacle avoidance paths and obstacle positions planned under different strategy networks (strategy networks corresponding to 128, 256, 512, and 1024 neurons) is also drawn. It can be found that the red line is the optimal route trained under obstacle-free conditions. It has a clear collision with the obstacle, while the other four obstacle avoidance paths obtained based on the real working environment images all avoid the obstacles and reach the target location. This shows that the above method of the embodiment of the present application is feasible.

[0083] This embodiment of the application builds multiple policy networks with different structures and trains them in a virtual work environment to find the optimal obstacle avoidance path. The performance of different policy networks is compared based on scores, and the obstacle avoidance path that best suits the current task is ultimately selected as the target obstacle avoidance path for the physical robot. This method significantly improves the accuracy and efficiency of the target obstacle avoidance path.

[0084] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides a robot obstacle avoidance path planning system, such as Figure 5 As shown, the system includes:

[0085] a coordinate recognition module, configured to receive an image of a real working environment transmitted by the physical robot and identify, from the image, the current position coordinates of an object to be grasped, the coordinates of obstacles, and the coordinates of a target position of the object to be grasped, wherein the image of the real working environment is acquired by a visual perception device carried by the physical robot;

[0086] An environment construction module, configured to construct a virtual working environment of the twin robot corresponding to the physical robot according to the current position coordinates of the object to be grasped, the coordinates of the obstacles, and the target position coordinates of the object to be grasped;

[0087] A training module is configured to train the twin robot using a reinforcement learning algorithm according to the virtual working environment to obtain an obstacle avoidance path with a maximum reward value, and determine a target obstacle avoidance path for the physical robot based on the obstacle avoidance path with a maximum reward value;

[0088] A path sending module is used to send the target obstacle avoidance path to the control device of the physical robot, so that the physical robot is controlled by the control device to move the object to be grasped from the current position to the target position based on the target obstacle avoidance path.

[0089] Optionally, the robot is a multi-axis parallel robot; and the training module is further used to:

[0090] Before training the twin robots using the reinforcement learning algorithm, defining an action space of the reinforcement learning algorithm, wherein each action in the action space is used to indicate a rotation angle change of each axis of the multi-axis parallel robot at a current position;

[0091] Defining a state space for a reinforcement learning algorithm, wherein the state representation in the state space is obtained based on the current angles of each axis of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform, and the current three-dimensional coordinates of the center point of the moving platform are obtained based on the forward kinematic modeling of the twin robots;

[0092] Define a reward function for the reinforcement learning algorithm, wherein the reward function consists of a sparse reward, an intermediate reward, and an action reward. The sparse reward is determined based on the current position coordinates of the end of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform. The intermediate reward is determined based on the starting three-dimensional coordinates, current three-dimensional coordinates, and end three-dimensional coordinates of the center point of the moving platform of the multi-axis parallel robot. The action reward is determined based on the number of actions of the twin robot.

[0093] Accordingly, the training module is further used to:

[0094] The twin robot is trained according to the virtual working environment, and the action space, state space and reward function of the defined reinforcement learning algorithm to obtain an obstacle avoidance path with the maximum reward value.

[0095] Optionally, the calculation formula of the sparse reward is as follows:

[0096]

[0097] Among them, R sp Represents a sparse reward for the current action; when the current position coordinates of the end of the multi-axis parallel robot coincide with the target position coordinates, it is determined that the end of the multi-axis parallel robot has reached the target position, and the current position coordinates of the end of the multi-axis parallel robot are determined based on the current position coordinates of the center point of the moving platform; when the current position coordinates of the center point of the moving platform of the multi-axis parallel robot intersect with the obstacle coordinate range, it is determined that the moving platform of the multi-axis parallel robot has collided with an obstacle; when the current position coordinates of the center point of the moving platform of the multi-axis parallel robot exceed the coordinate range corresponding to the preset workspace, it is determined that the moving platform of the multi-axis parallel robot exceeds the preset workspace;

[0098] The calculation formula for the intermediate reward is as follows:

[0099]

[0100] Among them, Rt represents the intermediate reward of the current action; x, y, z represent the current three-dimensional coordinates of the center point of the moving platform, x0, y0, z0 represent the starting three-dimensional coordinates of the center point of the moving platform, x d ,y d ,z d represents the three-dimensional coordinates of the end point of the center point of the moving platform, and ω is the weight value;

[0101] The calculation formula of the reward function is as follows:

[0102] R=R sp +R t +R s ;

[0103] Among them, R represents the final reward of the current action, R s (R s <0) indicates the action reward of the current action.

[0104] Optionally, the coordinate identification module is used to:

[0105] Performing color space transformation processing on the real working environment image to obtain a target environment image, wherein the target environment image is represented based on a hue channel, a saturation channel, and a lightness channel;

[0106] Extracting a channel sub-image corresponding to the saturation channel from the target environment image, and binarizing the channel sub-image based on a first preset saturation threshold to obtain a first sub-image containing only the object to be grasped, binarizing the channel sub-image based on a second preset saturation threshold to obtain a second sub-image containing only the obstacle, and binarizing the channel sub-image based on a third preset saturation threshold to obtain a third sub-image containing only the target position of the object to be grasped;

[0107] performing denoising processing on each of the first sub-image, the second sub-image, and the third sub-image, and performing template matching on each of the denoised first sub-image, the second sub-image, and the third sub-image, and determining, based on the template matching results, the current position coordinates of the object to be grasped in the first sub-image, the obstacle coordinates of the obstacle in the second sub-image, and the target position coordinates of the target position of the object to be grasped in the third sub-image;

[0108] The coordinates of the current position of the object to be grasped, the obstacle coordinates and the target position coordinates are transformed by a preset coordinate transformation model to obtain the current position of the object to be grasped, the obstacle coordinates and the target position coordinates in the coordinate system of the multi-axis parallel robot.

[0109] Optionally, the device further includes a calibration module, wherein the calibration module is configured to:

[0110] The rotation coefficient and displacement coefficient in the initial coordinate transformation model are solved by the nine-point calibration method, wherein the formula of the initial coordinate transformation model is as follows:

[0111]

[0112] in, Represents the coordinates in the multi-axis parallel robot coordinate system, represents the rotation coefficient, Represents the coordinates in the camera coordinate system, represents the displacement coefficient;

[0113] The obtained rotation coefficient and displacement coefficient are substituted into the initial coordinate transformation model to obtain the preset coordinate transformation model.

[0114] Optionally, the device further comprises:

[0115] a data receiving module, configured to receive target data returned by the physical robot in real time after the target obstacle avoidance path is sent to the control device of the physical robot, wherein the target data includes an image of the current working environment collected by the visual perception device and working parameters collected by sensors carried by the physical robot;

[0116] an offset determination module, configured to control the twin robot to follow the physical robot based on the current working environment image and the working parameters, and to determine the path offset between the following path of the twin robot and the target obstacle avoidance path during the following process;

[0117] The training module is further configured to send a return instruction to the physical robot when the path deviation exceeds a preset deviation threshold, and retrain the twin robot using a reinforcement learning algorithm based on the virtual working environment and the path deviation to obtain a new target obstacle avoidance path;

[0118] The path sending module is also used to resend the new target obstacle avoidance path to the control device of the physical robot.

[0119] Optionally, the reinforcement learning algorithm includes a DQN algorithm; and the device further includes:

[0120] A network building module is used to build multiple strategy networks using the DQN algorithm before training the twin robots using the reinforcement learning algorithm according to the virtual working environment to obtain an obstacle avoidance path with a maximum reward value, wherein the multiple strategy networks are all single-layer neural networks and the number of neurons contained in each strategy network is different;

[0121] Accordingly, the training module is further used to:

[0122] According to the virtual working environment, each policy network is trained using the DQN algorithm to determine the obstacle avoidance path with the maximum reward value corresponding to each policy network, and the number of actions corresponding to the obstacle avoidance path and the maximum reward value are recorded;

[0123] According to the number of actions and the maximum reward value corresponding to each obstacle avoidance path, the score corresponding to the obstacle avoidance path is calculated, and the obstacle avoidance path with the highest score is used as the target obstacle avoidance path of the physical robot.

[0124] It should be noted that for other corresponding descriptions of the functional units involved in the robot obstacle avoidance path planning system provided in the embodiment of the present application, please refer to Figures 1 to 4 The corresponding description in the method will not be repeated here.

[0125] The present application also provides a computer device, which can be a personal computer, a server, a network device, etc. Figure 6 As shown, the computer device includes a bus, a processor, a memory, and a communication interface, and may also include an input / output interface and a display device. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the steps of each method embodiment are implemented.

[0126] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0127] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0128] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0129] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0130] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0131] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A robot obstacle avoidance path planning method, characterized in that: include: Receiving a real working environment image transmitted by a physical robot, and identifying the current position coordinates of an object to be grasped, obstacle coordinates, and target position coordinates of the object to be grasped from the real working environment image, wherein the real working environment image is acquired by a visual perception device carried by the physical robot; Constructing a virtual working environment of the twin robot corresponding to the physical robot according to the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped; According to the virtual working environment, the twin robot is trained using a reinforcement learning algorithm to obtain an obstacle avoidance path with a maximum reward value, and a target obstacle avoidance path of the physical robot is determined based on the obstacle avoidance path with the maximum reward value; Sending the target obstacle avoidance path to the control device of the physical robot, so that the control device controls the physical robot to move the object to be grasped from the current position to the target position based on the target obstacle avoidance path; The robot is a multi-axis parallel robot; before using the reinforcement learning algorithm to train the twin robots, a reward function of the reinforcement learning algorithm is first defined, and the reward function consists of a sparse reward, an intermediate reward, and an action reward; The calculation formula of the sparse reward is as follows: ； in, Represents a sparse reward for the current action; when the current position coordinates of the end of the multi-axis parallel robot coincide with the target position coordinates, it is determined that the end of the multi-axis parallel robot has reached the target position, and the current position coordinates of the end of the multi-axis parallel robot are determined based on the current position coordinates of the center point of the moving platform; when the current position coordinates of the center point of the moving platform of the multi-axis parallel robot intersect with the obstacle coordinate range, it is determined that the moving platform of the multi-axis parallel robot has collided with an obstacle; when the current position coordinates of the center point of the moving platform of the multi-axis parallel robot exceed the coordinate range corresponding to the preset workspace, it is determined that the moving platform of the multi-axis parallel robot exceeds the preset workspace; The calculation formula for the intermediate reward is as follows: ； in, Represents the intermediate reward of the current action; represents the current three-dimensional coordinates of the center point of the moving platform, represents the starting three-dimensional coordinates of the center point of the moving platform, represents the three-dimensional coordinates of the end point of the center point of the moving platform, is the weight value; The calculation formula of the reward function is as follows: ； in, represents the final reward of the current action, represents the action reward of the current action, .

2. The method according to claim 1, characterized in that Before training the twin robots using the reinforcement learning algorithm, the method further includes: defining an action space of a reinforcement learning algorithm, wherein each action in the action space is used to indicate a rotation angle change of each axis of the multi-axis parallel robot at a current position; Defining a state space for a reinforcement learning algorithm, wherein the state representation in the state space is obtained based on the current angles of each axis of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform, and the current three-dimensional coordinates of the center point of the moving platform are obtained based on the forward kinematic modeling of the twin robots; Define a reward function for the reinforcement learning algorithm, wherein the reward function consists of a sparse reward, an intermediate reward, and an action reward. The sparse reward is determined based on the current position coordinates of the end of the multi-axis parallel robot and the current three-dimensional coordinates of the center point of the moving platform. The intermediate reward is determined based on the starting three-dimensional coordinates, current three-dimensional coordinates, and end three-dimensional coordinates of the center point of the moving platform of the multi-axis parallel robot. The action reward is determined based on the number of actions of the twin robot. Accordingly, the training of the twin robots using a reinforcement learning algorithm according to the virtual working environment to obtain an obstacle avoidance path with a maximum reward value includes: The twin robot is trained according to the virtual working environment, and the action space, state space and reward function of the defined reinforcement learning algorithm to obtain an obstacle avoidance path with the maximum reward value.

3. The method according to claim 1, characterized in that The identifying the current position coordinates of the object to be grasped, the obstacle coordinates, and the target position coordinates of the object to be grasped from the real working environment image includes: Performing color space transformation processing on the real working environment image to obtain a target environment image, wherein the target environment image is represented based on a hue channel, a saturation channel, and a lightness channel; Extracting a channel sub-image corresponding to the saturation channel from the target environment image, and binarizing the channel sub-image based on a first preset saturation threshold to obtain a first sub-image containing only the object to be grasped, binarizing the channel sub-image based on a second preset saturation threshold to obtain a second sub-image containing only the obstacle, and binarizing the channel sub-image based on a third preset saturation threshold to obtain a third sub-image containing only the target position of the object to be grasped; performing denoising processing on each of the first sub-image, the second sub-image, and the third sub-image, and performing template matching on each of the denoised first sub-image, the second sub-image, and the third sub-image, and determining, based on the template matching results, the current position coordinates of the object to be grasped in the first sub-image, the obstacle coordinates of the obstacle in the second sub-image, and the target position coordinates of the target position of the object to be grasped in the third sub-image; The coordinates of the current position of the object to be grasped, the obstacle coordinates and the target position coordinates are transformed by a preset coordinate transformation model to obtain the current position of the object to be grasped, the obstacle coordinates and the target position coordinates in the coordinate system of the multi-axis parallel robot.

4. The method according to claim 3, characterized in that The method further comprises: The rotation coefficient and displacement coefficient in the initial coordinate transformation model are solved by the nine-point calibration method, wherein the formula of the initial coordinate transformation model is as follows: ； in, Represents the coordinates in the multi-axis parallel robot coordinate system, represents the rotation coefficient, Represents the coordinates in the camera coordinate system, represents the displacement coefficient; The obtained rotation coefficient and displacement coefficient are substituted into the initial coordinate transformation model to obtain the preset coordinate transformation model.

5. The method according to claim 1, wherein After sending the target obstacle avoidance path to the control device of the physical robot, the method further includes: receiving target data returned by the physical robot in real time, wherein the target data includes an image of the current working environment collected by the visual perception device and working parameters collected by sensors carried by the physical robot; Based on the current working environment image and the working parameters, controlling the twin robot to follow the physical robot, and determining the path offset between the following path of the twin robot and the target obstacle avoidance path during the following process; When the path deviation exceeds a preset deviation threshold, a return instruction is sent to the physical robot, and the twin robot is retrained using a reinforcement learning algorithm based on the virtual working environment and the path deviation to obtain a new target obstacle avoidance path; The new target obstacle avoidance path is resent to the control device of the physical robot.

6. The method according to claim 1, characterized in that The reinforcement learning algorithm includes a DQN algorithm; before training the twin robots using the reinforcement learning algorithm according to the virtual working environment to obtain an obstacle avoidance path with a maximum reward value, the method further includes: Using the DQN algorithm, multiple policy networks are constructed, wherein each of the policy networks is a single-layer neural network and each policy network contains a different number of neurons; Accordingly, the twin robots are trained using a reinforcement learning algorithm according to the virtual working environment to obtain an obstacle avoidance path with a maximum reward value, and a target obstacle avoidance path of the physical robot is determined based on the obstacle avoidance path with a maximum reward value, including: According to the virtual working environment, each policy network is trained using the DQN algorithm to determine the obstacle avoidance path with the maximum reward value corresponding to each policy network, and the number of actions corresponding to the obstacle avoidance path and the maximum reward value are recorded; According to the number of actions and the maximum reward value corresponding to each obstacle avoidance path, the score corresponding to the obstacle avoidance path is calculated, and the obstacle avoidance path with the highest score is used as the target obstacle avoidance path of the physical robot.

7. A robot obstacle avoidance path planning system, characterized in that: include: a coordinate recognition module, configured to receive an image of a real working environment transmitted by the physical robot and identify, from the image, the current position coordinates of an object to be grasped, the coordinates of obstacles, and the coordinates of a target position of the object to be grasped, wherein the image of the real working environment is acquired by a visual perception device carried by the physical robot; An environment construction module, configured to construct a virtual working environment of the twin robot corresponding to the physical robot according to the current position coordinates of the object to be grasped, the coordinates of the obstacles, and the target position coordinates of the object to be grasped; A training module is configured to train the twin robot using a reinforcement learning algorithm according to the virtual working environment to obtain an obstacle avoidance path with a maximum reward value, and determine a target obstacle avoidance path for the physical robot based on the obstacle avoidance path with a maximum reward value; a path sending module, configured to send the target obstacle avoidance path to the control device of the physical robot, so as to control the physical robot through the control device to move the object to be grasped from the current position to the target position based on the target obstacle avoidance path; The robot is a multi-axis parallel robot; the training module is further used to define a reward function of a reinforcement learning algorithm, wherein the reward function is composed of a sparse reward, an intermediate reward, and an action reward; The calculation formula of the sparse reward is as follows: ； in, Represents a sparse reward for the current action; when the current position coordinates of the end of the multi-axis parallel robot coincide with the target position coordinates, it is determined that the end of the multi-axis parallel robot has reached the target position, and the current position coordinates of the end of the multi-axis parallel robot are determined based on the current position coordinates of the center point of the moving platform; when the current position coordinates of the center point of the moving platform of the multi-axis parallel robot intersect with the obstacle coordinate range, it is determined that the moving platform of the multi-axis parallel robot has collided with an obstacle; when the current position coordinates of the center point of the moving platform of the multi-axis parallel robot exceed the coordinate range corresponding to the preset workspace, it is determined that the moving platform of the multi-axis parallel robot exceeds the preset workspace; The calculation formula for the intermediate reward is as follows: ； in, Represents the intermediate reward of the current action; represents the current three-dimensional coordinates of the center point of the moving platform, represents the starting three-dimensional coordinates of the center point of the moving platform, represents the three-dimensional coordinates of the end point of the center point of the moving platform, is the weight value; The calculation formula of the reward function is as follows: ； in, represents the final reward of the current action, represents the action reward of the current action, .

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

Images