Robot, obstacle avoidance method thereof, model training method, device, and storage medium

Abstract

The application relates to a robot and an obstacle avoidance method thereof, a model training method, a device and a storage medium. The robot obstacle avoidance method comprises the following steps: in the case that a starting instruction for indicating that a mechanical arm is unfolded is received, a target relative position between the mechanical arm and a target obstacle, a current pose of the mechanical arm and a target pose of the mechanical arm are acquired; the target relative position, the current pose and the target pose are processed by using a pre-trained obstacle avoidance neural network to obtain an obstacle avoidance motion sequence; and the mechanical arm is controlled to unfold from the current pose and be in the target pose based on the obstacle avoidance motion sequence. In the embodiment of the application, the correct obstacle avoidance motion sequence planned by using an existing navigation algorithm is used for learning of the neural network, so that the trained neural network can guarantee the accuracy of planning of the obstacle avoidance motion sequence, and meanwhile, the speed of planning of the obstacle avoidance motion sequence is improved and the data processing process is simplified.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence, specifically relating to a robot and its obstacle avoidance method, model training method, device and storage medium. Background Technology

[0002] Nowadays, doctors can use surgical robots to perform surgeries on patients, improving the precision of the procedures. However, surgical robots are expensive, and if the robotic arm collides with an obstacle, it can cause damage or malfunction. Therefore, the robotic arm of a surgical robot needs to avoid surrounding obstacles during deployment.

[0003] In existing technologies, surgical robots typically employ two types of obstacle avoidance algorithms in high-dimensional space: sampling-based and search-based. However, sampling-based navigation algorithms generally plan obstacle avoidance paths by randomly sampling in continuous space, and the planned obstacle avoidance path may not succeed on the first attempt, thus involving a certain degree of probability.

[0004] Search-based navigation algorithms typically divide a spatial image into grids and perform path searching and obstacle avoidance between these grids. During the search process, the location of obstacles is taken into account, and areas with obstacles are designated as prohibited zones. This allows the calculation of all paths that can avoid obstacles. However, while search-based navigation algorithms can find all paths that can avoid obstacles with a high success rate, they are computationally intensive and slow in complex environments. Summary of the Invention

[0005] Therefore, the purpose of this application is to provide a robot and its obstacle avoidance method, model training method, device and storage medium to improve the problems of poor obstacle avoidance effect or computational complexity and low efficiency of existing robot obstacle avoidance algorithms.

[0006] The embodiments of this application are implemented as follows:

[0007] In a first aspect, embodiments of this application provide a robot obstacle avoidance method, characterized in that it is applied to a robot, the robot including a robotic arm, the method comprising: upon receiving a start command instructing the robotic arm to deploy, acquiring a target relative position between the robotic arm and a target obstacle, the current pose of the robotic arm, and a target pose of the robotic arm; wherein the current pose represents the current position and current posture of the robotic arm, and the target pose represents the target position and target posture that the robotic arm is required to be in after deployment; processing the target relative position, the current pose, and the target pose using a pre-trained obstacle avoidance neural network to obtain an obstacle avoidance action sequence; wherein the obstacle avoidance action sequence is used to instruct the robotic arm to deploy from the current pose, avoiding the target obstacle, and to be in the target pose; and controlling the robotic arm to deploy from the current pose and be in the target pose based on the obstacle avoidance action sequence.

[0008] In this embodiment, the relative position between the robotic arm and the target obstacle, the current pose of the robotic arm, and the target pose of the robotic arm are input into the pre-trained obstacle avoidance neural network to quickly obtain the corresponding obstacle avoidance action sequence. This can improve the accuracy of generating obstacle avoidance action sequences, increase efficiency, and simplify the data processing process.

[0009] In one possible implementation of the first aspect embodiment, the distance between the plurality of sub-target positions is a target step length. Planning the plurality of sub-target positions includes: obtaining a plurality of step lengths for the robotic arm to move along the obstacle avoidance path, wherein the computation time required for the robotic arm to plan the sub-target positions along the obstacle avoidance path based on each of the plurality of step lengths is within a preset time range; obtaining the success rate of the robotic arm in avoiding obstacles along the obstacle avoidance path based on each of the plurality of step lengths, obtaining a plurality of success rates corresponding to the plurality of step lengths; selecting the step length with the highest success rate among the plurality of step lengths as the target step length; and planning the plurality of sub-target positions based on the target step length and the obstacle avoidance path. In this embodiment, by selecting the step length with the highest success rate as the target step length within a step length range where the computation time is within a preset range, the efficiency of the robotic arm in avoiding obstacles along the obstacle avoidance path is improved while ensuring the success rate of the robotic arm in avoiding obstacles along the obstacle avoidance path.

[0010] In one possible implementation of the first aspect embodiment, a pre-trained obstacle avoidance neural network is used to process the target relative position, the current pose, and the target pose to obtain an obstacle avoidance action sequence, including: using the obstacle avoidance neural network to determine an obstacle avoidance path based on the target relative position, the current pose, and the target pose; using the obstacle avoidance neural network to plan multiple sub-target positions based on the obstacle avoidance path, and planning the posture that the robotic arm needs to perform at each sub-target position.

[0011] In this embodiment of the application, during the process of using the obstacle avoidance neural network to process the target relative position, current pose, and target pose, the obstacle avoidance neural network first determines the obstacle avoidance path based on the target relative position, current pose, and target pose. Then, based on the obstacle avoidance path, the obstacle avoidance neural network determines multiple sub-target positions in the obstacle avoidance path and plans the posture to be executed at each sub-target position. By refining the obstacle avoidance path into multiple sub-target positions and planning the posture to be executed by the robotic arm at each sub-target position, the actions performed by the robotic arm to avoid the target obstacle during the deployment process are more precise, reducing the possibility of the robotic arm colliding with the target obstacle during the deployment process.

[0012] In one possible implementation of the first aspect embodiment, the obstacle avoidance action sequence includes an obstacle avoidance path for the robotic arm to avoid the target obstacle. The obstacle avoidance path includes multiple sub-target positions, and each sub-target position includes a posture that the robotic arm needs to perform at that sub-target position. Driving the robotic arm to unfold from the current pose and be in the target pose based on the obstacle avoidance action sequence includes: based on the sequential order of the multiple sub-target positions in the obstacle avoidance path, sequentially controlling the end effector of the robotic arm to reach each sub-target position and perform the posture required at that sub-target position until the robotic arm is in the target pose.

[0013] In this embodiment, during the process of controlling the robotic arm to unfold from the current pose and reach the target pose based on the obstacle avoidance action sequence, the current pose of the robotic arm includes the current position and current posture of the robotic arm, and the target pose includes the target position and target posture of the robotic arm. Based on the sequential order of multiple sub-target positions in the obstacle avoidance path planned between the current position and the target position, the end effector of the robotic arm is sequentially controlled to reach each sub-target position and execute the posture required at that sub-target position. Since the posture of the robotic arm is planned at each sub-target position, the robotic arm's action to avoid obstacles during unfolding is more precise, thereby reducing the possibility of the robotic arm colliding with the target obstacle during unfolding.

[0014] In one possible implementation of the first aspect embodiment, obtaining the target relative position between the robotic arm and the target obstacle includes: acquiring multiple point cloud images, each point cloud image containing a predetermined number of point cloud data points, each point cloud data point containing corresponding three-dimensional coordinates and a depth value; wherein, different point cloud images are point cloud images captured by depth cameras located at different positions of the robotic arm; each point cloud image contains a predetermined number of point cloud data points, each point cloud data point containing corresponding three-dimensional coordinates and a depth value; stitching the multiple point cloud images to obtain a spatial point cloud image of the robotic arm; and extracting the target relative position between the robotic arm and the target obstacle based on a preset image algorithm and the three-dimensional coordinates and depth values ​​corresponding to the point cloud data points contained in the spatial point cloud image.

[0015] In this embodiment, to obtain the target relative position between the robotic arm and the target obstacle, depth cameras are set at different positions in the space where the robotic arm is located. These cameras can capture point cloud images of the robotic arm from different directional perspectives. Each point cloud image contains a set number of point cloud data points, and each point cloud data point contains corresponding three-dimensional coordinates and depth values. Since each point cloud image can reflect the three-dimensional data in the space, multiple point cloud images obtained by depth cameras at different positions are stitched together to obtain a spatial point cloud image that can reflect the three-dimensional data of the entire space where the robotic arm is located. Based on the preset algorithm image and the point cloud data contained in the spatial point cloud image, the target relative position between the robotic arm and the target obstacle can be extracted more accurately. This allows the neural network to plan a more accurate obstacle avoidance action sequence, thereby reducing the possibility of the robotic arm colliding with the target obstacle during deployment.

[0016] In one possible implementation of the first aspect embodiment, the method further includes: acquiring multiple sets of simulated obstacle avoidance data, wherein the simulated obstacle avoidance data includes a simulated initial pose of the robotic arm, a simulated target pose, a simulated obstacle avoidance action sequence, and a simulated relative position between the simulated obstacle and the robotic arm, wherein each set of simulated obstacle avoidance data corresponds to a specific weight; using the multiple sets of simulated obstacle avoidance data to initially train an initial neural network without prior knowledge, thereby obtaining a trained initial neural network, wherein the initial training of the initial neural network without prior knowledge is a process in which the initial neural network learns patterns based on the multiple sets of simulated obstacle avoidance data; inputting the simulated initial pose, the simulated target pose, and the simulated relative position from the multiple sets of simulated obstacle avoidance data into the trained initial neural network to obtain multiple new obstacle avoidance action sequences; comparing the multiple simulated obstacle avoidance action sequences from the multiple sets of simulated obstacle avoidance data with the multiple new simulated obstacle avoidance action sequences to obtain a comparison result; and adjusting the parameters of the trained initial neural network based on the comparison result to obtain an obstacle avoidance neural network.

[0017] In this embodiment, during the training of the obstacle avoidance neural network, multiple sets of simulated obstacle avoidance data, including the simulated initial pose of the robotic arm, the simulated target pose, the simulated obstacle avoidance action sequence, and the simulated relative position between the obstacle and the robotic arm, are first used to train the initial neural network to learn patterns. To verify the accuracy of the obstacle avoidance action sequence planned by the trained neural network, the simulated initial pose, simulated target pose, and simulated relative position from the multiple sets of simulated obstacle avoidance data are input into the trained initial neural network. The trained initial neural network then plans multiple new obstacle avoidance action sequences. The simulated obstacle avoidance action sequences in each set of simulated obstacle avoidance data are then compared with the new simulated action sequences to obtain a comparison result. Based on the obtained comparison result, the parameters of the trained initial neural network are adjusted to obtain the obstacle avoidance neural network. The obstacle avoidance neural network obtained by training the initial neural network through the above steps has higher accuracy in planning the obstacle avoidance action sequence.

[0018] Secondly, embodiments of this application also provide a model training method, comprising: acquiring multiple sets of simulated obstacle avoidance data, wherein the simulated obstacle avoidance data includes a simulated initial pose of a robotic arm, a simulated target pose, a simulated obstacle avoidance action sequence, and a simulated relative position between a simulated obstacle and the robotic arm, wherein each set of simulated obstacle avoidance data corresponds to a specific weight; performing initial training on an initial neural network without prior knowledge using the multiple sets of simulated obstacle avoidance data to obtain a trained initial neural network, wherein the initial training on the initial neural network without prior knowledge is a process in which the initial neural network learns patterns based on the multiple sets of simulated obstacle avoidance data; inputting the simulated initial pose, the simulated target pose, and the simulated relative position from the multiple sets of simulated obstacle avoidance data into the trained initial neural network to obtain multiple new obstacle avoidance action sequences; comparing the multiple simulated obstacle avoidance action sequences from the multiple sets of simulated obstacle avoidance data with the multiple new simulated obstacle avoidance action sequences to obtain a comparison result; and adjusting the parameters of the trained initial neural network based on the comparison result to obtain an obstacle avoidance neural network.

[0019] In this embodiment, during the training of the obstacle avoidance neural network, multiple sets of simulated obstacle avoidance data, including the simulated initial pose of the robotic arm, the simulated target pose, the simulated obstacle avoidance action sequence, and the simulated relative position between the obstacle and the robotic arm, are first used to train the initial neural network to learn patterns. To verify the accuracy of the obstacle avoidance action sequence planned by the trained neural network, the simulated initial pose, simulated target pose, and simulated relative position from the multiple sets of simulated obstacle avoidance data are input into the trained initial neural network. The trained initial neural network then plans multiple new obstacle avoidance action sequences. The simulated obstacle avoidance action sequences in each set of simulated obstacle avoidance data are then compared with the new simulated action sequences to obtain a comparison result. Based on the obtained comparison result, the parameters of the trained initial neural network are adjusted to obtain the obstacle avoidance neural network. The obstacle avoidance neural network obtained by training the initial neural network through the above steps has higher accuracy in planning the obstacle avoidance action sequence.

[0020] In one possible implementation of the second aspect embodiment, acquiring multiple sets of simulated obstacle avoidance data includes: acquiring multiple sets of obstacle avoidance sample data, each set of obstacle avoidance sample data containing the corresponding simulated initial pose, simulated target pose, simulated obstacle avoidance action sequence, and simulated relative position; assigning weights to the simulated obstacle avoidance action sequences corresponding to each set of obstacle avoidance sample data to obtain the multiple sets of simulated obstacle avoidance data; wherein, a first weight is assigned to the obstacle avoidance sample data corresponding to the best simulated obstacle avoidance action sequence with an evaluation score greater than a set score line, and a second weight is assigned to the obstacle avoidance sample data corresponding to the non-best simulated obstacle avoidance action sequence with an evaluation score not greater than the set score line; wherein, the first weight is greater than the second weight.

[0021] In this embodiment, before training the obstacle avoidance neural network, multiple sets of simulated obstacle avoidance data are acquired. Each set of obstacle avoidance sample data includes a simulated initial pose, a simulated target pose, a simulated obstacle avoidance action sequence, and a simulated relative position. Since the obstacle avoidance quality of the simulated obstacle avoidance sequences in different sets varies, to improve the accuracy of the obstacle avoidance action sequences planned by the trained obstacle avoidance neural network, the simulated obstacle avoidance sequences corresponding to each set of obstacle avoidance sample data are scored, and then assigned corresponding weights based on the scores. Specifically, the obstacle avoidance sample data corresponding to the best simulated obstacle avoidance action sequence with an evaluation score greater than a set score threshold is assigned a first weight, and the obstacle avoidance sample data corresponding to the non-best simulated obstacle avoidance action sequence with an evaluation score less than a set score threshold is assigned a second weight. The first weight is greater than the second weight. By weighting each set of obstacle avoidance sample data, the best simulated obstacle avoidance action sequence with an evaluation score greater than a set score threshold is assigned a higher first weight, and the non-best obstacle avoidance action sequence with an evaluation score less than a set score threshold is assigned a lower second weight. This allows the obstacle avoidance neural network to be more inclined to learn from high-quality simulated obstacle avoidance data during training, thereby improving the quality of the obstacle avoidance action sequences planned by the trained obstacle avoidance neural network.

[0022] Thirdly, this application also provides a model training device, comprising: an acquisition module and a training module; the acquisition module is used to acquire multiple sets of simulated obstacle avoidance data, wherein the simulated obstacle avoidance data includes a simulated initial pose of a robotic arm, a simulated target pose, a simulated obstacle avoidance action data sequence, and a simulated relative position between a simulated obstacle and the robotic arm, wherein each set of simulated obstacle avoidance data corresponds to a weight; the training module is used to perform initial training on an initial neural network without prior knowledge using the multiple sets of simulated obstacle avoidance data to obtain a trained initial neural network, wherein the initial training on the initial neural network without prior knowledge is a process in which the initial neural network learns patterns based on the multiple sets of simulated obstacle avoidance data; the simulated initial pose, the simulated target pose, and the simulated relative position from the multiple sets of simulated obstacle avoidance data are input into the trained initial neural network to obtain multiple new obstacle avoidance action sequences; multiple simulated obstacle avoidance action sequences from the multiple sets of simulated obstacle avoidance data are compared with multiple new simulated obstacle avoidance action sequences to obtain a comparison result; and the parameters of the trained initial neural network are adjusted based on the comparison result to obtain an obstacle avoidance neural network.

[0023] Fourthly, embodiments of this application also provide a robot, the robot comprising: a robotic arm, a processor, and a controller; the processor, configured to, upon receiving a start command instructing the robotic arm to deploy, acquire the target relative position between the robotic arm and a target obstacle, the current pose of the robotic arm, and the target pose of the robotic arm; wherein the current pose represents the current position and current posture of the robotic arm, and the target pose represents the target position and target posture that the robotic arm is required to be in after deployment; and to process the target relative position, the current pose, and the target pose using a pre-trained obstacle avoidance neural network to obtain an obstacle avoidance action sequence; wherein the obstacle avoidance action sequence is used to instruct the robotic arm to deploy from the current pose, avoiding the target obstacle, and to be in the target pose; the controller, connected to the processor and the robotic arm, is configured to control the robotic arm to deploy from the current pose and be in the target pose based on the obstacle avoidance action sequence.

[0024] In one possible implementation of the fourth aspect embodiment, the robot further includes: multiple depth cameras connected to the processor, the multiple depth cameras being positioned at different locations on the robotic arm to capture point cloud images of the robotic arm from different directional perspectives; the processor is further configured to acquire multiple point cloud images, each point cloud image containing a predetermined number of point cloud data points, each point cloud data point containing corresponding three-dimensional coordinates and depth values; and to stitch the multiple point cloud images together to obtain a spatial point cloud image of the robotic arm; and to obtain the target relative position between the robotic arm and the target obstacle based on a preset image algorithm and the three-dimensional coordinates and depth values ​​corresponding to the point cloud data contained in the spatial point cloud image.

[0025] Other features and advantages of this application will be set forth in the following description. The objectives and other advantages of this application can be realized and obtained through the structures specifically pointed out in the written description and the accompanying drawings. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The above and other objects, features, and advantages of this application will become clearer through the drawings. The same reference numerals indicate the same parts in all the drawings. The drawings are not intentionally drawn to scale to actual size; the focus is on illustrating the main points of this application.

[0027] Figure 1A schematic flowchart of a model training method provided in an embodiment of this application is shown.

[0028] Figure 2 A flowchart illustrating a robot obstacle avoidance method provided in an embodiment of this application is shown.

[0029] Figure 3 This diagram illustrates the positional relationship between a robotic arm and multiple depth cameras, as provided in an embodiment of this application.

[0030] Figure 4 A schematic diagram of the structure of a robot provided in an embodiment of this application is shown.

[0031] Figure 5 A schematic diagram of the structure of a model training device provided in an embodiment of this application is shown. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. The following embodiments are provided as examples to more clearly illustrate the technical solutions of this application, and should not be used to limit the scope of protection of this application. Those skilled in the art will understand that, without conflict, the following embodiments and features can be combined with each other.

[0033] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Without further limitations, an element defined by the phrase "comprising a…" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0034] Furthermore, the term "and / or" in this application is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0035] Given the problems of poor obstacle avoidance performance or computational complexity leading to low efficiency in existing robot obstacle avoidance algorithms, this application provides a novel robot obstacle avoidance method to address the shortcomings of existing navigation algorithms. This application utilizes the acquired correct obstacle avoidance action sequence to train a neural network, enabling the trained neural network to plan obstacle avoidance actions that maintain accuracy while improving the speed of the obstacle avoidance action sequence planning process. The following will combine... Figure 1 The model training method provided in the embodiments of this application will be described;

[0036] Step S101: Obtain multiple sets of simulated obstacle avoidance data.

[0037] In this embodiment, to enable the initial neural network to learn patterns in the correct obstacle avoidance action sequence, multiple sets of simulated obstacle avoidance data need to be acquired. By learning the simulated obstacle avoidance action sequence that enables the robotic arm to successfully avoid obstacles, the initial neural network can achieve higher accuracy in planning obstacle avoidance action sequences.

[0038] Each set of simulated obstacle avoidance data includes the simulated initial pose, the simulated target pose, the simulated obstacle avoidance sequence, and the simulated relative position between the obstacle and the robotic arm. The simulated initial pose and simulated relative position in each set of simulated obstacle avoidance data can simulate the positional state of the robotic arm and the target obstacle in the space where the robotic arm is located.

[0039] The simulated target pose of the robotic arm can be the position state that the robotic arm needs to achieve after it is deployed.

[0040] Each set of simulated obstacle avoidance action sequences can be a series of positions that the robotic arm needs to reach from its current position to the target position to avoid the target obstacle, as well as the actions performed in the corresponding posture at each position. Based on the above multiple sets of simulated obstacle avoidance action sequences, the robotic arm can avoid the target obstacle and unfold.

[0041] In one implementation, multiple sets of simulated obstacle avoidance data can be obtained by downloading pre-processed sets of simulated obstacle avoidance data from a database to train the initial neural network.

[0042] In this implementation, the weights corresponding to the multiple sets of simulated obstacle avoidance data obtained can be different or the same.

[0043] In another implementation, the method for obtaining multiple sets of simulated obstacle avoidance data can be as follows: obtain multiple sets of obstacle avoidance sample data, each set of obstacle avoidance sample data containing the corresponding simulated initial pose, simulated target pose, simulated obstacle avoidance action sequence, and simulated relative position; assign weights to the simulated obstacle avoidance action sequence corresponding to each set of obstacle avoidance sample data to obtain multiple sets of simulated obstacle avoidance data; wherein, assign a first weight to the obstacle avoidance sample data corresponding to the best simulated obstacle avoidance action sequence with an evaluation score greater than a set score line, and assign a second weight to the obstacle avoidance sample data corresponding to the non-best simulated obstacle avoidance action data with an evaluation score not greater than a set score line, wherein the first weight is greater than the second weight.

[0044] In this implementation, the simulated obstacle avoidance action sequence in each set of obstacle avoidance sample data can instruct the robotic arm to deploy and avoid obstacles. Although the robotic arm can deploy according to the simulated obstacle avoidance action sequence and avoid obstacles, the quality of the simulated obstacle avoidance action sequences in different obstacle avoidance sample data is different. To improve the quality of the obstacle avoidance action sequence output by the neural network, corresponding weights can be assigned to the acquired multiple sets of obstacle avoidance sample data. Specifically, the obstacle avoidance sample data corresponding to the best simulated obstacle avoidance action sequence with an evaluation score greater than a set score line is assigned a first weight, and the obstacle avoidance sample data corresponding to the non-best simulated obstacle avoidance action sequence with an evaluation score less than the set score line is assigned a second weight (for example, the first weight can be 10, and the second weight can be 1; the specific values ​​of the first and second weights can be set according to requirements and are not limited here). By assigning weights to multiple sets of obstacle avoidance sample data, multiple sets of simulated obstacle avoidance data are obtained, making the obstacle avoidance neural network more inclined to learn from high-quality simulated obstacle avoidance data during training, thereby improving the quality of the obstacle avoidance action sequence planned by the trained obstacle avoidance neural network.

[0045] The criteria for evaluating the optimal simulated obstacle avoidance sequence can be the distance between the robotic arm and the obstacle when the robotic arm avoids it. A sequence with a distance greater than the safe distance is considered optimal, while a sequence with a distance less than the safe distance is considered suboptimal. Alternatively, an optimal simulated obstacle avoidance sequence can be defined as one where the path length corresponding to the obstacle avoidance is less than a certain set threshold, while a suboptimal sequence is defined as one where the path length corresponding to the obstacle avoidance is not less than a certain set threshold. The set threshold can be arbitrarily set and is not limited here. Other criteria can also be used to evaluate the optimal simulated obstacle avoidance sequence, and are not limited here.

[0046] Step S102: Use multiple sets of simulated obstacle avoidance data to perform initial training on the initial neural network without prior knowledge, and obtain the trained initial neural network.

[0047] In this embodiment, the initial training process of the initial neural network without prior knowledge is a process in which the initial neural network learns patterns based on multiple sets of simulated obstacle avoidance data. Since the neural network has a high computing speed and strong adaptability to learning samples, the large and well-trained initial neural network obtained after learning patterns from multiple sets of simulated obstacle avoidance data can plan the obstacle avoidance action sequence more quickly.

[0048] The initial neural network mentioned above can be a network model such as a convolutional neural network. Step S103: Input the simulated initial pose, simulated target pose, and simulated relative position from multiple sets of simulated obstacle avoidance data into the trained initial neural network to obtain multiple new obstacle avoidance action sequences.

[0049] To verify the accuracy of the new obstacle avoidance action sequences planned by the trained initial neural network, the simulated initial pose, simulated target pose, and simulated relative position from multiple sets of simulated obstacle avoidance data were input into the trained initial neural network again. This yielded multiple new obstacle avoidance action sequences planned by the trained initial neural network, providing reference data for verifying the accuracy of the trained initial neural network.

[0050] Step S104: Compare multiple simulated obstacle avoidance action sequences in multiple sets of simulated obstacle avoidance data with multiple new simulated obstacle avoidance action sequences to obtain comparison results; based on the comparison results, adjust the parameters of the trained initial neural network to obtain the obstacle avoidance neural network.

[0051] Since multiple simulated obstacle avoidance action sequences in multiple sets of simulated obstacle avoidance data can instruct the robotic arm to avoid obstacles during deployment, these sequences are accurate. By comparing multiple simulated obstacle avoidance action sequences from multiple sets of simulated obstacle avoidance data with multiple new simulated obstacle avoidance action sequences, the comparison results verify the accuracy of the multiple simulated obstacle avoidance action sequences. Based on the comparison results, the parameters of the trained initial neural network are adjusted accordingly, resulting in a more accurate obstacle avoidance neural network and thus a more accurate planned obstacle avoidance action sequence.

[0052] The above model training method can be used to obtain a trained obstacle avoidance neural network model. Then, the pre-trained neural network model is used to plan the obstacle avoidance action sequence when the robotic arm needs to avoid the target obstacle. The planned obstacle avoidance action can improve the speed of the obstacle avoidance action sequence planning process while ensuring the accuracy of the planned obstacle avoidance action sequence.

[0053] The following will combine Figure 2 The present application describes a robot obstacle avoidance method provided in its embodiments, wherein the robot obstacle avoidance method provided in its embodiments is applied to a robot, and the robot includes a robotic arm.

[0054] Step S201: Upon receiving a start command to instruct the robotic arm to deploy, acquire the target relative position between the robotic arm and the target obstacle, the current pose of the robotic arm, and the target pose of the robotic arm.

[0055] In this embodiment of the application, when a start command for instructing the robotic arm to unfold is received, the current pose obtained represents the current position and current posture of the robotic arm, the target pose obtained represents the target position and target posture that the robotic arm should be in after unfolding, and the target relative position obtained represents the relative position between the robotic arm and the target obstacle, so that the obstacle avoidance neural network can plan an accurate obstacle avoidance action sequence based on the current pose, the target pose and the target relative position.

[0056] The situations in which the start command for instructing the robotic arm to unfold is received can be either when the operator presses the mechanical button that instructs the robot to unfold the robotic arm, and the start command is transmitted by the sensor at the mechanical button, or when the operator uses the start button on the remote control to transmit the start command to the robot. The specific method of transmitting the start command is not limited here.

[0057] In one implementation, the relative position between the robotic arm and the target obstacle can be obtained by a sensor mounted on the robotic arm. The sensor can be a lidar, camera, ultrasonic sensor, etc. The sensor can be selected as needed and no limitation is made here.

[0058] In another implementation, obtaining the target relative position between the robotic arm and the target obstacle can be achieved by: acquiring multiple point cloud images, each containing a set number of point cloud data points, each point cloud data point containing corresponding three-dimensional coordinates and depth values; wherein, the different point cloud images are point cloud images captured by depth cameras located at different positions of the robotic arm, and depth cameras at different positions in the space where the robotic arm is located are used to capture point cloud images of the robotic arm from different directional perspectives; stitching the multiple point cloud images together to obtain a spatial point cloud image of the robotic arm; and obtaining the target relative position between the robotic arm and the target obstacle based on a preset image algorithm and the three-dimensional coordinates and depth values ​​corresponding to the point cloud data contained in the spatial point cloud image.

[0059] In this embodiment, a depth camera is a camera device capable of capturing scene depth information, also known as a depth sensor or 3D camera. The method for acquiring scene depth information can be by illuminating the scene with a structured light source (such as an infrared laser or projected texture) and using an infrared camera or sensor on the camera to record the pattern after reflection or deformation of the light source. The depth value of each pixel is obtained by analyzing the captured pattern changes. Alternatively, the time-of-flight principle can be used to measure the time it takes for light to travel from the camera to the object's surface and back. The depth camera emits short-pulse beams of light and records the time it takes for the beam to travel from the camera, be reflected by the object's surface, and return to the camera. By measuring the time difference, the time of flight of light can be calculated, thereby obtaining the distance or depth of the object. The specific method for acquiring depth information can be selected according to the actual situation and is not limited here. The set number of point cloud data contained in the point cloud image can be understood as a set number of pixels. Each pixel has its corresponding point cloud data, and each point cloud data includes the pixel's three-dimensional coordinates and depth value. The three-dimensional coordinates and depth value reflect the spatial coordinates of the pixel in the space where the robotic arm is located; therefore, the point cloud image can reflect three-dimensional data in space. In this embodiment, by setting depth cameras at different positions in the space where the robotic arm is located, and then stitching together multiple point cloud images obtained by the depth cameras at different positions, the resulting spatial point cloud image can reflect the three-dimensional data of the entire space where the robotic arm is located. Based on the preset algorithm image and the point cloud data contained in the spatial point cloud image, the relative position between the robotic arm and the target obstacle can be extracted more accurately, thereby enabling the neural network to plan a more accurate obstacle avoidance action sequence, thus reducing the possibility of the robotic arm colliding with the target obstacle during the deployment process.

[0060] For example, such as Figure 3 As shown, four arc-shaped camera mounts are set up around the robot, located in front, behind, left, and right of the robot. Three depth cameras are evenly distributed on each arc-shaped camera mount, with the three depth cameras on each arc-shaped camera mount located at the top, middle, and bottom of the mount, respectively. This arrangement can capture point cloud images of the robot from 12 different perspectives. By stitching together these 12 point cloud images, a spatial point cloud image reflecting the three-dimensional data of the entire space in which the robotic arm is located can be obtained, which can more accurately extract the target relative position between the robotic arm and the target obstacle.

[0061] Step S202: Use a pre-trained obstacle avoidance neural network to process the target's relative position, current pose, and target pose to obtain an obstacle avoidance action sequence.

[0062] In this embodiment of the application, a pre-trained obstacle avoidance neural network is used to process the target relative position, current pose and target pose. The resulting obstacle avoidance action sequence can be used to instruct the robotic arm to avoid the target obstacle from the current pose and unfold to the target pose.

[0063] In one implementation, the method of using a pre-trained obstacle avoidance neural network to process the target relative position, current pose, and target pose to obtain an obstacle avoidance action sequence can be as follows: input the target relative position, current pose, and target pose into the trained obstacle avoidance neural network, and sequentially generate the sub-target positions and the postures that the robotic arm needs to perform at each sub-target position.

[0064] In another implementation, the method of using a pre-trained obstacle avoidance neural network to process the target relative position, current pose, and target pose to obtain an obstacle avoidance action sequence can be as follows: using the obstacle avoidance neural network to determine the obstacle avoidance path based on the target relative position, current pose, and target pose; using the obstacle avoidance neural network to plan multiple sub-target positions based on the obstacle avoidance path, and planning the posture that the robotic arm needs to perform at each sub-target position.

[0065] In this implementation, during the process of using the obstacle avoidance neural network to process the target relative position, current pose, and target pose, the obstacle avoidance neural network first determines the obstacle avoidance path based on the target relative position, current pose, and target pose. Then, based on the obstacle avoidance path, the obstacle avoidance neural network determines multiple sub-target positions within the obstacle avoidance path and plans the posture to be executed at each sub-target position. By refining the obstacle avoidance path into multiple sub-target positions and planning the posture to be executed by the robotic arm at each sub-target position, the actions performed by the robotic arm to avoid the target obstacle during the deployment process are more precise, reducing the possibility of the robotic arm colliding with the target obstacle during the deployment process.

[0066] In one implementation, the method for planning multiple sub-target positions can be to calculate the target step length through multiple experiments, and then plan multiple sub-target positions at intervals of the target step length along the obstacle avoidance path. The success rate of the robotic arm avoiding obstacles along the obstacle avoidance path based on the target step length is within a set success rate range, and the calculation time for the robotic arm to plan multiple sub-target positions along the obstacle avoidance path based on the target step length is within a set time range. The set success rate and set time range can be adjusted according to requirements and are not limited here.

[0067] In another implementation, the method for planning multiple sub-target positions can be to determine the target step length using a bisection method, and then plan multiple sub-target positions at intervals of the target step length along the obstacle avoidance path. In the bisection process, the distance at which the robotic arm is fully extended is set as the maximum step length, and the success rate of the robotic arm avoiding obstacles along the obstacle avoidance path based on the maximum step length is tested. Then, the maximum step length is divided by 2, and the success rate of the robotic arm avoiding obstacles along the obstacle avoidance path based on this step length is tested again. The success rate corresponding to the maximum step length is compared with the success rate corresponding to this step length. If the success rate corresponding to this step length is greater than that corresponding to the maximum step length, the optimal step length interval is determined to be 0 - (maximum step length / 2). If the success rate corresponding to this step length is less than that corresponding to the maximum step length, the optimal step length interval is determined to be (maximum step length / 2) - (maximum step length). This process is repeated iteratively until the final interval length is less than 1 mm. The integer values ​​within this interval are then determined as the target step length, and multiple sub-target positions are planned at intervals of the target step length along the obstacle avoidance path.

[0068] In another implementation, the method for planning multiple sub-target positions may be to obtain multiple step lengths of the robotic arm moving along the obstacle avoidance path, wherein the computation time required for the robotic arm to plan the sub-target positions along the obstacle avoidance path based on each of the multiple step lengths is within a preset time range; to obtain the success rate of the robotic arm avoiding obstacles along the obstacle avoidance path based on each of the multiple step lengths, thereby obtaining multiple success rates corresponding to the multiple step lengths; to select the step length with the highest success rate among the multiple step lengths as the target step length; and to plan multiple sub-target positions based on the target step length and the obstacle avoidance path.

[0069] In this implementation, by acquiring multiple step lengths of the robotic arm moving along the obstacle avoidance path, the computation time required for the robotic arm to plan the sub-target position along the obstacle avoidance path based on each of the multiple step lengths is within a preset time range. This preset time range is obtained based on a large amount of experimental data. The fact that the computation time is within the preset range can ensure improved computational efficiency when planning the sub-target position in the obstacle avoidance path. The success rate of the robotic arm avoiding obstacles along the obstacle avoidance path is acquired for each of the multiple step lengths with a computation time within the preset range. The step length with the highest success rate is selected as the target step length. This can improve the efficiency of the robotic arm avoiding obstacles along the obstacle avoidance path while ensuring the success rate of the robotic arm's obstacle avoidance along the obstacle avoidance path.

[0070] The step length is the distance between adjacent sub-target positions. The success rate of the robotic arm in avoiding obstacles along the obstacle avoidance path based on each of the multiple step lengths can be obtained by simulating multiple environments in which the robotic arm performs path planning, and testing the success rate of different step lengths in these multiple environments.

[0071] The pre-trained obstacle avoidance neural network can be trained by the robot itself or by a third party, and the trained obstacle avoidance neural network can be obtained directly from the third party. If the obstacle avoidance data is obtained through robot training, then before step S202, the robot obstacle avoidance method further includes: acquiring multiple sets of simulated obstacle avoidance data, wherein the simulated obstacle avoidance data includes the simulated initial pose of the robotic arm, the simulated target pose, the simulated obstacle avoidance action sequence, and the simulated relative position between the simulated obstacle and the robotic arm, wherein each set of simulated obstacle avoidance data has a corresponding weight; using the multiple sets of simulated obstacle avoidance data to perform initial training on an initial neural network without prior knowledge, thereby obtaining a trained initial neural network, wherein the initial training on the initial neural network without prior knowledge is a process of the initial neural network learning patterns based on multiple sets of simulated obstacle avoidance data; inputting the simulated initial pose, simulated target pose, and simulated relative position from the multiple sets of simulated obstacle avoidance data into the trained initial neural network to obtain multiple new obstacle avoidance action sequences; comparing the multiple simulated obstacle avoidance action sequences from the multiple sets of simulated obstacle avoidance data with the multiple new simulated obstacle avoidance action sequences to obtain a comparison result; and adjusting the parameters of the trained initial neural network based on the comparison result to obtain the obstacle avoidance neural network.

[0072] The specific process of model training is the same as the training process shown in the above-described model training method embodiment, and will not be elaborated here.

[0073] Step S203: Based on the obstacle avoidance action sequence, control the robotic arm to unfold from the current pose and be in the target pose.

[0074] In this embodiment of the application, when the obstacle avoidance neural network plans an obstacle avoidance action sequence based on the current pose, the target pose, and the relative position of the target, the robotic arm can be controlled to unfold from the current pose and be in the target pose based on the obstacle avoidance action sequence, while ensuring that it does not collide with the target obstacle.

[0075] In one implementation, the robot directly unfolds from its current pose and reaches the target pose based on the obstacle avoidance action sequence.

[0076] In another approach, the obstacle avoidance sequence includes an obstacle avoidance path for the robotic arm to avoid the target obstacle. The obstacle avoidance path contains multiple sub-target positions, and each sub-target position contains the posture that the robotic arm needs to perform at that sub-target position. The method of controlling the robotic arm to unfold from the current pose and reach the target pose based on the obstacle avoidance sequence can be: based on the sequential order of the multiple sub-target positions in the obstacle avoidance path, control the end effector of the robotic arm to reach each sub-target position in turn and perform the posture required at that sub-target position until the robotic arm is in the target pose.

[0077] In this implementation, during the process of controlling the robotic arm to unfold from the current pose and reach the target pose based on the obstacle avoidance action sequence, the current pose of the robotic arm includes the current position and current posture of the robotic arm, and the target pose includes the target position and target posture of the robotic arm. Based on the sequential order of multiple sub-target positions in the obstacle avoidance path planned between the current position and the target position, the end effector of the robotic arm is sequentially controlled to reach each sub-target position and execute the posture required at that sub-target position. Since the posture of the robotic arm is planned at each sub-target position, the robotic arm's action to avoid obstacles during unfolding is more precise, thereby reducing the possibility of the robotic arm colliding with the target obstacle during unfolding.

[0078] The robot 100 provided in this application embodiment, such as Figure 4 As shown. The robot includes: a robotic arm 110, a processor 120, and a controller 130.

[0079] Processor 120: Upon receiving a start command instructing the robotic arm to deploy, it acquires the target relative position between the robotic arm and the target obstacle, the current pose of the robotic arm, and the target pose of the robotic arm; wherein, the current pose represents the current position and current posture of the robotic arm, and the target pose represents the target position and target posture that the robotic arm should be in after deployment; it processes the target relative position, current pose, and target pose using a pre-trained obstacle avoidance neural network to obtain an obstacle avoidance action sequence; wherein, the obstacle avoidance action sequence is used to instruct the robotic arm to deploy from the current pose to avoid the target obstacle and to be in the target pose.

[0080] Controller 130: Used to control the robotic arm to unfold from the current pose and to the target pose based on the obstacle avoidance action sequence.

[0081] Optionally, the processor 120 is specifically used to determine an obstacle avoidance path based on the relative position of the target, the current pose, and the target pose using an obstacle avoidance neural network; to plan multiple sub-target positions based on the obstacle avoidance path using the obstacle avoidance neural network; and to plan the posture that the robotic arm needs to perform at each sub-target position.

[0082] Optionally, the processor 120 is specifically used to plan multiple sub-target positions by taking the distance between the multiple sub-target positions as the target step length, including: obtaining multiple step lengths for the robotic arm to move along the obstacle avoidance path, wherein the computation time required for the robotic arm to plan the sub-target positions along the obstacle avoidance path based on each of the multiple step lengths is within a preset time range; obtaining the success rate of the robotic arm in avoiding obstacles along the obstacle avoidance path based on each of the multiple step lengths, and obtaining multiple success rates corresponding to the multiple step lengths; selecting the step length with the highest success rate among the multiple step lengths as the target step length; and planning multiple sub-target positions based on the target step length and the obstacle avoidance path. Optionally, the processor 120 is specifically used to acquire multiple point cloud images, each containing a set number of point cloud data, and each point cloud data containing corresponding three-dimensional coordinates and depth values; wherein, different point cloud images are point cloud images captured by depth cameras located at different positions of the robotic arm; multiple point cloud images are stitched together to obtain a spatial point cloud image in which the robotic arm is located; based on a preset image algorithm and the three-dimensional coordinates and depth values ​​corresponding to the point cloud data contained in the spatial point cloud image, the target relative position between the robotic arm and the target obstacle is obtained.

[0083] Optionally, the processor 120 is specifically used to acquire multiple sets of simulated obstacle avoidance data, wherein the simulated obstacle avoidance data includes the simulated initial pose of the robotic arm, the simulated target pose, the simulated obstacle avoidance action sequence, and the simulated relative position between the simulated obstacle and the robotic arm, wherein each set of simulated obstacle avoidance data corresponds to a certain weight; the processor uses multiple sets of simulated obstacle avoidance data to perform initial training on an initial neural network without prior knowledge, thereby obtaining a trained initial neural network, wherein the initial training on the initial neural network without prior knowledge is a process of the initial neural network learning patterns based on multiple sets of simulated obstacle avoidance data; the processor inputs the simulated initial pose, simulated target pose, and simulated relative position from the multiple sets of simulated obstacle avoidance data into the trained initial neural network to obtain multiple new obstacle avoidance action sequences; the processor compares the multiple simulated obstacle avoidance action sequences from the multiple sets of simulated obstacle avoidance data with the multiple new simulated obstacle avoidance action sequences to obtain a comparison result; based on the comparison result, the processor adjusts the parameters of the trained initial neural network to obtain the obstacle avoidance neural network.

[0084] Optionally, the obstacle avoidance sequence includes an obstacle avoidance path for the robotic arm to avoid the target obstacle. The obstacle avoidance path includes multiple sub-target positions, and each sub-target position includes the posture that the robotic arm needs to perform at that sub-target position. The controller 130 is specifically used to control the end effector of the robotic arm to reach each sub-target position and perform the posture that needs to be performed at that sub-target position based on the sequential order of the multiple sub-target positions in the obstacle avoidance path, until the robotic arm is in the target pose.

[0085] Optionally, the robot 100 also includes multiple depth cameras 140 connected to the processor 120. The multiple depth cameras 140 are located at different positions of the robotic arm to capture point cloud images of the robotic arm from different directional perspectives. The processor 120 is also used to acquire multiple point cloud images, each containing a set number of point cloud data, and each point cloud data containing corresponding three-dimensional coordinates and depth values. The processor 120 is also used to stitch together multiple point cloud images to obtain a spatial point cloud image of the robotic arm. Based on a preset image algorithm and the three-dimensional coordinates and depth values ​​corresponding to the point cloud data contained in the spatial point cloud image, the processor 120 obtains the target relative position between the robotic arm and the target obstacle.

[0086] The robot 100 can be a surgical robot, an endoscope robot, etc. Furthermore, the robot is not limited to medical robots; it can be any robot requiring obstacle avoidance and containing a robotic arm. The robotic arm 110 can be designed with different mechanical structures depending on the function of the robot 100. The specific structure of the robotic arm 110 is well known to those skilled in the art and will not be elaborated upon here.

[0087] The processor 120 may be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a central processing unit (CPU), network processor (NP), microprocessor, etc.; it can also be a digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. Alternatively, the processor 120 can also be any conventional processor.

[0088] The robot 100 provided in this application embodiment has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0089] This application embodiment also provides a model training device 200, such as... Figure 5 As shown. The model training device 200 includes: an acquisition module 210 and a training module 220.

[0090] The acquisition module 210 is used to acquire multiple sets of simulated obstacle avoidance data. The simulated obstacle avoidance data includes the simulated initial pose of the robotic arm, the simulated target pose, the simulated obstacle avoidance action data sequence, and the simulated relative position between the simulated obstacle and the robotic arm. Each set of simulated obstacle avoidance data has a corresponding weight.

[0091] The training module 220 is used to initially train an initial neural network without prior knowledge using multiple sets of simulated obstacle avoidance data, resulting in a trained initial neural network. This initial training of the initial neural network without prior knowledge is a process of learning patterns based on multiple sets of simulated obstacle avoidance data. The simulated initial pose, simulated target pose, and simulated relative position from the multiple sets of simulated obstacle avoidance data are input into the trained initial neural network to obtain multiple new obstacle avoidance action sequences. These sequences are compared with the new simulated obstacle avoidance action sequences from the multiple sets of simulated obstacle avoidance data to obtain a comparison result. Based on the comparison result, the parameters of the trained initial neural network are adjusted to obtain the obstacle avoidance neural network.

[0092] In one implementation, the process of the acquisition module 210 acquiring multiple sets of simulated obstacle avoidance data may be as follows: acquiring multiple sets of obstacle avoidance sample data, each set of obstacle avoidance sample data containing the corresponding simulated initial pose, simulated target pose, simulated obstacle avoidance action sequence, and simulated relative position; assigning weights to the simulated obstacle avoidance action sequence corresponding to each set of obstacle avoidance sample data to obtain multiple sets of simulated obstacle avoidance data; wherein, a first weight is assigned to the obstacle avoidance sample data corresponding to the best simulated obstacle avoidance action sequence with an evaluation score greater than a set score line, and a second weight is assigned to the obstacle avoidance sample data corresponding to the non-best simulated obstacle avoidance action data with an evaluation score not greater than a set score line, wherein the first weight is greater than the second weight.

[0093] The model training device 200 provided in this application embodiment has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0094] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0095] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0096] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0097] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A robot obstacle avoidance method, characterized in that, Applied to a robot, the robot including a robotic arm, the method includes: Upon receiving a start command instructing the robotic arm to deploy, the system acquires the target relative position between the robotic arm and the target obstacle, the current pose of the robotic arm, and the target pose of the robotic arm; wherein the current pose represents the current position and current posture of the robotic arm, and the target pose represents the target position and target posture that the robotic arm is required to be in after deployment. The relative position of the target, the current pose, and the target pose are processed using a pre-trained obstacle avoidance neural network to obtain an obstacle avoidance action sequence; wherein, the obstacle avoidance action sequence is used to instruct the robotic arm to deploy from the current pose to avoid the target obstacle and to be in the target pose; Based on the obstacle avoidance sequence, the robotic arm is controlled to unfold from the current pose and be in the target pose; Specifically, a pre-trained obstacle avoidance neural network is used to process the target's relative position, the current pose, and the target pose to obtain an obstacle avoidance action sequence, including: The obstacle avoidance neural network is used to determine an obstacle avoidance path based on the target's relative position, the current pose, and the target's pose. Using the obstacle avoidance neural network based on the obstacle avoidance path, multiple sub-target positions are planned, and in each sub-target position, the posture that the robotic arm needs to perform at that sub-target position is planned; The distance between the multiple sub-target locations is the target step size. Multiple sub-target locations are planned, including: The robot arm moves along the obstacle avoidance path by obtaining multiple step lengths, and the calculation time required for the robot arm to plan the sub-target position along the obstacle avoidance path based on each of the multiple step lengths is within a preset time range; The success rate of the robotic arm in avoiding obstacles along the obstacle avoidance path based on each of the plurality of step lengths is obtained, and a plurality of success rates corresponding to the plurality of step lengths are obtained. The step with the highest success rate among the multiple step sizes is selected as the target step size; Based on the target step size and the obstacle avoidance path, multiple sub-target positions are planned.

2. The method according to claim 1, characterized in that, in, The obstacle avoidance sequence includes an obstacle avoidance path for the robotic arm to avoid the target obstacle. The obstacle avoidance path includes multiple sub-target positions, and each sub-target position includes the posture that the robotic arm needs to perform at that sub-target position. The step of controlling the robotic arm to unfold from the current pose and be in the target pose based on the obstacle avoidance action sequence includes: Based on the sequential order of the multiple sub-target positions in the obstacle avoidance path, the robotic arm end effector is sequentially controlled to reach each sub-target position and execute the required posture at that sub-target position until the robotic arm is in the target pose.

3. The method according to claim 1, characterized in that, Obtaining the target relative position between the robotic arm and the target obstacle includes: Multiple point cloud images are acquired, each containing a set number of point cloud data, and each point cloud data contains corresponding three-dimensional coordinates and depth values; wherein, different point cloud images are point cloud images captured by depth cameras located at different positions of the robotic arm. By stitching together the multiple point cloud images, a spatial point cloud image of the robotic arm is obtained. Based on a preset image algorithm and the three-dimensional coordinates and depth values ​​corresponding to the point cloud data contained in the spatial point cloud image, the target relative position between the robotic arm and the target obstacle is obtained.

4. The method according to claim 1, characterized in that, The method further includes: Multiple sets of simulated obstacle avoidance data are acquired, wherein the simulated obstacle avoidance data includes the simulated initial pose of the robotic arm, the simulated target pose, the simulated obstacle avoidance action sequence, and the simulated relative position between the simulated obstacle and the robotic arm, wherein each set of simulated obstacle avoidance data has a corresponding weight. The initial neural network without prior knowledge is initially trained using the multiple sets of simulated obstacle avoidance data to obtain a trained initial neural network. The initial training of the initial neural network without prior knowledge is the process by which the initial neural network learns patterns based on the multiple sets of simulated obstacle avoidance data. The simulated initial pose, the simulated target pose, and the simulated relative position from the multiple sets of simulated obstacle avoidance data are input into the trained initial neural network to obtain multiple new obstacle avoidance action sequences. By comparing multiple simulated obstacle avoidance action sequences with multiple new obstacle avoidance action sequences in the multiple sets of simulated obstacle avoidance data, a comparison result is obtained; Based on the comparison results, the parameters of the trained initial neural network are adjusted to obtain the obstacle avoidance neural network.

5. A model training method, characterized in that, The obstacle avoidance neural network according to any one of claims 1-4 is trained in the following manner: Multiple sets of simulated obstacle avoidance data are acquired, wherein the simulated obstacle avoidance data includes the simulated initial pose of the robotic arm, the simulated target pose, the simulated obstacle avoidance action sequence, and the simulated relative position between the simulated obstacle and the robotic arm; The initial neural network without prior knowledge is initially trained using the multiple sets of simulated obstacle avoidance data to obtain a trained initial neural network. The initial training of the initial neural network without prior knowledge is the process by which the initial neural network learns patterns based on the multiple sets of simulated obstacle avoidance data. The simulated initial pose, the simulated target pose, and the simulated relative position from the multiple sets of simulated obstacle avoidance data are input into the trained initial neural network to obtain multiple new obstacle avoidance action sequences. By comparing multiple simulated obstacle avoidance action sequences with multiple new obstacle avoidance action sequences in the multiple sets of simulated obstacle avoidance data, a comparison result is obtained; Based on the comparison results, the parameters of the trained initial neural network are adjusted to obtain the obstacle avoidance neural network.

6. The method according to claim 5, characterized in that, Acquire multiple sets of simulated obstacle avoidance data, including: Multiple sets of obstacle avoidance sample data are acquired. Each set of obstacle avoidance sample data includes the corresponding simulated initial pose, simulated target pose, simulated obstacle avoidance action sequence, and simulated relative position. Weights are assigned to the simulated obstacle avoidance action sequences corresponding to each set of obstacle avoidance sample data to obtain the multiple sets of simulated obstacle avoidance data; wherein, the obstacle avoidance sample data corresponding to the best simulated obstacle avoidance action sequence with an evaluation score greater than a set score line is assigned a first weight, and the obstacle avoidance sample data corresponding to the non-best simulated obstacle avoidance action sequence with an evaluation score not greater than the set score line is assigned a second weight, wherein the first weight is greater than the second weight.

7. A model training device, characterized in that, The apparatus for training the obstacle avoidance neural network according to any one of claims 1-4, the apparatus comprising: The acquisition module is used to acquire multiple sets of simulated obstacle avoidance data, wherein the simulated obstacle avoidance data includes the simulated initial pose of the robotic arm, the simulated target pose, the simulated obstacle avoidance action data sequence, and the simulated relative position between the simulated obstacle and the robotic arm; The training module is used to initially train an initial neural network without prior knowledge using the multiple sets of simulated obstacle avoidance data to obtain a trained initial neural network. The initial training of the initial neural network without prior knowledge is a process in which the initial neural network learns patterns based on the multiple sets of simulated obstacle avoidance data. The simulated initial pose, simulated target pose, and simulated relative position from the multiple sets of simulated obstacle avoidance data are input into the trained initial neural network to obtain multiple new obstacle avoidance action sequences. The multiple simulated obstacle avoidance action sequences in the multiple sets of simulated obstacle avoidance data are compared with the multiple new obstacle avoidance action sequences to obtain a comparison result. Based on the comparison result, the parameters of the trained initial neural network are adjusted to obtain the obstacle avoidance neural network.

8. A robot, characterized in that, The robot includes: robotic arm; A processor, upon receiving a start command instructing the robotic arm to deploy, acquires the target relative position between the robotic arm and a target obstacle, the current pose of the robotic arm, and the target pose of the robotic arm; wherein the current pose represents the current position and current posture of the robotic arm, and the target pose represents the target position and target posture that the robotic arm should be in after deployment; processes the target relative position, the current pose, and the target pose using a pre-trained obstacle avoidance neural network to obtain an obstacle avoidance action sequence; wherein the obstacle avoidance action sequence is used to instruct the robotic arm to deploy from the current pose, avoiding the target obstacle, and to be in the target pose; wherein the processor is configured to execute the method as described in any one of claims 1-4; A controller, connected to the processor and the robotic arm, is used to control the robotic arm to unfold from the current pose and be in the target pose based on the obstacle avoidance action sequence.

9. The robot according to claim 8, characterized in that, The robot also includes: Multiple depth cameras are connected to the processor; the multiple depth cameras are located at different positions of the robotic arm to capture point cloud images of the robotic arm from different directional perspectives. The processor is also configured to acquire multiple point cloud images, each containing a set number of point cloud data, each point cloud data containing corresponding three-dimensional coordinates and depth values; and to stitch the multiple point cloud images to obtain a spatial point cloud image in which the robotic arm is located; and to obtain the target relative position between the robotic arm and the target obstacle based on a preset image algorithm and the three-dimensional coordinates and depth values ​​corresponding to the point cloud data contained in the spatial point cloud image.

Images