Robot path planning device based on deep learning

Abstract

A robot path planning device based on deep learning. The device comprises: an initial-path generation apparatus, an object detection apparatus, an obstacle analysis apparatus and an optimal-path generation apparatus, wherein the initial-path generation apparatus is used for generating an initial path on the basis of target point information and global path information; the object detection apparatus is used for performing object detection on environment information, so as to acquire three-dimensional environment information; the obstacle analysis apparatus is used for analyzing an obstacle on the initial path on the basis of the three-dimensional environment information, so as to acquire an analysis result; and the optimal-path generation apparatus is used for generating an optimal path on the basis of the analysis result.

Description

Robot path planning device based on deep learning TECHNICAL FIELD

[0001] The present application belongs to the technical field of robots, and particularly relates to a robot path planning device based on deep learning. BACKGROUND

[0002] The existing robot path planning system can obtain the current driving environment of the mobile robot through the perception module; the decision module is responsible for deciding the action (forward, backward, etc.) of the mobile robot in the current environment. Although DRL has achieved good results in many fields, there are still great challenges in actual engineering implementation. And the rapid development of automation technology, the robot system in this field is an automatic operation system composed of robots and peripheral equipment and tools, and the robot inevitably needs to move between multiple stations during the automatic operation. The movement of the robot in the prior art is mostly according to the set path, so that it cannot adapt to different working conditions. Therefore, there is an urgent need for a robot path planning device based on deep learning.

[0003] SUMMARY

[0004] To solve the above technical problems, the present application provides a robot path planning device based on deep learning, which can automatically plan the path for obstacles existing on the moving path according to different working conditions.

[0005] To achieve the above purpose, the present application provides a robot path planning device based on deep learning, comprising:

[0006] An initial path generation device, a target detection device, an obstacle analysis device and an optimal path generation device;

[0007] The initial path generation device is used to generate an initial path according to target point information and global path information;

[0008] The target detection device is used to detect targets in the environment information and obtain three-dimensional environment information;

[0009] The analysis device is used to analyze obstacles on the initial path according to the three-dimensional environment information and obtain an analysis result;

[0010] The optimal path generation device is used to generate an optimal path according to the analysis result.

[0011] Optionally, the initial path generation device comprises a visual sensor;

[0012] The visual sensor is used to obtain video information around the robot body and obtain an initial path according to the video information.

[0013] Optionally, the target detection device comprises an obstacle detection device and an image synthesis device.

[0014] The obstacle detection device is configured to detect the obstacle contour feature.

[0015] The image synthesis device is configured to acquire the obstacle three-dimensional information according to the obstacle contour feature.

[0016] Optionally, the obstacle detection device comprises an obstacle model.

[0017] The obstacle model is trained based on pictures in the video information, and the pictures are obtained by performing gray value mapping on pictures intercepted from the video information.

[0018] Optionally, the analysis device comprises an obstacle analysis device.

[0019] The analysis model is trained based on a neural network model and training data, and the training set is composed of training signal sequences and artificial labels corresponding to the training signal sequences.

[0020] Optionally, the obstacle analysis device comprises a ground plane segmentation unit, a ground plane height calculation unit and a judgment unit.

[0021] The ground plane segmentation unit is configured to segment the position information in the environment three-dimensional information to acquire ground plane information.

[0022] The ground plane height calculation unit is configured to calculate the ground plane height and the height gradient according to the ground plane information.

[0023] The judgment unit is configured to judge whether the robot can pass through the obstacle according to the ground plane height and the height gradient.

[0024] Optionally, the judgment unit comprises a top judgment unit and a bottom judgment unit.

[0025] The top judgment unit is configured to perform top passability judgment according to the calculated height and the robot height.

[0026] The bottom judgment unit is configured to perform bottom passability judgment according to the calculated height gradient and the maximum passing angle of the robot body.

[0027] Compared with the prior art, the present application has the following advantages and technical effects:

[0028] The robot in the application has a visual sensor, and environment three-dimensional information can be generated through a target detection module; meanwhile, whether the robot can pass through the obstacle is determined according to the environment three-dimensional information, and an obstacle analysis model based on a neural network model and trained by training data is used when analyzing whether the robot can pass through, the robot whether can pass through the obstacle is judged by using the top passability judgment and the bottom passability judgment, so that the analysis result is more accurate, and finally the optimal path is generated. BRIEF DESCRIPTION OF DRAWINGS

[0029] The drawings constituting a part of this application are used to provide further understanding of the application, the illustrative embodiments of the application and the description thereof are used to explain the application, and do not constitute improper limitation on the application. In the drawings:

[0030] Fig. 1 is a structure diagram of a robot path planning device based on deep learning according to an embodiment of the application. DETAILED DESCRIPTION

[0031] It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict. The application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0032] It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0033] The application proposes a robot path planning device based on deep learning, as shown in Fig. 1, specifically comprising: an initial path generation device, a target detection device, an obstacle analysis device and an optimal path generation device;

[0034] The initial path generation device is used to generate an initial path according to target point information and global path information;

[0035] The target detection device is used to detect targets for environment information and obtain environment three-dimensional information;

[0036] The analysis device is used to analyze obstacles on the initial path according to the environment three-dimensional information and obtain an analysis result;

[0037] The optimal path generation device is used to generate an optimal path according to the analysis result.

[0038] Further, the initial path generation device comprises a visual sensor;

[0039] The visual sensor is used to obtain video information around the robot body, and obtain the initial path according to the video information;

[0040] Specifically, the robot obtains environment information in the advancing direction of the robot body through a vision sensor arranged on the robot body.

[0041] Further, the target detection device comprises: an obstacle detection device and an image synthesis device.

[0042] The obstacle detection device is configured to detect the contour feature of the obstacle.

[0043] The image synthesis device is configured to obtain three-dimensional information of the obstacle according to the contour feature of the obstacle.

[0044] Specifically, the target detection module performs target detection operation on the environment information and judges the validity of the path, and whether the obstacle detection operation is needed is set to judge whether to connect to the data transmission module. When connected, the video data output by the vision sensor is started to be received, and then it is judged whether the obstacle detection operation is needed. When the obstacle detection is not needed, the video data is directly displayed, and when the obstacle detection is needed, the obstacle detection operation is started, the operation result is marked on the video data and displayed, and the three-dimensional information of the environment is generated.

[0045] The vision sensor directly obtains the video information in front of the robot, and the target detection module obtains the video information and intercepts multiple groups of adjacent frames as the judgment pictures. The judgment pictures of different angles and illuminations are synthesized into one tiff format picture, so that the contour feature of the obstacle is more obvious. Then, a deep learning method is applied to train an obstacle model to detect the obstacle contour in the image.

[0046] After the image synthesis, the target detection algorithm is used to load the synthesized image into the pre-set network framework for training. The obstacle contours calculated by the multiple groups of judgment pictures are matched in three dimensions to form the three-dimensional information of the obstacle.

[0047] Further, the obstacle detection device comprises: an obstacle model.

[0048] The obstacle model is trained based on the pictures in the video information, and the pictures are obtained by mapping the gray values of the intercepted pictures in the video information.

[0049] Specifically, the gray values of the detail contour tiff format pictures of the judgment pictures are mapped from 0-1 to 0-250 to obtain png format pictures. All the collected pictures are classified, and are divided into a training set and a test set in a ratio of 7:3. The training set is manually labeled, and the labels are classified according to the defect types. After the labeling is completed, the labeled pictures are imported into the target detection algorithm for training. After the training is completed, the training model is obtained.

[0050] Further, the analysis device comprises: an obstacle analysis device.

[0051] The analysis model is trained based on a neural network model and using training data, and the training set is composed of training signal sequences and artificial annotations corresponding to the training signal sequences.

[0052] Specifically, sample data of specified data is called from a preset sample database to form a sample set, and the sample set is divided into a training set and a validation set according to a preset ratio; wherein the sample data is composed of pre-collected signal sequences and artificial annotations corresponding to the pre-collected signal sequences.

[0053] A preset neural network model is called, and the training set is input into the neural network model for training, thereby obtaining a preliminary neural network model;

[0054] The preliminary neural network model is verified using the validation set to obtain a verification result;

[0055] It is determined whether the verification result is a verification pass;

[0056] If the verification result is a verification pass, the preliminary neural network model is recorded as an obstacle analysis model.

[0057] Further, the obstacle analysis device comprises a ground plane segmentation unit, a ground plane height calculation unit and a judgment unit.

[0058] The ground plane segmentation unit is used to segment according to the position information in the environmental three-dimensional information to obtain ground plane information.

[0059] The ground plane height calculation unit is used to calculate the ground plane height and the gradient of the height according to the ground plane information.

[0060] The judgment unit is used to determine whether the robot can pass through the obstacle according to the ground plane height and the gradient of the height.

[0061] Specifically, the normal vector information is calculated according to the position information of each information unit in the environmental three-dimensional information, and the ground plane is segmented; the height of each information unit in the environmental three-dimensional information relative to the ground plane and the gradient of the height are calculated; the advancement direction of the robot body is determined to be unable to pass through the obstacle according to the calculated height and the gradient of the height.

[0062] Further, the judgment unit comprises a top judgment unit and a bottom judgment unit.

[0063] The top judgment unit is used to make a top passability judgment according to the calculated height and the height of the robot.

[0064] The bottom judgment unit is used to make a bottom passability judgment according to the calculated gradient of the height and the maximum pass angle of the robot body.

[0065] Specifically, the passing judgment type is determined according to the normal direction of the environmental three-dimensional information; if the passing judgment type is top passing judgment, the top passing judgment is performed according to the calculated height and the height of the robot to determine the obstacle that the robot cannot pass; if the passing judgment type is bottom passing judgment, the bottom passing judgment is performed according to the gradient of the calculated height and the maximum passing angle of the robot body to determine the obstacle that the robot cannot pass.

[0066] Each time the environmental three-dimensional information and the obstacle three-dimensional information are generated, the corresponding judgment picture is stored in the comparison library at the same time, and each time the environmental three-dimensional information and the obstacle three-dimensional information are generated, the data in the comparison library is searched first, if consistent, it is directly called, otherwise it is regenerated.

[0067] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited to this, any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A robot path planning device based on deep learning, characterized in that, include: Initial path generation device, target detection device, obstacle analysis device, and optimal path generation device; The initial path generation device is used to generate an initial path based on the target point information and the global path information; The target detection device is used to detect targets in the environment and acquire three-dimensional environmental information; The analysis device is used to analyze obstacles on the initial path based on the three-dimensional environmental information and obtain analysis results; The optimal path generation device is used to generate the optimal path based on the analysis results.

2. The robot path planning device based on deep learning according to claim 1, characterized in that, The initial path generation device includes: a visual sensor; The visual sensor is used to acquire video information around the robot's body, and to obtain an initial path based on the video information.

3. The robot path planning device based on deep learning according to claim 2, characterized in that, The target detection device includes: an obstacle detection device and an image synthesis device; The obstacle detection device is used to detect the outline features of obstacles; The image synthesis device is used to acquire three-dimensional information of obstacles based on their outline features.

4. The robot path planning device based on deep learning according to claim 3, characterized in that, The obstacle detection device includes: an obstacle model; The obstacle model is trained based on images from the video information, which are obtained by mapping grayscale values ​​of images extracted from the video information.

5. The robot path planning device based on deep learning according to claim 3, characterized in that, The analysis device includes: an obstacle analysis device; The analytical model is based on a neural network model and trained using training data. The training set consists of... The training signal sequence consists of a training signal sequence and corresponding manual annotations.

6. The robot path planning device based on deep learning according to claim 5, characterized in that, The obstacle analysis device includes: a ground plane segmentation unit, a ground plane height calculation unit, and a judgment unit; The ground plane segmentation unit is used to segment the ground plane based on the location information in the three-dimensional environmental information to obtain ground plane information; The ground plane height calculation unit is used to calculate the ground plane height and the height gradient based on the ground plane information; The judgment unit is used to determine whether the robot can pass through the obstacle based on the ground plane height and the height gradient.

7. The robot path planning device based on deep learning according to claim 6, characterized in that, The judgment unit includes: a top judgment unit and a bottom judgment unit; The top judgment unit is used to determine the top passability based on the calculated height and the robot height; The bottom judgment unit is used to judge the bottom passability based on the calculated height gradient and the maximum passing angle of the robot body.

Images