Robot positioning method and system based on deep learning

Abstract

The invention discloses a robot positioning method and system based on deep learning. Relates to the technical field of artificial intelligence. The method comprises the following steps: sending an obtained global environment map and a local environment map under the global environment map into a trained deep neural network model to obtain an accurate candidate pose of a robot; fine correction is conducted on the candidate poses through robot motion, and finally the optimal robot pose is obtained; the deep neural network model comprises a convolutional layer conv0, a maximum value pooling layer maxpool, a convolutional layer conv, a cross attention layer crossattn and a candidate pose acquisition layer Maxop; the maximum pooling layer maxpool and the convolution layer conv are connected in sequence to form a layer group, the multiple layer groups are connected in sequence, the layer group located on the uppermost layer serves as the shallowest scale to be connected to the output of the convolution layer conv0, and the output of the convolution layer conv0 and the output of all the layer groups are connected to the cross attention layer crossattn and the candidate pose obtaining layer Maxop in sequence so as to output the optimal pose of the robot; according to the robot positioning method and system, the accuracy and robustness of robot positioning are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a robot positioning method and system based on deep learning. Background Art

[0002] Robot positioning means that when an environmental map has been established, the robot obtains environmental information about the surrounding environment based on its own measurement equipment, and can accurately know its position and posture by comparing it with the environmental map information. Precise robot positioning is the key to achieving efficient autonomous navigation and task execution.

[0003] In the past 20 years, robot positioning technology has been a research hotspot in the field of machine vision and artificial intelligence, and many classic algorithms have emerged, such as the Monte Carlo positioning algorithm, the adaptive Monte Carlo positioning algorithm, the extended Kalman filter algorithm, and the cartographer-based positioning algorithm. These algorithms are mainly based on line laser radar to perceive the surrounding environment information, and then find the best position and posture through feature matching with the existing 2D grid environment map to complete the self-positioning operation. These positioning methods can achieve very good positioning in a smaller environment, but in a larger map environment, these methods cannot perform real-time high-precision positioning due to the large amount of calculation required. At the same time, these methods are easily disturbed by factors such as lighting and occlusion, resulting in large positioning errors. Summary of the invention

[0004] Based on the technical problems existing in the background technology, the present invention proposes a robot positioning method and system based on deep learning, which improves the accuracy and robustness of robot positioning.

[0005] The present invention proposes a robot positioning method based on deep learning, comprising the following steps:

[0006] The obtained global environment map and the local environment map under the global environment map are sent to the trained deep neural network model to obtain the accurate candidate poses of the robot;

[0007] Finely correct the candidate postures through robot movement, and finally obtain the optimal robot posture;

[0008] The deep neural network model includes a convolution layer conv0, a maximum pooling layer maxpool, a convolution layer conv, a cross attention layer cross_attn and a candidate pose acquisition layer Maxop;

[0009] The maximum pooling layer maxpool and the convolution layer conv are connected in sequence to form a layer group. Multiple layer groups are connected in sequence. The layer group at the top is connected to the output of the convolution layer conv0 as the shallowest scale. The convolution layer conv0 inputs the 2D grid global environment map and the local environment map. The convolution layer conv0 and the convolution layer conv in each layer group output the multi-scale feature vector of the global environment map and the multi-scale feature vector of the local environment map respectively.

[0010] The multi-scale feature vector of the global environment map and the multi-scale feature vector of the local environment map are sequentially input into the cross attention layer cross_attn and the candidate pose acquisition layer Maxop to output the optimal pose of the robot.

[0011] Furthermore, the training process of the deep neural network model is as follows:

[0012] Obtain a 2D grid global environment map and a local environment map under the 2D grid global environment map to form a training sample data pair, and set a label for the training sample data pair;

[0013] The target loss function is constructed based on the weighted mean square error loss function and the cross entropy loss function;

[0014] The training sample data pairs are input into the deep neural network model to learn the model parameters to train the deep neural network model.

[0015] Furthermore, the process of obtaining the local environment map under the 2D grid global environment map is as follows:

[0016] In the 2D grid global environment map, randomly select a posture without obstacles, use the posture as the reference origin to emit rays in all directions, record the first intersection posture of each ray and the obstacle in the 2D grid global environment map, and collect the intersection posture information of all rays;

[0017] Obtain the minimum bounding rectangle of all intersection positions according to the standard of containing 95% of the intersection positions;

[0018] The 2D grid global environment map is cropped according to the minimum bounding rectangle, and the obtained local map is used as the local environment map of the robot at the current position;

[0019] 5% of the random obstacle positions are erased and added to the local environment map to obtain the local environment map under the 2D grid global environment map.

[0020] Furthermore, the labeling of the training sample data is specifically as follows:

[0021] Create a pure black background image of the same size as the 2D grid global environment map, with the reference origin as the center and a circle with a radius of 10 pixels according to the Gaussian distribution as the supervision label of the current pose.

[0022] Further, each training sample data pair includes a 2D grid global environment map, a local environment map, and a supervision label corresponding to the local environment map;

[0023] A plurality of training sample data pairs are formed based on local environment maps of different positions, thereby constituting a training data set to serve as training data for a deep neural network model.

[0024] A robot positioning system based on deep learning, which sends the acquired global environment map and the local environment map under the global environment map into the trained deep neural network model to obtain the optimal posture of the robot;

[0025] The deep neural network model includes a convolution layer conv0, a maximum pooling layer maxpool, a convolution layer conv1, a cross attention layer cross_attn and a candidate pose acquisition layer Maxop;

[0026] The maximum pooling layer maxpool and the convolution layer conv are connected in sequence to form a layer group. Multiple layer groups are connected in sequence. The layer group at the top is connected to the output of the convolution layer conv0 as the shallowest scale. The convolution layer conv0 inputs the 2D grid global environment map and the local environment map. The convolution layer conv0 and the convolution layer conv in each layer group output the multi-scale feature vector of the global environment map and the multi-scale feature vector of the local environment map respectively.

[0027] The multi-scale feature vector of the global environment map and the multi-scale feature vector of the local environment map are sequentially input into the cross attention layer cross_attn and the candidate pose acquisition layer Maxop to output the optimal pose of the robot.

[0028] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the robot positioning method as described above is implemented when the processor executes the computer program.

[0029] A computer-readable storage medium stores a plurality of classification programs, and the plurality of classification programs are used to be called by a processor and execute the robot positioning method as described above.

[0030] The advantages of a robot positioning method and system based on deep learning provided by the present invention are: with the help of the powerful expression ability of the deep neural network model, the precise position and posture of the input local environment map can be directly obtained end-to-end; with the help of a multi-scale feature step-by-step positioning prediction strategy, while significantly improving the positioning operation speed, the positioning accuracy of the local environment map can be greatly improved; the entire network model has strong versatility and fast recognition speed, and is highly applicable and robust to robot positioning needs in various scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the process of the present invention;

[0032] Figure 2 This is a structural diagram of a deep neural network model;

[0033] Figure 3 This is an example of a 2D grid global environment map. DETAILED DESCRIPTION

[0034] Below, the technical solution of the present invention is described in detail through specific embodiments. Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific implementation disclosed below.

[0035] In recent years, with the development of deep learning technology, artificial intelligence technology has gradually been applied to the field of image similarity acquisition. With the help of deep learning technologies such as convolutional neural networks, attention neural networks, and transformer networks, significant differential features of two images can be extracted, and the matching degree or similarity of the two images can be directly output, which greatly improves the speed and accuracy of feature matching.

[0036] In view of the advantages and disadvantages of various existing robot positioning technologies, this embodiment proposes a robot positioning method based on deep learning. With the help of artificial intelligence technology, it directly realizes the rapid matching and positioning of local environment maps and global environment maps, greatly improving the accuracy and robustness of robot positioning.

[0037] This embodiment is applicable to scenarios where the robot needs to be accurately positioned in any form of 2D environment map. For the sake of convenience, this embodiment takes a 2D laser radar robot and a 2D grid global environment map as an example. Figure 3 shown.

[0038] This embodiment uses a convolutional neural network (CNN) structure to design a deep neural network model. In order to facilitate the description of this embodiment, some terms are defined: feature map resolution refers to feature map width × feature map height, feature map size refers to feature map channel number × feature map width × feature map height, kernel size refers to kernel width × kernel height, span refers to width direction span × height direction span, and each convolution layer is followed by a batch normalization layer and a nonlinear activation layer.

[0039] like Figures 1 to 3 As shown, a robot positioning method based on deep learning proposed by the present invention comprises the following steps:

[0040] S1. Design a deep neural network model. The deep neural network model is the core of artificial intelligence technology. Its main function is to extract key details and features of the input environment map. The network structure of the deep neural network model designed in this embodiment is as follows: Figure 2 As shown, the globalmap layer represents a pre-built 2D grid global environment map; the localmap layer represents the local environment map of the current posture collected by the robot; the globalmap layer and the localmap layer are both single-channel grayscale images, which can be images of any input resolution. Generally, the resolution of the localmap layer image is much smaller than that of the globalmap layer image. For the convenience of description, this embodiment refers to the resolution of the input image as WxH.

[0041] The deep neural network model includes a convolution layer conv0, a maximum pooling layer maxpool, a convolution layer conv, a cross attention layer cross_attn and a candidate pose acquisition layer Maxop; the maximum pooling layer maxpool and the convolution layer conv are connected in sequence to form a layer group, and multiple layer groups are connected in sequence. The layer group at the top is connected to the output of the convolution layer conv0 as the shallowest scale. The convolution layer conv0 inputs a 2D grid global environment map and a local environment map. The convolution layer conv0 and the convolution layer conv in each layer group output a multi-scale feature vector of the global environment map and a multi-scale feature vector of the local environment map, respectively.

[0042] The conv layers in different layer groups correspond to conv0, conv1, conv2, conv3, and convn layers, all of which are convolutional layers with a kernel size of 3x3 and a span of 1x1; the maxpool layers in different layer groups correspond to maxpool1, maxpool2, maxpool3, and maxpooln layers, all of which are maximum pooling layers with a kernel size of 2x2 and a span of 2x2; ... represents any number of layer groups. According to the specific robot positioning environment, the appropriate number of network combination layers can be flexibly selected.

[0043] F0_g, F1_g, F2_g, F3_g, …, Fn_g represent the multi-scale feature vectors of the globalmap layer (global environment map) input image after being operated by conv0 layer, conv1 layer, conv2 layer, conv3 layer, …, convn layer; F0_l, F1_l, F2_l, F3_l, …, Fn_l represent the multi-scale feature vectors of the localmap layer (local environment map) input image after being operated by conv0 layer, conv1 layer, conv2 layer, conv3 layer, …, convn layer; the cross_attn layer represents the cross attention layer, which is used to calculate the similarity between the multi-scale feature vector of the input global environment map and the multi-scale feature vector of the local environment map. The specific calculation method is to use the multi-scale feature vector of the local environment map as the convolution kernel of the convolution operation, the multi-scale feature vector of the global environment map as the input feature layer, use the standard convolution operation, and the output feature vector is the similarity heat map, i.e., similarity map.

[0044] The Loss_0 layer, Loss_1 layer, Loss_2 layer, Loss_3 layer, ..., Loss_n layer are all supervised function layers, used for training deep neural network models, and correspond to the F0_g layer and F0_l layer, the F1_g layer and F1_l layer, the F2_g layer and F2_l layer, the F3_g layer and F3_l layer, ..., the Fn_g layer and Fn_l layer one by one. The Maxop layer is a candidate pose acquisition layer, used for prediction of deep neural network models, mainly based on the values ​​in the similarity heat map, selects the largest poses as the candidate poses of the current pose of the robot for output, that is, the candidate pose acquisition layer Maxop arranges all similarity heat maps in descending order, takes the first k poses as the candidate poses of the current pose of the robot for output.

[0045] S2. Training the deep neural network model mainly involves optimizing the parameters of the deep neural network model through a large amount of training sample data of paired environment map images with known posture information, so that the deep neural network model can accurately obtain the optimal posture of the local environment map perceived by the robot in the global environment map. The specific steps are as follows:

[0046] S21, obtaining training sample data pairs, mainly collecting various environment maps and local environment maps of various positions under the environment maps, and forming training data pairs. The specific steps are as follows:

[0047] S211. Collect environmental maps, mainly collecting 2D grid global environmental maps in various scenarios.

[0048] S212. Obtain the local environment map. The main method is to arbitrarily select a posture without obstacles in the 2D grid global environment map, and use the posture as the reference origin to emit rays in all directions, record the first intersection posture of each ray and the obstacle in the grid map, and collect the intersection posture information of all rays. Then, according to the standard of containing 95% of the ray intersections, obtain the minimum bounding rectangle of all ray intersection postures. Then, crop the grid map according to the minimum bounding rectangle. The obtained local map is the local environment map of the robot in the current posture. Finally, erase and add 5% of random obstacle postures to the local environment map.

[0049] S213. Obtain training data labels. First, create a pure background black image with the same size as the 2D grid global environment map. With the reference origin posture in step S212 as the center, draw a circle with a radius of 10 pixels according to the Gaussian distribution as the supervision label of the current posture.

[0050] S214, obtaining training sample data pairs, mainly combining the data images obtained in step S211, step S212, and step S213 as a training sample data pair, that is, each training sample data pair includes a 2D grid global environment map, a local environment map, and a supervision label corresponding to the local environment map; a plurality of training sample data pairs are formed based on local environment maps of different postures, thereby constituting a training data set to serve as training data for a deep neural network model.

[0051] S22. Design a target loss function for the deep neural network model. The loss functions used in the deep neural network designed in this embodiment are the mean square error loss function and the cross entropy loss function. These two loss functions are weighted according to a certain ratio to form the final target loss function.

[0052] S23. Training the deep neural network model mainly involves sending the training sample data pairs into the defined deep neural network model to learn the relevant model parameters.

[0053] S3. Using a deep neural network model, any 2D grid global environment map and the local environment map under the map are directly fed into the trained deep neural network model to output the best candidate pose corresponding to the local environment map. The specific prediction process is as follows:

[0054] S31. Obtaining multi-scale feature vectors of the global environment map mainly involves sending the 2D grid global environment map into a deep neural network model to obtain output feature vectors at various scales. During the use of the robot, the global environment map is fixed and unchanged. At the same time, the global environment map is relatively large and requires a large amount of computation. For any global environment map, this embodiment only needs to calculate the multi-scale feature vector once.

[0055] S32. Obtain a multi-scale feature vector of the local environment map, send the local environment map into a deep neural network model, and obtain output feature vectors at each scale.

[0056] S33. Obtain the best candidate pose. First, obtain the similarity heat map at the deepest scale, and then obtain the candidate pose through the Maxop layer. Since the error of the candidate pose at the deepest scale is large, the candidate pose at the deepest scale can be used as the reference pose origin as needed, and the corresponding pose origin of the pose origin on the shallower scale feature layer is found. Based on the corresponding pose origin, a local feature area is selected on the shallower scale feature layer, and the similarity heat map and Maxop operation of the local area are calculated to achieve fast and efficient acquisition of accurate candidate poses.

[0057] S34, obtaining the best posture. Since similar postures may exist in the 2D grid global environment map, the best candidate posture obtained in step S33 may not be unique. In this case, the robot needs to move, perform fine correction on the candidate posture, and finally obtain the best robot posture.

[0058] This embodiment makes use of the powerful expression ability of the deep neural network model to directly obtain the precise position and posture of the input local environment map from end to end; with the help of the multi-scale feature step-by-step positioning prediction strategy, it can greatly improve the positioning accuracy of the local environment map while significantly improving the positioning operation speed; the entire network model has strong versatility and fast recognition speed, and is highly applicable and robust to the robot positioning needs in various scenarios.

[0059] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A robot positioning method based on deep learning, characterized in that: The steps include: The obtained global environment map and the local environment map under the global environment map are sent to the trained deep neural network model to obtain the accurate candidate poses of the robot; Finely correct the candidate postures through robot movement, and finally obtain the optimal robot posture; The deep neural network model includes a convolution layer conv0, a maximum pooling layer maxpool, a convolution layer conv, a cross attention layer cross_attn and a candidate pose acquisition layer Maxop; The maximum pooling layer maxpool and the convolution layer conv are connected in sequence to form a layer group. Multiple layer groups are connected in sequence. The layer group at the top is connected to the output of the convolution layer conv0 as the shallowest scale. The convolution layer conv0 inputs the 2D grid global environment map and the local environment map. The convolution layer conv0 and the convolution layer conv in each layer group output the multi-scale feature vector of the global environment map and the multi-scale feature vector of the local environment map respectively. The multi-scale feature vector of the global environment map and the multi-scale feature vector of the local environment map are sequentially input into the cross attention layer cross_attn and the candidate pose acquisition layer Maxop to output the optimal pose of the robot.

2. The robot positioning method based on deep learning according to claim 1, characterized in that: The training process of the deep neural network model is as follows: Obtain a 2D grid global environment map and a local environment map under the 2D grid global environment map to form a training sample data pair, and set a label for the training sample data pair; The target loss function is constructed based on the weighted mean square error loss function and the cross entropy loss function; The training sample data pairs are input into the deep neural network model to learn the model parameters to train the deep neural network model.

3. The robot positioning method based on deep learning according to claim 2, characterized in that: The process of obtaining the local environment map under the 2D grid global environment map is as follows: In the 2D grid global environment map, randomly select a posture without obstacles, use the posture as the reference origin to emit rays in all directions, record the first intersection posture of each ray and the obstacle in the 2D grid global environment map, and collect the intersection posture information of all rays; Obtain the minimum bounding rectangle of all intersection positions according to the standard of containing 95% of the intersection positions; The 2D grid global environment map is cropped according to the minimum bounding rectangle, and the obtained local map is used as the local environment map of the robot at the current position; 5% of the random obstacle positions are erased and added to the local environment map to obtain the local environment map under the 2D grid global environment map.

4. The robot positioning method based on deep learning according to claim 3, characterized in that: The labeling of the training sample data is specifically as follows: Create a pure black background image of the same size as the 2D grid global environment map, with the reference origin as the center and a circle with a radius of 10 pixels according to the Gaussian distribution as the supervision label of the current pose.

5. The robot positioning method based on deep learning according to claim 4, characterized in that: Each training sample data pair includes a 2D grid global environment map, a local environment map, and a supervision label corresponding to the local environment map; A plurality of training sample data pairs are formed based on local environment maps of different positions, thereby constituting a training data set to serve as training data for a deep neural network model.

6. A robot positioning system based on deep learning, characterized in that: The obtained global environment map and the local environment map under the global environment map are sent to the trained deep neural network model to obtain the optimal posture of the robot; The deep neural network model includes a convolution layer conv0, a maximum pooling layer maxpool, a convolution layer conv1, a cross attention layer cross_attn and a candidate pose acquisition layer Maxop; The maximum pooling layer maxpool and the convolution layer conv are connected in sequence to form a layer group. Multiple layer groups are connected in sequence. The layer group at the top is connected to the output of the convolution layer conv0 as the shallowest scale. The convolution layer conv0 inputs the 2D grid global environment map and the local environment map. The convolution layer conv0 and the convolution layer conv in each layer group output the multi-scale feature vector of the global environment map and the multi-scale feature vector of the local environment map respectively. The multi-scale feature vector of the global environment map and the multi-scale feature vector of the local environment map are sequentially input into the cross attention layer cross_attn and the candidate pose acquisition layer Maxop to output the optimal pose of the robot.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the robot positioning method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of classification programs, and the plurality of classification programs are used to be called by a processor and execute the robot positioning method according to any one of claims 1 to 5.

Images