A robot positioning method and system based on deep learning

Abstract

The application discloses a robot positioning method and system based on deep learning, relates to the technical field of artificial intelligence, and sends a global environment map and a local environment map under the global environment map into a trained deep neural network model to obtain an accurate candidate pose of a robot; the candidate pose is finely corrected through robot movement, and finally the best robot pose is obtained; the deep neural network model comprises a convolution layer conv0, a maximum value pooling layer maxpool, a convolution layer conv, a cross attention layer cross_attn and a candidate pose acquisition layer Maxop; the maximum value pooling layer maxpool and the convolution layer conv are sequentially connected to form a layer group, a plurality of layer groups are sequentially connected, the layer group located at the uppermost layer is connected to the output of the convolution layer conv0 as the shallowest scale, and the output of the convolution layer conv0 and all layer groups are sequentially connected to the cross attention layer cross_attn and the candidate pose acquisition layer Maxop to output the best pose of the robot; the robot positioning method and system improve the accuracy and robustness of robot positioning.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a robot localization method and system based on deep learning. Background Technology

[0002] Robot localization refers to the process by which a robot, given an established environmental map, uses its own measurement equipment to acquire information about its surroundings and compares it with the map data to accurately determine its position and orientation. Precise robot localization is crucial for achieving efficient autonomous navigation and task execution.

[0003] For nearly 20 years, robot localization technology has been a research hotspot in the fields of machine vision and artificial intelligence, giving rise to many classic algorithms, such as Monte Carlo localization, adaptive Monte Carlo localization, extended Kalman filter, and Cartographer-based localization. These algorithms mainly rely on line LiDAR to perceive surrounding environmental information, and then perform feature matching with existing 2D grid environment maps to find the optimal pose and complete the localization operation. These localization methods can achieve very good localization in small environments, but in larger map environments, these methods require a large amount of computation and cannot achieve real-time high-precision localization. At the same time, these methods are easily affected by factors such as lighting and occlusion, resulting in large localization errors. Summary of the Invention

[0004] Based on the technical problems existing in the background technology, this invention proposes a robot localization method and system based on deep learning, which improves the accuracy and robustness of robot localization.

[0005] The present invention proposes a robot localization method based on deep learning, comprising the following steps:

[0006] The acquired global environment map and the local environment map under the global environment map are fed into the trained deep neural network model to obtain the robot's accurate candidate pose.

[0007] The candidate poses are finely corrected by the robot's motion, and the optimal robot pose is finally obtained.

[0008] The deep neural network model includes a convolutional layer conv0, a max pooling layer maxpool, a convolutional layer conv, a cross attention layer cross_attn, and a candidate pose acquisition layer Maxop;

[0009] The maxpooling layer and the convolutional layer conv are connected in sequence to form a layer group. Multiple layer groups are connected in sequence. The layer group at the top is connected to the output of the convolutional layer conv0 as the shallowest scale. The convolutional layer conv0 takes a 2D grid global environment map and a local environment map as input. The convolutional layer conv0 and the convolutional layer conv in each layer group output the multi-scale feature vector of the global environment map and the multi-scale feature vector of the local environment map, respectively.

[0010] The multi-scale feature vectors of the global environment map and the multi-scale feature vectors of the local environment map are sequentially input into the cross-attention layer cross_attn and the candidate pose acquisition layer Maxop to output the robot's optimal pose.

[0011] Furthermore, the training process of the deep neural network model is as follows:

[0012] Obtain a 2D raster global environment map and a local environment map under the 2D raster global environment map, form training sample data pairs, and assign labels to the training sample data pairs;

[0013] The target loss function is constructed by weighting the mean squared error loss function and the cross-entropy loss function;

[0014] The training sample data pairs are input into the deep neural network model to learn the model parameters in order to train the deep neural network model.

[0015] Furthermore, the process of acquiring the local environment map under the 2D raster global environment map is as follows:

[0016] In the 2D grid global environment map, arbitrarily select a pose without obstacles, and emit rays in all directions from the origin of the pose. Record the first intersection pose of each ray with the obstacle in the 2D grid global environment map, and collect the intersection pose information of all rays.

[0017] According to the standard that it can contain 95% of the intersection poses, obtain the minimum bounding rectangle of all intersection poses;

[0018] The 2D grid global environment map is clipped according to the minimum bounding rectangle, and the resulting local map is used as the local environment map of the robot in its current pose.

[0019] By erasing and adding 5% random obstacle poses to the local environment map, a local environment map is obtained under the 2D raster global environment map.

[0020] Furthermore, the process of assigning labels to the training sample data pairs specifically involves:

[0021] Create a pure black background map with the same size as the 2D raster global environment map. Draw a circle with a radius of 10 pixels centered at the origin and following a Gaussian distribution as the supervision label for the current pose.

[0022] Furthermore, each training sample data pair includes a 2D raster global environment map, a local environment map, and a supervision label corresponding to the local environment map;

[0023] Multiple training sample data pairs are formed from local environment maps in different poses, thus constituting a training dataset, which is used as training data for deep neural network models.

[0024] A robot localization system based on deep learning feeds the acquired global environment map and the local environment map under the global environment map into a trained deep neural network model to obtain the robot's optimal pose.

[0025] The deep neural network model includes a convolutional layer conv0, a max pooling layer maxpool, a convolutional layer conv1, a cross attention layer cross_attn, and a candidate pose acquisition layer Maxop;

[0026] The maxpooling layer and the convolutional layer conv are connected in sequence to form a layer group. Multiple layer groups are connected in sequence. The layer group at the top is connected to the output of the convolutional layer conv0 as the shallowest scale. The convolutional layer conv0 takes a 2D grid global environment map and a local environment map as input. The convolutional layer conv0 and the convolutional layer conv in each layer group output the multi-scale feature vector of the global environment map and the multi-scale feature vector of the local environment map, respectively.

[0027] The multi-scale feature vectors of the global environment map and the multi-scale feature vectors of the local environment map are sequentially input into the cross-attention layer cross_attn and the candidate pose acquisition layer Maxop to output the robot's optimal pose.

[0028] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the robot localization method as described above.

[0029] A computer-readable storage medium storing a plurality of classification programs, the plurality of classification programs being invoked by a processor and executed as described above in the robot localization method.

[0030] The advantages of the robot localization method and system based on deep learning provided by this invention are as follows: Leveraging the powerful expressive capabilities of deep neural network models, it can directly acquire the precise pose of the input local environment map end-to-end; Utilizing a multi-size feature-based hierarchical localization prediction strategy, it significantly improves the local environment map localization accuracy while greatly increasing the localization computation speed; The entire network model is highly versatile, has a fast recognition speed, and is highly applicable and robust to robot localization needs in various scenarios. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the process of the present invention;

[0032] Figure 2 This is a diagram of the deep neural network model structure.

[0033] Figure 3 This is an example image of a 2D raster global environment map. Detailed Implementation

[0034] The technical solution of the present invention will now be described in detail through specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0035] In recent years, with the development of deep learning technology, artificial intelligence technology has gradually been applied to the field of image similarity acquisition. With the help of deep learning technologies such as convolutional neural networks, attention neural networks, and transformer networks, significant difference features can be extracted from two images, and the matching degree or similarity between the two images can be directly output, which greatly improves the speed and accuracy of feature matching.

[0036] This embodiment addresses the advantages and disadvantages of various existing robot localization technologies by proposing a deep learning-based robot localization method. By leveraging artificial intelligence technology, it directly achieves rapid matching and localization between local and global environment maps, greatly improving the accuracy and robustness of robot localization.

[0037] This embodiment is applicable to scenarios requiring precise robot localization within any type of 2D environment map. For ease of explanation, this embodiment uses a 2D LiDAR robot and a 2D grid-based global environment map as examples. The 2D grid-based global environment map is as follows: Figure 3 As shown.

[0038] This embodiment uses a convolutional neural network (CNN) structure to design a deep neural network model. For ease of description, some terms are defined: feature map resolution refers to feature map width × feature map height, feature map size refers to feature map channel number × feature map width × feature map height, kernel size refers to kernel width × kernel height, and span refers to span in the width direction × span in the height direction. In addition, each convolutional layer is followed by a batch normalization layer and a non-linear activation layer.

[0039] like Figures 1 to 3 As shown, the robot localization method based on deep learning proposed in this invention includes the following steps:

[0040] S1. Design a deep neural network model. The deep neural network model is the core of artificial intelligence technology, and its main function is to extract key detailed information features from the input environment map. The network structure of the deep neural network model designed in this embodiment is as follows: Figure 2 As shown, the globalmap layer represents a pre-built 2D grid global environment map; the localmap layer represents a local environment map of the robot's current pose. Both the globalmap layer and the localmap layer are single-channel grayscale images, which can be images of any input resolution. Generally, the resolution of the localmap layer image is much smaller than that of the globalmap layer image. For ease of description, this embodiment refers to the resolution of the input image as WxH.

[0041] The deep neural network model includes a convolutional layer conv0, a max pooling layer, a convolutional layer conv, a cross attention layer cross_attn, and a candidate pose acquisition layer Maxop. The max pooling layer and the convolutional layer conv are connected in sequence to form a layer group. Multiple layer groups are connected in sequence, and the layer group at the top is connected to the output of the convolutional layer conv0 as the shallowest scale. The convolutional layer conv0 takes a 2D grid global environment map and a local environment map as input. The convolutional layer conv0 and the convolutional layers conv in each layer group output multi-scale feature vectors of the global environment map and the local environment map, respectively.

[0042] The conv layers in different layer groups are conv0, conv1, conv2, conv3, and convn, all of which are convolutional layers with a kernel size of 3x3 and a span of 1x1; the maxpool layers in different layer groups are maxpool1, maxpool2, maxpool3, and maxpooln, all of which are max pooling layers with a kernel size of 2x2 and a span of 2x2; ... represents any number of layer groups, and the appropriate number of network combination layers can be flexibly selected according to the specific robot localization environment.

[0043] F0_g, F1_g, F2_g, F3_g, ..., Fn_g represent the multi-scale feature vectors of the input image of the global map layer (global environment map) after processing by the conv0, conv1, conv2, conv3, ..., convn layers; F0_l, F1_l, F2_l, F3_l, ..., Fn_l represent the multi-scale feature vectors of the input image of the local map layer (local environment map) after processing by the conv0, conv1, conv2, conv3, ..., convn layers; the cross_attn layer represents the cross-attention layer, used to calculate the similarity between the multi-scale feature vectors of the input global environment map and the multi-scale feature vectors of the local environment map. Specifically, the multi-scale feature vectors of the local environment map are used as the convolution kernel, and the multi-scale feature vectors of the global environment map are used as the input feature layer. Using standard convolution operations, the output feature vector is the similarity heatmap, i.e., the similarity map.

[0044] Loss_0, Loss_1, Loss_2, Loss_3, ..., Loss_n are all supervision function layers used for training the deep neural network model. They correspond one-to-one with F0_g and F0_l, F1_g and F1_l, F2_g and F2_l, F3_g and F3_l, ..., Fn_g and Fn_l, respectively. The Maxop layer is the candidate pose acquisition layer, used for prediction in the deep neural network model. It primarily selects the highest values ​​from the similarity heatmap as candidate poses for the robot's current pose. Specifically, the Maxop layer sorts all similarity heatmaps in descending order and outputs the top k poses as candidate poses for the robot's current pose.

[0045] S2. Training the deep neural network model mainly involves optimizing the deep neural network model parameters using a large amount of training sample data of paired environment map images with known pose information. This enables the deep neural network model to accurately obtain the optimal pose of the robot's perceived local environment map within the global environment map. The specific steps are as follows:

[0046] S21. Obtain training sample data pairs, mainly by collecting various environment maps and local environment maps of various poses within those environment maps, and forming training data pairs. The specific steps are as follows:

[0047] S211. Collect environment maps, mainly collecting 2D raster global environment maps in various scenarios.

[0048] S212. Obtaining the local environment map: The main method is to arbitrarily select an obstacle-free pose in the 2D grid global environment map, and emit rays in all directions from this pose as the origin. Record the first intersection pose of each ray with an obstacle in the grid map, collect the intersection pose information of all rays, and then obtain the minimum bounding rectangle of all ray intersection poses according to the standard of containing 95% of the ray intersections. Next, crop the grid map according to the minimum bounding rectangle. The obtained local map is the local environment map of the robot in the current pose. Finally, 5% of the random obstacle poses are erased and added to the local environment map.

[0049] S213. Obtain training data labels. First, create a pure black background image with the same size as the 2D grid global environment map. Using the reference origin pose in step S212 as the center, draw a circle with a radius of 10 pixels according to the Gaussian distribution as the supervision label for the current pose.

[0050] S214. Obtain training sample data pairs, which mainly involves combining the data images obtained in steps S211, S212, and S213 as a training sample data pair. Each training sample data pair includes a 2D grid global environment map, a local environment map, and a supervision label corresponding to the local environment map. Multiple training sample data pairs are formed based on local environment maps with different poses, thereby constituting a training dataset to serve as training data for the deep neural network model.

[0051] S22. Design the target loss function of the deep neural network model. The loss function used in the deep neural network designed in this embodiment is the mean squared error loss function and the cross-entropy loss function. These two loss functions are weighted according to a certain ratio to form the final target loss function.

[0052] S23. Training a deep neural network model mainly involves feeding training sample data into a predefined deep neural network model to learn the relevant model parameters.

[0053] S3. Using a deep neural network model, for any 2D grid global environment map and its local environment map, directly feed them into the trained deep neural network model, and output the best candidate pose corresponding to the local environment map. The specific prediction process is as follows:

[0054] S31. Obtaining the multi-scale feature vector of the global environment map mainly involves feeding the 2D grid global environment map into a deep neural network model to obtain the output feature vectors at each scale. Since the global environment map is fixed during robot use and is relatively large, requiring a large amount of computation, this embodiment only needs to calculate the multi-scale feature vector once for any global environment map.

[0055] S32. Obtain multi-scale feature vectors of the local environment map, feed the local environment map into the deep neural network model, and obtain the output feature vectors at each scale.

[0056] S33. To obtain the best candidate pose, first obtain the similarity heatmap at the deepest scale, and then obtain the candidate pose through the Maxop layer. Since the candidate pose at the deepest scale has a large error, the candidate pose at the deepest scale can be used as the reference pose origin as needed. Find the corresponding pose origin of the pose origin on the shallower scale feature layer. Based on the corresponding pose origin, select a local feature region on the shallower scale feature layer, calculate the similarity heatmap of the local region and perform the Maxop operation to achieve fast, efficient and accurate candidate pose acquisition.

[0057] S34. Obtain the best pose. Since there may be similar poses in the 2D grid global environment map, the best candidate pose obtained in step S33 may not be unique. At this time, the robot needs to move to make fine corrections to the candidate poses and finally obtain the best robot pose.

[0058] This embodiment leverages the powerful expressive capabilities of deep neural network models to directly acquire the precise pose of the input local environment map end-to-end. Utilizing a multi-size feature-based hierarchical localization prediction strategy, it significantly improves localization speed while greatly enhancing the accuracy of local environment map localization. The entire network model is highly versatile, fast in recognition, and adaptable and robust to robot localization needs in various scenarios.

[0059] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A robot localization method based on deep learning, characterized in that, Includes the following steps: The acquired global environment map and the local environment map under the global environment map are fed into the trained deep neural network model to obtain the robot's accurate candidate pose. The candidate poses are finely corrected by the robot's motion, and the optimal robot pose is finally obtained. The deep neural network model includes a convolutional layer conv0, a max pooling layer maxpool, a convolutional layer conv, a cross attention layer cross_attn, and a candidate pose acquisition layer Maxop; The maxpooling layer and the convolutional layer conv are connected in sequence to form a layer group. Multiple layer groups are connected in sequence. The layer group at the top is connected to the output of the convolutional layer conv0 as the shallowest scale. The convolutional layer conv0 takes a 2D grid global environment map and a local environment map as input. The convolutional layer conv0 and the convolutional layer conv in each layer group output the multi-scale feature vector of the global environment map and the multi-scale feature vector of the local environment map, respectively. The multi-scale feature vectors of the global environment map and the multi-scale feature vectors of the local environment map are sequentially input into the cross-attention layer cross_attn and the candidate pose acquisition layer Maxop to output the robot's best pose. The process of acquiring the local environment map is as follows: In the 2D grid global environment map, arbitrarily select an obstacle-free pose, and emit rays in all directions from this pose as the origin. Record the first intersection pose of each ray with an obstacle in the 2D grid global environment map, and collect the intersection pose information of all rays. According to the standard of containing 95% of the intersection poses, obtain the minimum bounding rectangle of all intersection poses. Clip the 2D grid global environment map according to the minimum bounding rectangle, and the acquired local map is used as the local environment map of the robot in the current pose. Erase and add 5% random obstacle poses to the local environment map to obtain the local environment map under the 2D grid global environment map.

2. The robot localization method based on deep learning according to claim 1, characterized in that, The training process of the deep neural network model is as follows: Obtain a 2D raster global environment map and a local environment map under the 2D raster global environment map, form training sample data pairs, and assign labels to the training sample data pairs; The target loss function is constructed by weighting the mean squared error loss function and the cross-entropy loss function; The training sample data pairs are input into the deep neural network model to learn the model parameters in order to train the deep neural network model.

3. The robot localization method based on deep learning according to claim 2, characterized in that, The specific steps for assigning labels to the training sample data pairs are as follows: Create a pure black background map with the same size as the 2D raster global environment map. Draw a circle with a radius of 10 pixels centered at the origin and following a Gaussian distribution as the supervision label for the current pose.

4. The robot localization method based on deep learning according to claim 3, characterized in that, Each training sample data pair includes a 2D raster global environment map, a local environment map, and a supervision label corresponding to the local environment map; Multiple training sample data pairs are formed from local environment maps in different poses, thus constituting a training dataset, which is used as training data for deep neural network models.

5. A robot localization system based on deep learning, characterized in that, The acquired global environment map and the local environment map under the global environment map are fed into the trained deep neural network model to obtain the robot's optimal pose. The deep neural network model includes a convolutional layer conv0, a max pooling layer maxpool, a convolutional layer conv1, a cross attention layer cross_attn, and a candidate pose acquisition layer Maxop; The maxpooling layer and the convolutional layer conv are connected in sequence to form a layer group. Multiple layer groups are connected in sequence. The layer group at the top is connected to the output of the convolutional layer conv0 as the shallowest scale. The convolutional layer conv0 takes a 2D grid global environment map and a local environment map as input. The convolutional layer conv0 and the convolutional layer conv in each layer group output the multi-scale feature vector of the global environment map and the multi-scale feature vector of the local environment map, respectively. The multi-scale feature vectors of the global environment map and the multi-scale feature vectors of the local environment map are sequentially input into the cross-attention layer cross_attn and the candidate pose acquisition layer Maxop to output the robot's best pose. The process of acquiring the local environment map is as follows: In the 2D grid global environment map, arbitrarily select an obstacle-free pose, and emit rays in all directions from this pose as the origin. Record the first intersection pose of each ray with an obstacle in the 2D grid global environment map, and collect the intersection pose information of all rays. According to the standard of containing 95% of the intersection poses, obtain the minimum bounding rectangle of all intersection poses. Clip the 2D grid global environment map according to the minimum bounding rectangle, and the acquired local map is used as the local environment map of the robot in the current pose. Erase and add 5% random obstacle poses to the local environment map to obtain the local environment map under the 2D grid global environment map.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the robot localization method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of classification programs, which are used by a processor to call and execute the robot localization method as described in any one of claims 1-4.

Images