A Mobile Robot Autonomous Positioning System and Method Based on an Aerial 3D Model

Abstract

The present invention discloses a mobile robot autonomous positioning system and method based on an aerial three-dimensional model, belonging to the field of mobile robot autonomous positioning and navigation. This method fuses lidar point cloud and image information to achieve autonomous positioning and navigation of the mobile robot in the aerial three-dimensional model, breaking through the limitation in the prior art that the mobile robot can only perform positioning and navigation on the ground through a self-built map. An aerial three-dimensional model of a designated park is established by aerial photography as the positioning map of the mobile robot, and global positioning of the mobile robot in the aerial three-dimensional model is achieved by fusing lidar point cloud and images, broadening the scope of use of the mobile robot, enabling it to still achieve autonomous positioning and navigation of the mobile robot through the aerial three-dimensional model established by aerial photography under the condition of poor ground conditions, and avoiding problems such as error accumulation and drift caused by the existing mobile robot autonomous positioning relying on continuous and stable matching of rich feature points in the environment.

Description

Technical Field

[0001] The present invention relates to the field of autonomous positioning and navigation of mobile robots, autonomous vehicles, etc., and particularly relates to a mobile robot autonomous positioning system and method based on an aerial triangulation model. Background Art

[0002] With the rapid development of information technology, as the core function of mobile robots - SLAM (Simultaneous Localization and Mapping) has become a popular research direction. The ability of a mobile robot to calculate its current 6-DOF pose in real time is an important part of SLAM. Nowadays, SLAM algorithms often rely on the movement of the robot to achieve positioning and mapping in an unknown environment. By extracting key features between adjacent frames and calculating the relative pose between adjacent frames, real-time positioning and mapping of the robot are realized. However, this method is prone to introduce matching errors due to the influence of similar features in the scene and is prone to drift phenomena as the accumulation occurs. Therefore, how to achieve global stable and reliable positioning of the robot in the map has always been a very important research topic.

[0003] The methods based on continuous matching of features include the following:

[0004] Method based on registration between adjacent frames: The point clouds of the extracted adjacent frames are transformed to the same position through the information of the overlapping part, and the relative pose between two adjacent frames can be calculated by iterating enough points;

[0005] Method based on 3D features of point clouds: Extract pre-defined 3D features from the real-time local point cloud and the global map point cloud, and calculate the pose of the robot in the current map using the matching correspondence of the 3D features of the local map and the global map;

[0006] Method based on deep learning: Use a deep learning network to fit a deep learning model between the pose and features of the extracted local map features;

[0007] The methods based on continuous feature matching have achieved very good results for the problems of autonomous positioning and navigation of mobile robots under the condition of rich features. However, this method is prone to be affected by weak textures in the scene and introduce matching errors in the continuous positioning process, resulting in phenomena such as drift in autonomous positioning. Summary of the Invention

[0008] To solve the defects existing in the above prior art, the present invention proposes a mobile robot autonomous positioning system and method based on an aerial three-dimensional model, which uses the aerial three-dimensional model to replace the ground map established by the mobile robot for autonomous positioning, expands the usage range of the mobile robot and improves the usage efficiency. The camera pose of the image is estimated by using the texture information of the image through a deep learning network, which can globally roughly locate the mobile robot. On this basis, the point cloud data of the lidar is fused with the image data to achieve the global precise positioning of the mobile robot in the aerial three-dimensional model.

[0009] The technical solution of the present invention is realized as follows:

[0010] Aerial three-dimensional model global map feature database construction module: used to generate a global map feature descriptor search library and block index for mobile robot positioning;

[0011] Mobile robot global rough positioning module: uses image information to globally roughly locate the mobile robot in the aerial three-dimensional model and retrieves the block index of the global rough positioning;

[0012] Mobile robot global precise positioning module: indexes the rough positioning block feature descriptor search library in the global map feature descriptor search library according to the block index of the global rough positioning, enhances the point cloud feature information with the image and retrieves and matches in the rough positioning block feature descriptor search library, and calculates the precise pose of the mobile robot after successful matching.

[0013] Preferably, the aerial three-dimensional model global map feature database construction module specifically includes:

[0014] Aerial three-dimensional model establishment and model point cloud conversion unit: used to establish an aerial three-dimensional model, convert the three-dimensional model into point cloud, and provide a global map for the mobile robot to perform positioning;

[0015] Point cloud model segmentation and description unit: used to semantically segment the global map and extract the feature descriptors of each segmentation segment to generate a global map feature descriptor search library for mobile robot autonomous positioning;

[0016] Point cloud model block division unit: used to divide the global map and the feature descriptor search library into blocks and generate block indexes.

[0017] Preferably, the mobile robot global rough positioning module specifically includes:

[0018] Simulation data acquisition unit: used to simulate the mobile robot to collect images and their corresponding camera pose information on the ground in the aerial three-dimensional model;

[0019] Dataset generation unit: used to generate a training set and a test set for training a neural network for camera pose estimation based on a single-frame image;

[0020] Network construction and model training unit: used to construct a neural network for camera pose estimation based on a single-frame image and train it to obtain a rough camera pose estimation model based on a single-frame image;

[0021] Rough positioning module: uses the rough camera pose estimation model to roughly estimate the camera pose of the images collected in real time by the mobile robot, obtain a rough positioning of the mobile robot, and retrieve the block index of the rough positioning.

[0022] Preferably, the mobile robot global precise positioning module specifically includes:

[0023] Point cloud real-time segmentation and description unit: used to segment the local point cloud collected in real time by the lidar and extract its descriptors;

[0024] Feature information enhancement unit: used to extract the feature information of the image and enhance the local point cloud descriptors using the feature information of the image;

[0025] Precise positioning unit: used to search and match the enhanced local point cloud descriptors in the feature information enhancement unit with the point cloud information center of the specified block index, and calculate the precise pose of the robot after successful matching.

[0026] The present invention also provides a mobile robot autonomous positioning system and method based on an aerial three-dimensional model, which is characterized by including the following steps:

[0027] Step (1), generate a global map feature descriptor search library and a block index for the mobile robot to perform positioning;

[0028] Step (2), roughly position the mobile robot and retrieve the block index of the rough positioning;

[0029] Step (3), find the rough positioning block feature descriptor search library according to the rough positioning block index, use the image to enhance the point cloud feature information and retrieve and match it in the rough positioning block feature descriptor search library, and calculate the precise pose of the robot after successful matching.

[0030] The present invention uses an aerial three-dimensional model as the data source of the global map, point-clouds the aerial three-dimensional model as the global map for the autonomous positioning and navigation of a mobile robot, segments and compresses the description of the global map to generate a subset of feature descriptions, provides a global descriptor map search library for the autonomous positioning of the mobile robot, and establishes an index between the block sequence and the subset of block descriptions by partitioning the aerial three-dimensional model; at the same time, using the point clouds and images collected in real time by the mobile robot as the data source of the local map, respectively segmenting and compressing the descriptions of the real-time local point clouds and images by means of deep learning, and fusing the descriptors of the point clouds and images as the features of the local map, describing, roughly positioning the mobile robot through the image, indexing to the block of the rough positioning according to the pose of the rough positioning to narrow the search scope in the retrieval matching step; finally, fusing the point cloud data and image information in the block of the rough positioning to achieve the precise positioning of the mobile robot in the aerial three-dimensional model.

[0031] The advantages of the present invention are as follows: An aerial three-dimensional model is used as the map for the global positioning of a mobile robot. An aerial three-dimensional model is established by means of aerial photography by a drone. The field of view is wide and the efficiency is high. The aerial three-dimensional model of a specified area can be quickly established as the local matching map for the positioning of the mobile robot. This method realizes the autonomous positioning and navigation of the mobile robot in the aerial three-dimensional model by fusing lidar point clouds and image information, breaking through the limitation in the prior art that mobile robots and unmanned vehicles can only perform positioning and navigation in a map autonomously established on the ground, broadening the application scope of the mobile robot, so that in some situations with poor ground conditions such as earthquake-stricken areas and flood-stricken areas, the mobile robot can still realize autonomous positioning and navigation through the aerial three-dimensional model established by aerial photography, eliminating the situation where real-time ground positioning and mapping cannot be performed when the ground conditions are limited. Moreover, the image information and point cloud information are fused. Based on the rough positioning by the image, the image is used to enhance the point cloud information to achieve the global positioning of the mobile robot in the aerial three-dimensional model, eliminating the problems of low stability and easy positioning failure caused by the interruption of the movement path when the mobile robot performs positioning by moving in the prior art, and avoiding the problems of error accumulation and drift caused by the continuous and stable matching of the existing mobile robot's autonomous navigation relying on rich feature points in the environment, making the use of mobile robots and unmanned vehicles on the ground more extensive and convenient. Description of the Drawings

[0032] In order to more clearly illustrate the technical solution of the present invention, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following-described drawings are only some examples of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0033] Figure 1 Schematic diagram of the process of an autonomous positioning and navigation method for a mobile robot based on an aerial 3D model;

[0034] Figure 2 Schematic diagram of the process of constructing a global map feature database for an aerial 3D model;

[0035] Figure 3 Schematic diagram of the process of the global rough positioning module.

[0036] Figure 4 Schematic diagram of the process of the global precise positioning module;

[0037] Figure 5 Schematic diagram of the process of a method for enhancing the feature information of an image feature enhanced point cloud;

[0038] Figure 6 Schematic diagram of the VGG-16 network framework;

[0039] Figure 7 Schematic diagram of the CNN processing flow. Specific implementation manner

[0040] The following elaborates on the preferred embodiments of the present invention in conjunction with the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making the protection scope of the present invention more clearly defined.

[0041] A mobile robot autonomous positioning and navigation system and method based on an aerial 3D model, and the basic framework process implemented by the entire system is as Figure 1As shown in the figure, it includes a global map feature database construction module for the aerial three-dimensional model, a rough positioning module for the robot in the aerial three-dimensional model based on images, and a global precise positioning module for the robot in the aerial three-dimensional model based on the fusion of lidar point cloud and image information. Among them, the global map feature database construction module for the aerial three-dimensional model obtains aerial images through oblique photography and establishes an aerial three-dimensional model, divides, segments, and describes the aerial three-dimensional model to establish a global map feature database for the global precise positioning module of the robot in the aerial three-dimensional model based on the fusion of point cloud and image information to search and match; the rough positioning module for the robot in the aerial three-dimensional model based on image information establishes a deep learning network to train a camera pose restoration model based on a single-frame image, and roughly locates the pose to narrow the search range in the global search descriptor subset provided by the global map and feature description subset construction module of the mobile robot in the global positioning of the mobile robot in the aerial three-dimensional model, improve the real-time performance of the real-time positioning of the mobile robot, and minimize possible errors in matching as much as possible; the global precise positioning module for the robot in the aerial three-dimensional model based on the fusion of point cloud and image information segments and describes the real-time point cloud to extract the descriptor of the point cloud, enhances the descriptor of the point cloud through the feature vector in the ROI region of the image, and searches and matches the enhanced feature vector in the block-indexed global descriptor subset provided by the global map and feature description subset construction module of the mobile robot in the global positioning of the mobile robot. When the number of successful matching pairs reaches a certain number, the precise pose of the current robot can be calculated.

[0042] The construction process of the global map feature database of the aerial three-dimensional model is as Figure 2 shown, including the following steps:

[0043] S1) Establish an aerial three-dimensional model: Use a drone and a 5-lens oblique photography camera to collect oblique photography aerial images of a designated park, and establish an aerial three-dimensional model of the park through 3D real-scene modeling software. Finally, point-cloud the model as the global map for mobile robot positioning.

[0044] S2) Model blockization and index establishment: Divide the aerial three-dimensional model into several appropriate blocks, and establish an index between the point cloud blocks and the block sequences.

[0045] S3) Point cloud segmentation and description: Semantically segment the point-clouded model in step S1, and extract the feature descriptor of each segmented unit using a convolutional network to establish a feature description subset of the global point cloud map, that is, the search library. According to the block index in step S2, an index between the feature information and the block sequence can be established.

[0046] Specifically, for example, to complete the point cloud data segmentation of the block index, considering that the point cloud data of the lidar is relatively sparse, it can be realized by using the angle threshold method based on the depth map. First, a depth map is established using the point cloud, and the points in the three-dimensional space are projected onto a cylindrical surface that can be unfolded to make it planar, so as to convert the three-dimensional rectangular coordinates of the point cloud into two-dimensional coordinates and establish the mapping relationship between the point cloud and the depth map. Secondly, the ground point cloud data in the point cloud data is removed to improve the efficiency and accuracy of the segmentation. The angle threshold method based on the depth map can be used to achieve this. The significant difference in the angle values formed by the laser scanning lines between the ground and obstacles is used as the basis for separating the ground point cloud. The BFS (Breadth First Search) algorithm is adopted to traverse the depth map and calculate the angle difference between each point and its adjacent points to achieve the removal of the entire ground point cloud. Finally, the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm is used to achieve the segmentation of the point cloud, and thus the segmentation of the point cloud can be realized.

[0047] To complete the description of the point cloud data of the block index, in the technical solution of the present invention, 3 fully convolutional layers, 2 max-pooling layers, and 2 fully connected layers are used to extract the feature vectors of the segmented point cloud fragment data. In the first connection layer, the data obtained by reducing the dimension of the starting input fragment point cloud through PCA (principal components analysis) is mixed and input to improve the robustness of the feature vectors of the finally output point cloud.

[0048] In the above technical solution, BFS is one of the simplest graph search algorithms and belongs to a blind search method. Its purpose is to systematically expand and check all nodes in the graph to find the result. In other words, it does not consider the possible location of the result and thoroughly searches the entire graph until the result is found; DBSCAN is a relatively representative density-based clustering algorithm. It defines a cluster as the largest set of density-connected points, can divide the areas with sufficient high density into clusters, and can discover clusters of any shape in the spatial database with noise; PCA is the principal component analysis, aiming to use the idea of dimensionality reduction to convert multiple indicators into a few comprehensive indicators.

[0049] The global rough positioning process of the mobile robot is as Figure 3 shown, including the following steps:

[0050] S1) Simulated data acquisition. Convert the coordinate system of the aerial three-dimensional model to the ENU (Local Cartesian Coordinates Coordinate System, East-North-Up coordinate system), and write a program to simulate the mobile robot automatically collecting images and their corresponding camera poses on the road of the aerial three-dimensional model;

[0051] S2) Construction and training of a deep learning network for rough positioning. Use the data collected in step S1 to make a training set and a test set, and build a deep learning network to train a camera pose estimation model based on a single-frame image.

[0052] S3) Rough positioning. According to the model trained in step S2, perform a rough camera pose estimation on the images collected in real time by the mobile robot, and locate the approximate pose of the mobile robot in the aerial three-dimensional model based on the estimated camera pose, and output the block index of the mobile robot's position and the block indexes of its adjacent blocks in the global map feature database construction module of the aerial three-dimensional model;

[0053] Specifically, for example, to complete rough positioning based on images, first make a data set. The method of making the data set is as Figure 3 shown. Simulate the mobile robot collecting image information and corresponding camera pose information on the ground in the aerial three-dimensional model in the ENU coordinate system. At the same time, use inertial navigation and industrial cameras to collect a small amount of real pictures and corresponding poses, and mix the data in the empty three-dimensional model with the data in the real scene to make a training set and a test set. Then use deep learning to train a camera pose estimation model based on a single-frame image. When the mobile robot is in the real scene, use the model obtained above to perform camera pose estimation on the images taken by the camera on the mobile robot, and obtain the approximate position of the camera in the aerial three-dimensional model, that is, roughly locate the pose of the mobile robot. Locate the sequence of blocks in the aerial three-dimensional model according to the approximate pose of the calculated camera, and take out the point cloud descriptors corresponding to the current sequence and its surrounding 8 sequences according to the sequence. This 9-block descriptor set is used as the mother vector set for matching the current point cloud segment feature vector, and provides a search set for the robot precise positioning module in the aerial three-dimensional model based on the fusion of point cloud and image information.

[0054] The present invention uses GoogLeNet to construct a neural network for camera pose estimation based on a single-frame image, changes the final output to the angular quaternion and position coordinates of the camera pose, and constructs a loss function for angles and positions for training to obtain a neural network for camera pose estimation of a single-frame image based on deep learning.

[0055] The process of the mobile robot global precise positioning module is as Figure 4 shown, including the following steps:

[0056] S1) Point cloud segmentation and description module. First, to ensure the accuracy and real-time performance of segmentation, the point cloud scanned by the lidar in real time is processed to remove the ground part of the point cloud. Then, the point cloud is segmented into several small segments. For each segmented point cloud segment, we use a CNN to build a deep learning network for extracting point cloud features and extract its feature descriptor for each segment;

[0057] S2) ROI (Region Of Interest) feature vector extraction of the image and enhancement of the point cloud descriptor. According to the point cloud and the image collected at the same moment, the point cloud is projected onto the image and the ROI covered by the point cloud on the image is intercepted. Then, a deep learning network for extracting image feature information is built using a CNN, and the feature information is fused with the feature information of the point cloud to enhance the local point cloud descriptor in step S1;

[0058] S3) Narrow the search range through rough positioning. Use the model obtained by training the global rough positioning module of the mobile robot to estimate the pose of the image collected by the robot, and according to the calculated pose, the global positioning map, and the method of dividing the point cloud model by the map description subset construction module, take out a total of 9 small pieces of the point cloud model where the pose is located and its surrounding area as the rough positioning pose area of the pose;

[0059] S4) Precise positioning of point cloud features based on image enhancement. Use the point cloud feature descriptor based on image enhancement in step S2 to search and match in the result descriptor subset of the segmentation description in step S1 for the 9 voxelized models segmented in step S3. When the number of matching pairs reaches a certain amount, calculate the pose of the current point cloud to achieve precise positioning.

[0060] Specifically, for example, to complete the extraction of the ROI feature vector of the image, in the solution of the present invention, VGG-16 is used to extract the features of the image ROI, and the local aggregation vector of the image can be extracted by passing the features output by VGG-16 through the NetVLAD network. For the extraction of the feature vector of the local point cloud in step S1, the method of global point cloud map segmentation and description in step S3 of the aerial three-dimensional model map and feature descriptor construction module for global positioning of the mobile robot is adopted. A feature extractor is built using 3 fully convolutional layers, 2 max pooling layers, and 2 fully connected layers to extract the feature vector of the segmented point cloud segment data. In the first connection layer, the data obtained by PCA dimensionality reduction of the fragment point cloud of the starting input is mixed to improve the robustness of the finally output point cloud feature vector. For the precise positioning of point cloud features based on image enhancement in step S4, such as Figure 5As shown, first, the ROI feature vector of the image is input into the first connection layer of the feature extractor in the aforementioned step S1. The feature vector of the point cloud is enhanced using the ROI feature vector, and then the Kd-tree is used to perform nearest neighbor search and matching in the descriptor set provided in the aforementioned step S3. When the number of successful matches reaches a certain value, the precise pose of the current robot can be calculated.

[0061] In the above technical solution, VGG-16 is a convolutional neural network model proposed by Simonyan and Zisserman. This model participated in the ImageNet image classification and localization challenge in 2014, ranking second in the classification task and first in the localization task. Its structure is as Figure 6 shown. Its most important feature is that it uses several consecutive 3x3 convolutional kernels instead of the larger convolutional kernels in AlexNet. In this way, for a given receptive field, the network depth can be increased through multiple non-linear layers to ensure learning more complex patterns, and with fewer parameters, the structure is concise and the ability to extract image features by continuously increasing the depth can be optimized.

[0062] In the above technical solution, CNN is an efficient recognition method that has developed and attracted attention in recent years. In the 1960s, Hubel and Wiesel found that the unique network structure of neurons used for local sensitivity and direction selection in the cat brain cortex could effectively reduce the complexity of the feedback neural network, and then proposed CNN. Now, CNN has become one of the research hotspots in many scientific fields, especially in the field of pattern classification. Since this network avoids complex preprocessing of images and can directly input the original images, it has been more widely used.

[0063] Generally, the basic structure of CNN includes two layers. One is the feature extraction layer, where the input of each neuron is connected to the local receptive field of the previous layer and extracts the features of this local area. Once this local feature is extracted, the positional relationship between it and other features is also determined; the other is the feature mapping layer. Each computational layer of the network consists of multiple feature maps, and each feature map is a plane where the weights of all neurons on the plane are equal.

[0064] In the technical solution of the present invention, the feature mapping layer is used to extract the global underlying features in the video frame image, and then deeper processing is performed on the underlying features.

[0065] The general processing flow of CNN is as Figure 7 shown.

[0066] The layer to be used in the technical solution of the present invention is the Feature Map obtained after convolution. We extract six of them with sizes of (38, 38), (19, 19), (10, 10), (5, 5), (3, 3), and (1, 1) respectively, and then set multiple prior boxes with different scales or aspect ratios at each unit of the feature map. In this way, the feature map is formed. The detection result is obtained by convolving the feature map, and the detection values include the class confidence and the bounding box position. Each is completed by using a 3×3 convolution once.

[0067] The above are only the embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied to other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A mobile robot autonomous positioning system based on an aerial three-dimensional model, characterized in that, Including: An aerial 3D model global map feature database construction module, configured to generate a global map feature descriptor search library and a block index for mobile robot positioning, and the aerial 3D model global map feature database construction module includes: An aerial 3D model establishment and model point cloud conversion unit, configured to establish an aerial 3D model, convert the 3D model into point cloud, and provide a global map for mobile robot positioning, A point cloud model segmentation and description unit, configured to perform semantic segmentation on the global map, and extract feature descriptors of each segmentation segment to generate a global map feature descriptor search library for mobile robot positioning, A point cloud model block division unit, configured to perform block division on the global map and the feature descriptor search library and generate a block index; A mobile robot global rough positioning module, configured to use image information to achieve global rough positioning of the mobile robot in the aerial 3D model, and retrieve the block index of the global rough positioning, and the mobile robot global rough positioning module includes: A simulated data acquisition unit, configured to simulate the mobile robot collecting images and their corresponding camera pose information on the ground in the aerial 3D model, A dataset generation unit, configured to generate a training set and a test set for training a neural network for camera pose estimation based on a single-frame image, A network construction and model training unit, configured to construct a neural network for camera pose estimation based on a single-frame image, and train it to obtain a rough camera pose estimation model based on a single-frame image, A rough positioning module, configured to use the rough camera pose estimation model to perform rough camera pose estimation on the images collected by the mobile robot in real time, achieve rough positioning of the mobile robot, and retrieve the block index of the rough positioning; A mobile robot global precise positioning module, configured to index to the rough positioning block feature descriptor search library in the global map feature descriptor search library according to the block index of the global rough positioning, perform retrieval and matching in the rough positioning block feature descriptor search library after enhancing the point cloud feature information with the image, and calculate the precise pose of the mobile robot after successful matching, and the mobile robot global precise positioning module includes: A point cloud real-time segmentation and description unit: configured to segment the local point cloud collected by the lidar in real time and extract its descriptors; A feature information enhancement unit: configured to extract the feature information of the image, and enhance the local point cloud descriptor with the feature information of the image; A precise positioning unit: configured to search and match the enhanced local point cloud descriptor in the feature information enhancement unit in the point cloud information center of the specified block index, and calculate the precise pose of the mobile robot after successful matching.

2. A mobile robot autonomous positioning method based on an aerial three-dimensional model, characterized in that, The method is implemented based on the mobile robot autonomous positioning system based on an aerial 3D model described in claim 1, and the method includes the following steps: Step (1) Generate a global map feature descriptor search library and a block index for mobile robot positioning, and step (1) specifically includes: Step (1-1), Establish an aerial 3D model, convert the 3D model into point cloud, and provide a global map for mobile robot positioning, Step (1-2): Segment the global map and extract the feature descriptors of each segmented fragment to generate a search library of global map feature descriptors for mobile robot positioning. Step (1-3): Divide the global map and the search library of feature descriptors into blocks and generate block indexes. Step (2): Roughly locate the mobile robot and retrieve the block index of the rough location. Specifically, Step (2) includes: Step (2-1): Simulate the mobile robot collecting images on the ground and their corresponding camera pose information in the aerial three-dimensional model. Step (2-2): Generate a training set and a test set for training a neural network for camera pose estimation based on a single-frame image. Step (2-3): Construct a neural network for camera pose estimation based on a single-frame image and train it to obtain a rough camera pose estimation model based on a single-frame image. Step (2-4): Use the rough camera pose estimation model to roughly estimate the camera pose of the images collected by the mobile robot in real time, obtain the rough location of the mobile robot, and retrieve the block index of the rough location. Step (3): According to the block index of the rough location, find the search library of block feature descriptors of the rough location. After enhancing the point cloud feature information with the image, perform a search and match in the search library of block feature descriptors of the rough location. After successful matching, calculate the precise pose of the mobile robot. Specifically, Step (3) includes: Step (3-1): Segment the local point cloud collected by the lidar in real time and extract its descriptors. Step (3-2): Extract the feature information of the image and enhance the feature information of the point cloud with the image information. Step (3-3): Search and match the enhanced feature information in the center of the point cloud information of the specified block index, and calculate the precise pose of the mobile robot.

3. The autonomous positioning method of a mobile robot based on an aerial three-dimensional model according to claim 2, wherein Step (1-1) specifically includes: Using an unmanned aerial vehicle and a 5-lens oblique photography camera to collect oblique photography aerial images of a specified park, establishing an aerial three-dimensional model of the park through three-dimensional real-scene modeling software, and finally point-cloudifying the aerial three-dimensional model as the global map for mobile robot positioning.

Images