A global relocalization method, system, device and medium for a mobile robot

Abstract

The present invention discloses a global relocalization method, system, device and medium for a mobile robot. The method includes: the robot obtains the current scan frame through lidar scanning; for the point cloud data of the current scan frame, a BEV image is generated by voxelization; then the local features of each BEV image are extracted, and the local features are encoded to generate global features; the global features matching the global features of the current scan frame are retrieved in the feature library, and the local features and the robot pose information corresponding to the matching global features are obtained; the local features of the current scan frame and the obtained local features are violently matched to obtain corresponding points; and the corresponding points are subjected to pose estimation to obtain the relative transformation matrix between the two local features; then the relative transformation matrix is applied to the obtained robot pose information to obtain the current global position information of the robot, thereby improving the global localization ability of the robot in the case of lost localization or no initial pose.

Description

Technical Field

[0001] The present invention relates to a fast and robust global relocalization method based on search, and in particular to a global relocalization method, system, device and medium for a mobile robot, belonging to the field of automation technology. Background Art

[0002] With the rapid development of mobile robot technology, its applications in the fields of industrial automation, service industry, warehousing and logistics are becoming increasingly widespread. Mobile robots usually need to perform autonomous navigation and positioning in their working environment to complete specific tasks or operations. To achieve this goal, mobile robots are usually equipped with a variety of sensors, such as lidar (LiDAR), inertial measurement unit (IMU) and wheel speedometer, etc., and obtain their position information in the global map in real time through the simultaneous localization and mapping (SLAM) technology.

[0003] However, in practical applications, mobile robots may lose positioning information or be unable to initialize their poses due to various reasons (such as sensor failures, environmental changes, signal interference, etc.), resulting in their inability to accurately determine their positions in the global map. In this case, the robot needs to perform global relocalization to re-acquire its accurate position information and thus continue to execute tasks.

[0004] Existing global positioning solutions mainly include the following several types:

[0005] 1) Only position recognition method: This method directly uses the point cloud to generate a global descriptor, and then retrieves the most similar frame in the database according to the descriptor distance to obtain the position estimate. However, this method has problems such as inaccurate feature expression, limited generalization ability, and being affected by scale distortion, resulting in insufficient position recognition accuracy in complex environments or new scenarios.

[0006] 2) Global pose estimation method: This method uses local features of the point cloud (such as geometric relationships, traditional image descriptors or segmentation information, etc.) to generate local descriptors, and directly determines the global pose by matching with the global map. However, this method is vulnerable to environmental noise and point cloud density, and has problems such as unstable key point detection, low efficiency and poor adaptability.

[0007] 3) Local pose estimation method after position recognition: This method first uses the global descriptor to retrieve and obtain a rough position, and then estimates the relative pose based on local feature matching to obtain the global pose. However, this method has problems such as insufficient generalization ability, the accuracy of feature extraction being affected by the sparsity of the point cloud, poor adaptability to changes in data distribution, and unstable key point detection.

[0008] In summary, existing global positioning methods have many problems in terms of point cloud sparsity, robustness to rotational changes, generalization ability, dependence on precise pose supervision, and efficiency and resource requirements, making it difficult to meet the high-precision, robust, efficient, and easy-to-deploy global positioning requirements of mobile robots in complex environments.

[0009] Therefore, there is an urgent need for a new global relocalization method for mobile robots to solve the above problems. Summary of the Invention

[0010] Aiming at the existing technical problems, the present invention provides a global relocalization method, system, device, and medium for mobile robots to improve the global positioning ability of mobile robots in the case of lost positioning or without initial pose, thereby enhancing its reliability and efficiency in practical applications.

[0011] To achieve the above technical objectives, first, the present invention provides an automatic deployment system for a robot control system, including a mapping stage and a positioning stage.

[0012] The mapping stage includes the following steps:

[0013] The robot takes the starting position where the mapping operation is started as the map origin, estimates the moving distance of the robot relative to the starting position through the odometer, and updates the pose information of the robot in real time; every time the robot moves a fixed distance, it takes the scan frame obtained by the current lidar as a key frame, and takes the pose information of the current robot obtained from the odometer as the robot pose of the key frame.

[0014] For the point cloud data of each key frame, generate a BEV image through voxelization; then perform rotational data augmentation on each BEV image to generate multiple rotated BEV images; then extract the local features of each BEV image and generate global features from the local features.

[0015] Bind the robot pose, BEV image, local features, and global features corresponding to each key frame, and store them in the feature library; after mapping is completed, obtain the point cloud data of the global map and the feature library composed of all key frames.

[0016] The positioning stage includes the following steps:

[0017] The robot obtains the current scan frame through lidar scanning; for the point cloud data of the current scan frame, generate a BEV image through voxelization; then extract the local features of each BEV image and generate global features from the local features.

[0018] Retrieve the global features in the feature library that match the global features of the current scan frame, and obtain the local features and robot pose information corresponding to the matching global features.

[0019] The local features of the current scan frame and the acquired local features are used to obtain corresponding points through brute-force matching; and the corresponding points are used for pose estimation to obtain the relative transformation matrix between the local features of the current scan frame and the acquired local features; then the relative transformation matrix is applied to the acquired robot pose information to obtain the current global position information of the robot.

[0020] Further, in the method of the present invention, the fixed distance is 0.5 meters.

[0021] Further, in the method of the present invention, the way of rotational data augmentation is to rotate 45 degrees each time, generating 8 rotated BEV images.

[0022] Further, in the method of the present invention, the local features are extracted by the Resnet residual network.

[0023] Further, in the method of the present invention, the global features are generated by encoding through the NetVLAD network.

[0024] Further, in the method of the present invention, the brute-force matching uses a feature point matching algorithm.

[0025] Further, in the method of the present invention, the pose estimation uses the RANSAC algorithm.

[0026] Second, the present invention also provides a global relocalization system for a mobile robot, which uses the global relocalization method for a mobile robot described above, and includes a mapping module and a positioning module.

[0027] The mapping module includes a key frame generation unit, a key frame processing unit, and a feature library generation unit;

[0028] Among them, the key frame generation unit is used for: the robot takes the starting position where the mapping operation is started as the map origin, estimates the moving distance of the robot relative to the starting position through the odometer, and updates the pose information of the robot in real time; every time the robot moves a fixed distance, the scan frame acquired by the current lidar is used as a key frame, and the pose information of the current robot acquired from the odometer is used as the robot pose of the key frame.

[0029] The key frame processing unit is used for: for the point cloud data of each key frame, generating BEV images through voxelization; then performing rotational data augmentation on each BEV image to generate multiple rotated BEV images; further extracting the local features of each BEV image and generating global features from the local features.

[0030] The feature library generation unit is used for: binding the robot pose, BEV images, local features, and global features corresponding to each key frame, and storing them in the feature library; after mapping is completed, obtaining the point cloud data of the global map and the feature library composed of all key frames.

[0031] The positioning module includes a global feature generation unit, a feature library retrieval unit, and a global position acquisition unit;

[0032] Among them, the global feature generation unit is used for: the robot obtains the current scan frame through lidar scanning; for the point cloud data of the current scan frame, generates a BEV image by voxelization; then extracts the local features of each BEV image and generates global features from the local features;

[0033] The feature library retrieval unit is used for: retrieving in the feature library the global features that match the global features of the current scan frame, and obtaining the local features and the robot pose information corresponding to the matching global features;

[0034] The global position acquisition unit is used for: obtaining corresponding points by brute-force matching of the local features of the current scan frame and the obtained local features; and performing pose estimation on the corresponding points to obtain the relative transformation matrix between the local features of the current scan frame and the obtained local features; then applying the relative transformation matrix to the obtained robot pose information to obtain the current global position information of the robot.

[0035] Thirdly, the present invention also provides a global relocalization device for a mobile robot, including at least one processor; and a memory connected to at least one of the processors;

[0036] Among them, the memory stores instructions executable by the processor;

[0037] The instructions are used to be executed by the processor to implement the global relocalization method for a mobile robot.

[0038] Fourthly, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the global relocalization method for a mobile robot.

[0039] As can be seen from the above technical solutions, the present invention proposes a global relocalization method for a mobile robot based on generating a bird's-eye view (BEV) from point cloud data, and solves the problems of point cloud sparsity, insufficient robustness to rotational changes, limited generalization ability, and low efficiency existing in the existing global positioning methods through technological innovation, and has the following remarkable beneficial effects.

[0040] 1) Generating a BEV image from point cloud data to improve the stability of feature extraction:

[0041] The present invention converts point cloud data into BEV images through voxelization. Compared with the method of directly extracting features based on radar points in the prior art, it significantly reduces the impact of point cloud sparsity on feature extraction. BEV images can better retain the global structural information of the scene while reducing the interference caused by changes in point cloud density, thereby improving the stability and accuracy of feature extraction.

[0042] 2) Rotation data augmentation to enhance the robustness and generalization ability to rotation changes:

[0043] When generating global features in the mapping stage, the present invention performs data augmentation with rotation changes on the input BEV images, rotating 45 degrees each time, so that each BEV image is expanded into 8 images. This method not only increases the diversity of training data but also significantly improves the network's robustness and generalization ability to scene rotation changes. In this way, the network can better adapt to scene changes at different angles, thus achieving more accurate global positioning in complex environments.

[0044] 3) Generating global features by local feature encoding to improve feature robustness:

[0045] The present invention encodes local features to generate global features and further optimizes the feature expression through rotation data augmentation. Compared with the prior art, the global features generated in this way are more robust, can better adapt to environmental changes and noise interference, and exhibit higher robustness and accuracy.

[0046] 4) Brute-force matching of local features to improve computational efficiency:

[0047] In the local feature matching stage, the present invention uses brute-force matching to calculate the relative transformation matrix. On the basis of ensuring feature accuracy, the brute-force matching method can effectively improve computational efficiency and reduce the time overhead of global relocalization, thereby meeting the real-time requirements of mobile robots.

[0048] 5) Easy to deploy and reduce resource requirements:

[0049] Based on not introducing additional sensors, the present invention significantly reduces the consumption of computing resources by optimizing the processing of point cloud data and feature extraction methods. At the same time, this method is easy to deploy into existing mobile robot systems without complex hardware modification, and has high practicality and promotion value.

[0050] In summary, through key technologies such as generating BEV images from point cloud data, rotational data augmentation, generating global features by local feature encoding, and brute-force matching, the present invention realizes high-precision, robust, efficient, and easily deployable global relocalization of a mobile robot in a global map, improves the global localization ability of the mobile robot in the case of lost localization or without initial pose, thereby enhancing its reliability and efficiency in practical applications. Moreover, this method exhibits excellent performance in complex environments, can effectively solve many problems existing in the prior art, and provides strong technical support for the autonomous navigation and global relocalization of mobile robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a schematic block diagram when the method of the present invention is implemented;

[0052] Figure 2 is a BEV image obtained in the mapping stage when the method of the present invention is implemented;

[0053] Figure 3 is a flowchart of the localization stage when the method of the present invention is implemented;

[0054] Figure 4 is a schematic diagram of corresponding points obtained in the localization stage when the method of the present invention is implemented;

[0055] Figure 5 is a schematic block diagram when the system of the present invention is implemented;

[0056] Figure 6 is a schematic block diagram when the device of the present invention is implemented. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail below with reference to specific embodiments. It should be clear that the following embodiments are only used to explain the present invention and do not limit the scope of the present invention. Based on the technical solutions of the present invention, those of ordinary skill in the art can derive other feasible implementation manners without creative efforts, and these implementation manners all fall within the protection scope of the present invention.

[0058] Moreover, the embodiments of the present invention aim to help those skilled in the art better understand the technical solutions of the present invention through specific technical details and implementation manners. However, those skilled in the art should understand that the implementation of the present invention is not limited to the specific details described in the following embodiments. Without departing from the core idea of the present invention, the technical details in the embodiments can be appropriately adjusted, replaced, or combined to adapt to different application scenarios or requirements.

[0059] It should be noted that the present invention is applicable to a mobile robot equipped with a lidar, an IMU, and a wheel speedometer. The mobile robot can obtain its own position information in the global map in real time through the slam method, so as to realize the function of performing specific tasks or completing specific operations at specific positions.

[0060] Moreover, the present invention is applied to the positioning task of the mobile robot in the global map. When the robot is unable to obtain its position information in the global map due to certain reasons, the accurate global position information of the robot can be obtained through the technical solution proposed by the present invention.

[0061] Embodiment 1: The global relocalization method of the mobile robot of the present invention.

[0062] As Figure 1 shown, this embodiment provides a global relocalization method for a mobile robot, including a mapping stage and a positioning stage, which are specifically introduced as follows.

[0063] S1. Before the robot executes a task or an operation, it first enters the mapping stage. As Figure 1 shown, the mapping stage includes the following steps:

[0064] S1-1. The robot takes the starting position where the mapping operation is started as the map origin, and during the movement, estimates the moving distance of the robot relative to the starting position through the odometer. At the same time, the pose information of the robot is updated in real time.

[0065] Specifically, the pose of the robot usually includes the position (x, y, z) and the attitude (such as Euler angles or quaternions). Through the data fusion of sensors such as wheel speedometers and IMUs (inertial measurement units), the relative position and attitude changes of the robot in the global coordinate system are estimated. Moreover, the pose information is usually represented in matrix form. For example: ‌Pose matrix‌: a 4x4 transformation matrix containing rotation and translation information; ‌Pose vector‌: a 6D vector containing the position (x, y, z) and the attitude (roll, pitch, yaw).

[0066] S1-2. During the movement of the robot, it obtains the surrounding environment information by continuously scanning the lidar to obtain scan frames. Whenever the robot moves a fixed distance (such as 0.5 meters), the current lidar scan frame is used as a key frame.

[0067] S1-3. For each key frame, obtain the pose information of the current robot from the odometer and record it as the robot pose of the key frame.

[0068] Specifically, the pose recording is synchronized with the generation of key frames. When the robot moves 0.5 meters and generates a key frame, the system will record the pose information of the current robot at the same time.

[0069] S1-4. For the point cloud data of each key frame, generate a BEV image through voxelization, as Figure 2 shown.

[0070] Specifically, when implemented, convert the point cloud data into a BEV image through voxelization, thereby reducing the impact of point cloud sparsity on feature extraction.

[0071] S1-5. Perform rotation data augmentation on each BEV image, rotating 45 degrees each time to generate 8 rotated BEV images.

[0072] Specifically, when implemented, perform data augmentation with rotational changes on the input, that is, rotate the BEV image 45 degrees each time. Through this data augmentation method, each BEV image can ultimately become 8 images. Correspondingly, when encoding local features into global features, the input data of the network becomes eight times the original, thereby increasing the diversity of the input data of the subsequent global feature extraction network model and enhancing the robustness and generalization ability of the network to rotational changes in scene data.

[0073] In addition, the data augmentation in this embodiment is achieved by rotating the input data 45 degrees each time. Other rotation angles or data augmentation methods can also be used. Moreover, the rotation angle and number of times of rotation data augmentation can be adjusted according to the rotational change characteristics of the scene to further improve the robustness and generalization ability of the network.

[0074] S1-6. Extract the local features of each BEV image through the Resnet residual network.

[0075] Specifically, when implemented, this embodiment selects the Resnet residual network to extract the local features of the BEV image. In addition to the Resnet residual network, other methods can also be selected in the method of the present invention to extract the local features of the BEV image, such as SIFT, SURF, ORB, SuperPoint, SuperGlue, etc.

[0076] S1-7. Input the local features of each BEV image into the NetVLAD network to encode and generate robust global features.

[0077] Specifically, when implemented, this embodiment selects the NetVLAD network to encode local features into robust global features, and the positioning accuracy is improved through the Resnet residual network and the NetVLAD network. In addition to the NetVLAD network, other methods can also be selected in the present invention to generate global features, such as Patch-NetVLAD, Super-NetVLAD, etc.

[0078] S1-8. Bind the robot pose, BEV image, local features, and global features corresponding to each key frame, and store them in the feature library.

[0079] During specific implementation, when each key frame is generated, the system records the pose information of the current robot and binds and stores it one by one with other data of the key frame (such as BEV image, local features, global features). This association provides the necessary information for subsequent global relocalization.

[0080] S1-9. After map building is completed, generate the point cloud data of the global map and the feature library composed of all key frames.

[0081] During specific implementation, the feature library includes the robot pose, BEV image, local features, and global features corresponding to the key frame, and is used for the robot to perform retrieval and positioning when there is no global position.

[0082] S2. When the robot has no global position, such as losing its position or having no initial pose, global relocalization will be triggered. As Figure 1 and Figure 3 shown, the positioning stage includes the following steps.

[0083] S2-1. During the positioning stage of the robot, obtain the point cloud data of the current scan frame through lidar scanning.

[0084] S2-2. Generate a BEV image from the point cloud data of the current scan frame through voxelization.

[0085] S2-3. Extract the local features of the BEV image through the Resnet residual network, and encode the local features through the NetVLAD network to generate global features.

[0086] During specific implementation, in addition to the Resnet residual network selected in this embodiment, other methods can also be selected in the method of the present invention to extract the local features of the BEV image, such as SIFT, SURF, ORB, SuperPoint, SuperGlue, etc.

[0087] Moreover, in addition to the NetVLAD network selected in this embodiment, other methods can also be selected in the present invention to generate global features, such as Patch-NetVLAD, Super-NetVLAD, etc.

[0088] S2-4. Retrieve the global features in the feature library that match the global features of the current scan frame, and obtain the local features and robot pose information corresponding to the matching global features.

[0089] During specific implementation, load the feature library data generated in the mapping stage, and retrieve the global feature in the feature library that is most similar to the global feature of the current scan frame, so as to obtain the corresponding local feature and robot pose information.

[0090] S2-5. Obtain corresponding points by brute-force matching the local feature of the current scan frame with the retrieved local feature, as Figure 4 shown.

[0091] During specific implementation, the brute-force matching in this embodiment uses a feature point matching algorithm. In addition, the specific algorithms for local feature encoding and matching can be selected or optimized according to computing resources and real-time requirements, as long as they can achieve efficient and accurate global relocalization.

[0092] S2-6. Perform pose estimation on the corresponding points through the RANSAC algorithm to obtain the relative transformation matrix between the local feature of the current scan frame and the obtained local feature.

[0093] During specific implementation, this step selects the RANSAC algorithm to estimate the relative transformation matrix between two local features. And through brute-force matching and the RANSAC algorithm, efficient and accurate global relocalization is achieved.

[0094] S2-7. Apply the relative transformation matrix to the retrieved robot pose to obtain the current global position information of the mobile robot.

[0095] As can be seen from the above, the method of the present invention is applicable to the positioning task of a mobile robot equipped with a lidar, an IMU, and a wheel speedometer in a global map. In particular, global relocalization of the mobile robot is achieved without introducing additional sensors. By registering the point cloud data obtained from each lidar scan with the point cloud of the global map, the real-time pose of the robot in the global map is obtained, realizing more accurate, robust, efficient, and easily deployable global positioning of the mobile robot, improving the global positioning ability of the robot in case of lost positioning or without initial pose, and can be widely applied to fields such as industrial automation, warehousing logistics, and service robots.

[0096] Embodiment 2: The global relocalization system of the mobile robot of the present invention.

[0097] As Figure 5 shown, this embodiment provides a global relocalization system of a mobile robot, which uses the global relocalization method of a mobile robot described in Embodiment 1, and includes a mapping module and a positioning module, which are specifically introduced as follows.

[0098] (1) The mapping module includes a key frame generation unit, a key frame processing unit, and a feature library generation unit.

[0099] Among them, the key frame generation unit is used for: The robot takes the starting position where the mapping operation is started as the map origin, estimates the moving distance of the robot relative to the starting position through the odometer, and updates the pose information of the robot in real time; Every time the robot moves a fixed distance, it takes the scan frame obtained by the current lidar as the key frame, and obtains the pose information of the current robot from the odometer as the robot pose of the key frame.

[0100] The key frame processing unit is used for: For the point cloud data of each key frame, generate a BEV image through voxelization; Then perform rotation data augmentation on each BEV image to generate multiple rotated BEV images; Then extract the local features of each BEV image and generate global features from the local features.

[0101] The feature library generation unit is used for: Bind the robot pose, BEV image, local features and global features corresponding to each key frame, and store them in the feature library; After the mapping is completed, obtain the point cloud data of the global map and the feature library composed of all key frames.

[0102] (2) The positioning module includes a global feature generation unit, a feature library retrieval unit, and a global position acquisition unit.

[0103] Among them, the global feature generation unit is used for: The robot obtains the current scan frame through lidar scanning; For the point cloud data of the current scan frame, generate a BEV image through voxelization; Then extract the local features of each BEV image and generate global features from the local features.

[0104] The feature library retrieval unit is used for: Retrieve the global features matching the global features of the current scan frame in the feature library, and obtain the local features and robot pose information corresponding to the matching global features.

[0105] The global position acquisition unit is used for: Obtain the corresponding points by brute-force matching the local features of the current scan frame with the obtained local features; And perform pose estimation on the corresponding points to obtain the relative transformation matrix between the local features of the current scan frame and the obtained local features; Then apply the relative transformation matrix to the obtained robot pose information to obtain the current global position information of the robot.

[0106] In summary, through the above technical solutions, the system of the present invention realizes the accurate mapping of the mobile robot to the scene, as well as fast and accurate positioning in the scene where the mapping has been completed, improves the global positioning ability of the robot in the case of lost positioning or no initial pose, thereby enhancing its reliability and efficiency in practical applications.

[0107] Embodiment 3: The global relocalization device of the mobile robot of the present invention.

[0108] AsFigure 6 As shown in Figure 6 , this embodiment provides a global relocalization device for a mobile robot, including at least one processor; and a memory connected to at least one of the processors.

[0109] Among them, the memory stores instructions executable by the processor.

[0110] The instructions are used to be executed by the processor to implement the global relocalization method for a mobile robot described in Embodiment 1.

[0111] Specifically, the processor can be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and can control other components in this device to execute the desired functions and improve the algorithm efficiency.

[0112] The memory can include one or more computer program products, and the computer program products can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. One or more computer program instructions can be stored on the computer-readable storage media, and the processor can run the instructions to implement the methods and / or the functions of their systems described in the above embodiments.

[0113] The instructions are computer program codes written in one or more programming languages or combinations thereof for performing the operations of this application. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++, and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or completely executed on a remote computer or server.

[0114] Embodiment 4: The computer-readable storage medium of the present invention.

[0115] This embodiment provides a computer-readable storage medium, on which a computer program is stored; when the computer program is executed by a processor, it implements the global relocalization method for a mobile robot described in Embodiment 1.

[0116] In specific implementation, the computer program can be written in any combination of one or more programming languages for executing the program code of the embodiments of the present application. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, executed as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0117] The computer-readable storage medium can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0118] In addition, the block diagrams of the systems and devices involved in the present invention are only illustrative examples and do not intend to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these systems and devices can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open-ended words, meaning "including but not limited to", and can be used interchangeably with each other. The word "or" and "and" used herein refer to the word "and / or" and can be used interchangeably with it, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to" and can be used interchangeably with it.

[0119] Moreover, those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Optionally, they can be implemented by program code executable by the computing device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module to implement. Thus, the present application is not limited to any specific combination of hardware and software.

Claims

1. A global relocalization method for a mobile robot, characterized in that, It includes a mapping stage and a positioning stage; The mapping stage includes the following steps: The robot takes the starting position where the mapping operation is initiated as the map origin, estimates the moving distance of the robot relative to the starting position through the odometer, and updates the pose information of the robot in real time; every time the robot moves a fixed distance, the scan frame obtained by the current lidar is taken as a key frame, and the pose information of the current robot obtained from the odometer is taken as the robot pose of the key frame; For the point cloud data of each key frame, a BEV image is generated by voxelization; then, data augmentation by rotation is performed on each BEV image to generate multiple rotated BEV images; next, the local features of each BEV image are extracted, and the local features are extracted by the residual network Resnet; and the local features are encoded to generate global features, and the global features are encoded by the NetVLAD network; Bind the robot pose, BEV image, local features, and global features corresponding to each key frame, and store them in the feature library; after the mapping is completed, the point cloud data of the global map and the feature library composed of all key frames are obtained; The positioning stage includes the following steps: The robot obtains the current scan frame through lidar scanning; for the point cloud data of the current scan frame, a BEV image is generated by voxelization; then, the local features of each BEV image are extracted, and the local features are encoded to generate global features; Retrieve the global features that match the global features of the current scan frame in the feature library, and obtain the local features and robot pose information corresponding to the matching global features; Obtain the corresponding points by brute-force matching the local features of the current scan frame with the obtained local features, and the brute-force matching uses a feature point matching algorithm; and perform pose estimation on the corresponding points to obtain the relative transformation matrix between the local features of the current scan frame and the obtained local features; then apply the relative transformation matrix to the obtained robot pose information to obtain the current global position information of the robot.

2. The global relocalization method of a mobile robot according to claim 1, wherein The fixed distance is 0.5 meters.

3. A global relocalization method for a mobile robot according to claim 1, characterized in that, The method of data augmentation by rotation is to rotate 45 degrees each time, generating 8 rotated BEV images.

4. A global relocalization method for a mobile robot according to claim 1, characterized in that The pose estimation uses the RANSAC algorithm.

5. A global repositioning system for a mobile robot, which utilizes a global repositioning method for a mobile robot as described in claim 1, characterized in that It includes a mapping module and a positioning module; The mapping module includes a key frame generation unit, a key frame processing unit, and a feature library generation unit; Among them, the key frame generation unit is used for: the robot takes the starting position where the mapping operation is initiated as the map origin, estimates the moving distance of the robot relative to the starting position through the odometer, and updates the pose information of the robot in real time; every time the robot moves a fixed distance, the scan frame obtained by the current lidar is taken as a key frame, and the pose information of the current robot obtained from the odometer is taken as the robot pose of the key frame; The key frame processing unit is used for: for the point cloud data of each key frame, a BEV image is generated by voxelization; then, data augmentation by rotation is performed on each BEV image to generate multiple rotated BEV images; next, the local features of each BEV image are extracted, and the local features are generated into global features; The feature library generation unit is configured to: bind the robot pose, BEV image, local features, and global features corresponding to each key frame, and store them in the feature library; after mapping is completed, obtain the point cloud data of the global map and the feature library composed of all key frames; The positioning module includes a global feature generation unit, a feature library retrieval unit, and a global position acquisition unit; Among them, the global feature generation unit is configured to: the robot obtains the current scan frame through lidar scanning; for the point cloud data of the current scan frame, generate a BEV image by voxelization; then extract the local features of each BEV image, and generate global features from the local features; The feature library retrieval unit is configured to: retrieve the global features matching the global features of the current scan frame in the feature library, and obtain the local features and robot pose information corresponding to the matching global features; The global position acquisition unit is configured to: obtain the corresponding points by brute-force matching of the local features of the current scan frame and the obtained local features; and perform pose estimation on the corresponding points to obtain the relative transformation matrix between the local features of the current scan frame and the obtained local features; then apply the relative transformation matrix to the obtained robot pose information to obtain the current global position information of the robot.

6. A global relocalization device for a mobile robot, characterized in that, including at least one processor; and a memory connected to at least one of the processors; wherein, the memory stores instructions executable by the processor; The instructions are used to be executed by the processor to implement a global repositioning method for a mobile robot according to any one of claims 2 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a global repositioning method for a mobile robot according to any one of claims 2 to 4.

Images