Positioning methods, devices, equipment, and storage media for mobile robots

Abstract

This invention provides a method, apparatus, device, and storage medium for locating a mobile robot. The method includes: acquiring first data collected by the mobile robot; the first data includes at least spatial data, image data, and motion data; performing time alignment processing on the spatial data and the image data to obtain second data; performing coupling processing on the second data and the motion data to obtain third data; performing preprocessing on the spatial data and the image data respectively to obtain first target data corresponding to the spatial data and second target data corresponding to the image data; determining the pose information of the mobile robot based on the third data, the first target data, and the second target data; the pose information is used for locating the mobile robot.

Description

Technical Field

[0001] This invention relates to the field of system positioning technology, and in particular to a positioning method, apparatus, device, and storage medium for a mobile robot. Background Technology

[0002] With the continuous development of industrial automation and intelligence, more and more enterprises are adopting robots to replace manual labor for factory inspection tasks. While traditional manual inspection has been used for a long time, it has revealed many problems with the expansion of production scale and the increasing complexity of factory environments. First, manual inspection is inefficient, requiring a large amount of manpower and time, especially in large-scale production environments where it often cannot cover every corner and may even miss potential fault risks. Second, certain areas within the factory, such as those with high temperatures, high pressures, or toxic gases, pose a threat to the safety of inspection personnel, making manual inspection risky. Furthermore, manual inspection is susceptible to human fatigue and subjective factors, making it difficult to guarantee consistency and accuracy in each inspection, thus leading to the failure to detect potential problems in a timely manner. Therefore, factory inspection is gradually moving towards automation and intelligence, and using robots to replace manual inspection has become a key means to improve work efficiency, ensure safety, and enhance accuracy.

[0003] However, traditional inspection robots still face several technical challenges in practical applications. While existing LiDAR-based or vision-based inspection robots can achieve self-localization and environmental perception, their performance in complex factory environments remains problematic. First, traditional LiDAR SLAM (Simultaneous Localization and Mapping) technology, especially in dynamic environments, is prone to localization drift. Due to the constantly changing obstacles and irregular spatial layouts often present in factory environments, traditional methods cannot effectively handle these changes, leading to a gradual accumulation of localization errors that affect subsequent inspection tasks. Second, traditional inspection robots typically rely on a single sensor for self-localization. This method is susceptible to sensor noise and environmental interference in complex scenarios, resulting in insufficient localization accuracy. Furthermore, existing methods often fail to effectively address real-time issues. In fast-moving environments, robots need to process large amounts of sensor data in real time, a requirement that traditional methods often cannot meet, preventing robots from achieving real-time, efficient inspections. Summary of the Invention

[0004] This invention provides a positioning method, apparatus, device, and storage medium for mobile robots, which solves technical problems such as positioning drift and insufficient positioning accuracy in mobile robots.

[0005] This invention provides a method for locating a mobile robot, the method comprising:

[0006] The first data collected by the mobile robot is acquired; the first data includes at least spatial data, image data, and motion data.

[0007] The spatial data and the image data are time-aligned to obtain the second data;

[0008] The second data and the motion data are coupled to obtain the third data;

[0009] The spatial data and the image data are preprocessed respectively to obtain first target data corresponding to the spatial data and second target data corresponding to the image data;

[0010] The pose information of the mobile robot is determined based on the third data, the first target data, and the second target data; the pose information is used to locate the mobile robot.

[0011] In some embodiments, the mobile robot includes a lidar, a binocular camera, and an inertial measurement unit (IMU); acquiring the first data collected by the mobile robot includes:

[0012] The system acquires the spatial data collected by the lidar, the image data collected by the binocular camera, and the motion data collected by the IMU.

[0013] In some implementations, the step of performing time alignment processing on the spatial data and the image data to obtain the second data includes:

[0014] The spatial data and the image data are aligned through time synchronization to obtain fourth data under the same time reference;

[0015] The fourth data is then processed to align its spatial coordinates to obtain the second data.

[0016] In some implementations, the coupling process of the second data and the motion data to obtain the third data includes:

[0017] Based on the motion data, determine the motion estimation model of the mobile robot;

[0018] The motion model of the mobile robot is determined based on the second data;

[0019] The motion data, the second data, the motion estimation model, and the motion model are processed by a preset coupled lidar-inertial odometry method (LIO-SAM) to obtain the third data.

[0020] In some embodiments, the spatial data includes first point cloud data; the image data includes second point cloud data; the preprocessing of the spatial data and the image data to obtain first target data corresponding to the spatial data and second target data corresponding to the image data includes:

[0021] The first point cloud data is filtered and downsampled to obtain the third point cloud data after removing outliers and noise points from the first point cloud data.

[0022] The third point cloud data is processed by a preset feature extraction algorithm to obtain the first target data;

[0023] The second point cloud data is subjected to distortion correction processing to obtain the fourth point cloud data in the second point cloud data after removing invalid points;

[0024] The fourth point cloud data is matched using a preset feature matching algorithm to obtain the second target data.

[0025] In some implementations, determining the pose information of the mobile robot based on the third data, the first target data, and the second target data includes:

[0026] The first target data and the second target data are weighted by a preset point cloud matching algorithm NDT to obtain the fifth data.

[0027] The pose information of the mobile robot is determined based on the third and fifth data.

[0028] In some embodiments, the method further includes:

[0029] The first map stored in the mobile robot is updated based on the pose information to obtain the second map;

[0030] When the spatial data and the image data indicate the presence of an obstacle, environmental information about the obstacle is obtained;

[0031] The path information of the mobile robot is planned based on the environmental information and the second map.

[0032] This invention also provides a positioning device for a mobile robot, the device comprising:

[0033] The acquisition unit is used to acquire first data collected by the mobile robot; the first data includes at least spatial data, image data, and motion data.

[0034] The first processing unit is used to perform time alignment processing on the spatial data and the image data to obtain the second data;

[0035] The second processing unit is used to couple the second data and the motion data to obtain the third data.

[0036] The third processing unit is used to preprocess the spatial data and the image data respectively to obtain the first target data corresponding to the spatial data and the second target data corresponding to the image data;

[0037] The determining unit is used to determine the pose information of the mobile robot based on the third data, the first target data, and the second target data; the pose information is used to locate the mobile robot.

[0038] This invention provides a positioning device for a mobile robot, the device comprising: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, executes the steps of any of the methods described above.

[0039] This invention provides a storage medium storing a computer program; when the computer program is executed by a processor, it implements the steps of any of the methods described above.

[0040] This invention provides a method for locating a mobile robot. The method includes: acquiring first data collected by the mobile robot; the first data includes at least spatial data, image data, and motion data; performing time alignment processing on the spatial data and the image data to obtain second data; performing coupling processing on the second data and the motion data to obtain third data; performing preprocessing on the spatial data and the image data respectively to obtain first target data corresponding to the spatial data and second target data corresponding to the image data; determining the pose information of the mobile robot based on the third data, the first target data, and the second target data; the pose information is used for locating the mobile robot. Using the technical solution of this application, the second data is obtained by performing time alignment processing on the spatial data and image data collected by the mobile robot; the third data is obtained by coupling processing on the second data and the motion data collected by the mobile robot; the first target data corresponding to the spatial data and the second target data corresponding to the image data are preprocessed respectively; and pose information for locating the mobile robot is determined based on the third data, the first target data, and the second target data. That is, by processing the spatial data, image data and motion data collected by the mobile robot, and determining the pose information used to locate the mobile robot, the technical problems such as positioning drift and insufficient positioning accuracy of the mobile robot are solved, and the positioning accuracy is greatly improved. Attached Figure Description

[0041] Figure 1 A flowchart illustrating a positioning method for a mobile robot provided in an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of the structure of a positioning device for a mobile robot provided in an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of a hardware structure for a positioning device of a mobile robot according to an embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] The specific technical features described in the various embodiments in the detailed implementation can be combined in various ways without contradiction. For example, different implementation methods can be formed by combining different specific technical features. In order to avoid unnecessary repetition, the various possible combinations of the specific technical features in this invention will not be described separately.

[0046] It should also be noted that, in order to avoid obscuring the present invention with unnecessary details, only the structures and / or processing steps closely related to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0047] Additionally, it should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. In the following description, the terms "first," "second," etc., are used merely to distinguish different objects and do not indicate any similarity or connection between them. It should be understood that the directional descriptions such as "above," "below," "inside," and "outside" refer to the orientation under normal use conditions.

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the specific technical solutions of the invention will be further described in detail below with reference to the accompanying drawings of the embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0049] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0050] This invention provides a positioning method for a mobile robot, such as... Figure 1 As shown, Figure 1 This is a flowchart illustrating a positioning method for a mobile robot provided in an embodiment of the present invention; the method includes:

[0051] Step S101: Obtain the first data collected by the mobile robot; the first data includes at least spatial data, image data, and motion data.

[0052] Step S102: Perform time alignment processing on the spatial data and the image data to obtain the second data.

[0053] Step S103: Couple the second data and the motion data to obtain the third data.

[0054] Step S104: Preprocess the spatial data and the image data respectively to obtain the first target data corresponding to the spatial data and the second target data corresponding to the image data.

[0055] Step S105: Determine the pose information of the mobile robot based on the third data, the first target data, and the second target data; the pose information is used to locate the mobile robot.

[0056] In this embodiment, the localization method of the mobile robot can be determined according to the actual situation and is not limited here. As an example, the localization method of the mobile robot can be a real-time localization method for mobile robots based on LiDAR SLAM technology.

[0057] The mobile robot can be determined based on actual circumstances and is not limited here. As an example, the mobile robot may include an inspection robot.

[0058] In step S101, the first data includes at least spatial data, image data, and motion data; wherein the spatial data, image data, and motion data can all be determined according to actual conditions, and are not limited here. As an example, the spatial data may include three-dimensional point cloud data; the image data may include visual images; and the motion data may include acceleration and angular velocity data.

[0059] The specific acquisition process for obtaining the first data collected by the mobile robot can be determined according to the actual situation and is not limited here. As an example, the mobile robot includes a lidar, a binocular camera, and an inertial measurement unit (IMU); the acquisition of the first data collected by the mobile robot may include: acquiring the spatial data collected by the lidar, the image data collected by the binocular camera, and the motion data collected by the IMU.

[0060] In step S102, the specific processing procedure for time-aligning the spatial data and the image data to obtain the second data can be determined according to the actual situation and is not limited here. As an example, the time-aligning process for the spatial data and the image data to obtain the second data may include: aligning the spatial data and the image data through time synchronization to obtain fourth data where the spatial data and the image data are on the same time base; and performing spatial coordinate alignment on the fourth data to obtain the second data.

[0061] In step S103, the specific processing procedure for coupling the second data and the motion data to obtain the third data can be determined according to the actual situation and is not limited here. As an example, the coupling processing of the second data and the motion data to obtain the third data may include: determining the motion estimation model of the mobile robot based on the motion data; determining the motion model of the mobile robot based on the second data; and processing the motion data, the second data, the motion estimation model, and the motion model using a preset coupled LiDAR-Inertial Odometry (LIO-SAM) method to obtain the third data.

[0062] In step S104, the specific preprocessing steps for preprocessing the spatial data and the image data to obtain the first target data corresponding to the spatial data and the second target data corresponding to the image data can be determined according to the actual situation and are not limited here. As an example, the spatial data includes first point cloud data; the image data includes second point cloud data; the preprocessing of the spatial data and the image data to obtain the first target data corresponding to the spatial data and the second target data corresponding to the image data may include: filtering and downsampling the first point cloud data to obtain third point cloud data from the first point cloud data after removing outliers and noise points; extracting the third point cloud data using a preset feature extraction algorithm to obtain the first target data; performing distortion correction on the second point cloud data to obtain fourth point cloud data from the second point cloud data after removing invalid points; and matching the fourth point cloud data using a preset feature matching algorithm to obtain the second target data.

[0063] In step S105, the specific determination process for determining the pose information of the mobile robot based on the third data, the first target data, and the second target data can be determined according to the actual situation and is not limited here. As an example, determining the pose information of the mobile robot based on the third data, the first target data, and the second target data may include: weighting the first target data and the second target data using a preset point cloud matching algorithm NDT to obtain fifth data; and determining the pose information of the mobile robot based on the third data and the fifth data.

[0064] This invention provides a method for locating a mobile robot. The method involves time-aligning spatial and image data collected by the mobile robot to obtain second data; coupling the second data with motion data collected by the mobile robot to obtain third data; preprocessing the spatial and image data to obtain first target data corresponding to the spatial data and second target data corresponding to the image data; and determining pose information for locating the mobile robot based on the third data, the first target data, and the second target data. In other words, by processing the spatial, image, and motion data collected by the mobile robot and determining the pose information for locating the mobile robot, this method solves technical problems such as positioning drift and insufficient positioning accuracy, and significantly improves positioning accuracy.

[0065] In some embodiments, the mobile robot includes a lidar, a binocular camera, and an inertial measurement unit (IMU); acquiring the first data collected by the mobile robot includes:

[0066] The system acquires the spatial data collected by the lidar, the image data collected by the binocular camera, and the motion data collected by the IMU.

[0067] In this embodiment, the spatial data, image data, and motion data can all be determined according to actual conditions, and are not limited here. As an example, the spatial data may include 3D point cloud data; the image data may include visual images; and the motion data may include acceleration and angular velocity data. The acceleration data may include an acceleration vector, which can be denoted as... The angular velocity data may include an angular velocity vector, which can be denoted as ; .

[0068] The aforementioned lidar can also be called LiDAR.

[0069] In practical applications, the robot automatically powers on at a predetermined time, launching the operating system and various hardware drivers. During initialization, the robot self-checks key modules such as the LiDAR, binocular camera, and IMU to ensure the normal operation of each sensor. After initialization, the robot enters standby mode, ready to collect environmental data and execute tasks. Once the robot starts, the LiDAR and binocular camera begin collecting data about the surrounding environment in real time. The LiDAR collects 3D point cloud data, providing spatial information about the environment; the binocular camera acquires visual images, providing surface features of the environment. The IMU begins recording acceleration and angular velocity data to assist in dynamic motion estimation. This sensor data provides the foundation for subsequent map building and localization.

[0070] As an example, specifically, upon robot startup, the operating system and hardware drivers are first loaded and initialized. The robot initiates a self-test program to ensure the proper functioning of key sensors such as the LiDAR, binocular cameras, and IMU. After startup, the robot performs a hardware self-test to verify the sensor status, including the LiDAR's scanning range, the camera's image quality, and the IMU's acceleration and angular velocity data. Once the sensor self-test is complete, the robot enters standby mode, waiting to receive the start signal for the inspection task. When the task start command is received, the robot activates the data acquisition and path planning modules, preparing to collect environmental data and execute the task. After startup, the sensors begin collecting data about the surrounding environment in real time. The LiDAR begins generating 3D point cloud data through 360-degree scanning. This process measures the distance from points to the sensors using laser scanning, generating the 3D coordinates of each point. The binocular cameras acquire high-resolution images to capture visual features of the environment, assisting the robot's recognition and localization. The IMU begins recording the robot's acceleration and angular velocity data, which are used for subsequent dynamic motion estimation. The IMU's output is typically an acceleration vector. and angular velocity vector These sensors serve as motion state information for the robot, supporting subsequent algorithms. All the data from these sensors is fed into the robot's control system for real-time fusion and processing, ensuring data integrity and accuracy.

[0071] In some embodiments, performing time alignment processing on the spatial data and the image data to obtain second data includes:

[0072] The spatial data and the image data are aligned through time synchronization to obtain fourth data under the same time reference;

[0073] The fourth data is then processed to align its spatial coordinates to obtain the second data.

[0074] In this embodiment, the spatial data can be point cloud data, which can be understood as point cloud data collected by a LiDAR (Light Detection and Ranging) system. The image data can be understood as image data collected by a camera.

[0075] The specific processing procedure for aligning the spatial data and the image data through time synchronization to obtain the fourth data under the same time reference can be determined according to the actual situation and is not limited here. As an example, the alignment process for aligning the spatial data and the image data through time synchronization to obtain the fourth data under the same time reference can be the alignment process for aligning the point cloud data and the image data through time synchronization to obtain the fourth data under the same time reference.

[0076] The specific processing steps for aligning the fourth data to obtain the second data can be determined based on actual circumstances and are not limited here. As an example, aligning the fourth data to obtain the second data can be achieved by using a spatial transformation matrix to align the fourth data. The spatial transformation matrix can include a rotation matrix and a translation vector; the rotation matrix can be denoted as... The translation vector can be denoted as: .

[0077] In practical applications, point cloud data acquired by LiDAR and cameras undergo preprocessing to remove noise and invalid points. For the point cloud data, the robot performs filtering, downsampling, and feature point extraction (such as corner points and planar points). Camera data undergoes distortion correction, feature extraction, and matching. The preprocessed data provides high-quality information for subsequent weighted NDT matching and map construction. Specifically, as an example, to achieve accurate data fusion, the robot integrates camera image data and LiDAR point cloud data into the same coordinate frame. First, the camera and LiDAR data are aligned through time synchronization to ensure that the data from both sensors are used under the same time reference. Next, a spatial transformation matrix is ​​used to align the camera coordinate system and the LiDAR coordinate system. If the LiDAR coordinates are... The coordinates of the camera are Then through the rotation matrix Translation vector Align these two coordinate systems; the transformation formula is: In this way, the robot can integrate data from LiDAR and cameras into the same three-dimensional coordinate framework, ensuring accurate fusion of environmental information and thus improving the robot's positioning and navigation capabilities.

[0078] In some embodiments, the coupling process of the second data and the motion data to obtain the third data includes:

[0079] Based on the motion data, determine the motion estimation model of the mobile robot;

[0080] The motion model of the mobile robot is determined based on the second data;

[0081] The motion data, the second data, the motion estimation model, and the motion model are processed by a preset coupled lidar-inertial odometry method (LIO-SAM) to obtain the third data.

[0082] In this embodiment, the motion estimation model of the mobile robot can be denoted as: This can also be understood as the motion estimation model of the IMU.

[0083] The second data can be understood as data from the LiDAR and camera, and can be denoted as... and The motion model of the mobile robot can be understood as the motion model corresponding to the data from the lidar and camera, and can be denoted as... and .

[0084] The third data can be determined based on the actual situation and is not limited here. As an example, the third data can be the robot's state data, which can be denoted as... .

[0085] In practical applications, supported by the LIO-SAM algorithm, the robot tightly couples IMU data with camera and LiDAR data. The high-frequency motion information (such as acceleration and angular velocity) provided by the IMU helps the robot better estimate its dynamic motion and correct positioning errors from the LiDAR and camera data. This tight coupling method improves the system's robustness and accuracy in dynamic and complex environments. Specifically, as an example, supported by the LIO-SAM algorithm, IMU data is tightly coupled with camera and LiDAR data. The high-frequency acceleration provided by the IMU... and angular velocity Data helps robots correct positioning errors from LiDAR and camera data in highly dynamic environments. Through a tightly coupled approach, IMU data can directly influence the pose estimation process. The LIO-SAM algorithm achieves the fusion of IMU and sensor data by optimizing an objective function.

[0086] ;

[0087] in, Indicates the robot's state. and It contains data from LiDAR and cameras. and It is the corresponding motion model. It is the motion estimation model of the IMU. These are weighting coefficients. This method of fusing IMU data with LiDAR and camera data significantly improves positioning accuracy, especially in dynamic environments.

[0088] In some embodiments, the spatial data includes first point cloud data; the image data includes second point cloud data; the preprocessing of the spatial data and the image data to obtain first target data corresponding to the spatial data and second target data corresponding to the image data includes:

[0089] The first point cloud data is filtered and downsampled to obtain the third point cloud data after removing outliers and noise points from the first point cloud data.

[0090] The third point cloud data is processed by a preset feature extraction algorithm to obtain the first target data;

[0091] The second point cloud data is subjected to distortion correction processing to obtain the fourth point cloud data in the second point cloud data after removing invalid points;

[0092] The fourth point cloud data is matched using a preset feature matching algorithm to obtain the second target data.

[0093] In this embodiment, the feature extraction algorithm can be determined according to the actual situation and is not limited here. As an example, the feature extraction algorithm may include SIFT or SURF.

[0094] Both the first target data and the second target data can be determined according to the actual situation, and are not limited here. As an example, both the first target data and the second target data can be understood as key points, feature points, or key feature points.

[0095] In practical applications, point cloud data acquired by LiDAR and cameras undergo preprocessing to remove noise and invalid points. For the point cloud data, the robot performs filtering, downsampling, and feature point extraction (such as corner points and planar points). Camera data undergoes distortion correction, feature extraction, and matching. The preprocessed data provides high-quality information for subsequent weighted NDT matching and map construction. Specifically, as an example, point cloud data acquired by LiDAR and cameras is preprocessed to improve the accuracy and efficiency of subsequent processing. For LiDAR data, filtering and downsampling are first performed to remove outliers and noise. Then, feature extraction algorithms, such as SIFT or SURF, are used to extract key points or feature points. For camera data, distortion correction is first performed to eliminate the effects of lens distortion. Then, key feature points in the image are extracted using feature matching methods for matching with the point cloud data. The preprocessed data is then fed into the weighted NDT algorithm, providing accurate input for subsequent point cloud matching and map construction.

[0096] In some embodiments, determining the pose information of the mobile robot based on the third data, the first target data, and the second target data includes:

[0097] The first target data and the second target data are weighted by a preset point cloud matching algorithm NDT to obtain the fifth data.

[0098] The pose information of the mobile robot is determined based on the third and fifth data.

[0099] In this embodiment, the robot uses a weighted NDT algorithm to match the preprocessed point cloud data. Unlike traditional NDT, weighted NDT assigns different weights to points based on their distance and surface features. Points that are farther away and those with stronger surface features will have a greater impact on the matching results, thereby improving the accuracy of point cloud matching. During each scan, the algorithm calculates the optimal transformation between the current frame and historical frames.

[0100] The first target data and the second target data are weighted using a preset point cloud matching algorithm (NDT) to obtain the fifth data, which can be referenced. ;in, It is the first in point cloud data One point, It is the transformed point position. This is the weight of that point. Weight The settings are based on factors such as point distance and surface features. Weighted NDT makes important points (such as nearby points and points with obvious surface features) have a greater impact on the matching results, thereby improving matching accuracy.

[0101] The specific determination process for determining the pose information of the mobile robot based on the third and fifth data can be determined according to actual circumstances and is not limited here. As an example, determining the pose information of the mobile robot based on the third and fifth data can be done by using the maximum likelihood estimation (MLE) optimization method based on the third and fifth data. In practical applications, based on the weighted NDT calculation, the robot calculates the optimal registration between the current scan data and the existing map using an optimization algorithm. This process uses the maximum likelihood estimation (MLE) optimization method to minimize the matching error and obtain the robot's accurate pose. This step ensures that the robot's position information has high accuracy and reduces positioning drift. Specifically, the purpose of optimal matching is to calculate the optimal registration between the current point cloud data and the historical map using an optimization algorithm. The maximum likelihood estimation (MLE) method is used to minimize the point cloud matching error. Assuming the current point cloud is... Historical map is The optimization objective function is: ;in It is a transformation matrix, and the goal is to find the optimal transformation matrix. This minimizes the error between the current point cloud and the map. Through iterative optimization, the optimal robot pose is finally obtained, reducing localization drift and ensuring accurate robot localization.

[0102] In some embodiments, the method further includes:

[0103] The first map stored in the mobile robot is updated based on the pose information to obtain the second map;

[0104] When the spatial data and the image data indicate the presence of an obstacle, environmental information about the obstacle is obtained;

[0105] The path information of the mobile robot is planned based on the environmental information and the second map.

[0106] In this embodiment, the first map can be determined according to the actual situation, and is not limited here. As an example, the first map can be understood as an existing map, a current map, or a historical map.

[0107] The obstacles can be determined based on the actual situation and are not limited here. As an example, the obstacles may include walls, equipment, or other static obstacles.

[0108] The environmental information can be determined based on the actual situation and is not limited here. As an example, the environmental information can be understood as new environmental information.

[0109] The specific planning process for planning the mobile robot's path based on the environmental information and the second map can be determined according to the actual situation and is not limited here. As an example, planning the mobile robot's path based on the environmental information and the second map can be understood as calculating the shortest path based on the real-time map and environmental data.

[0110] In practical applications, after each point cloud matching, the robot merges the new point cloud data into the existing map. During real-time map building, the robot gradually expands the map of known areas while optimizing previous map sections to reduce error accumulation. This continuous updating process enables the robot to perform precise positioning and navigation in uncertain factory environments. The robot uses data from LiDAR and cameras to analyze the environment and identify obstacles on its path. When an obstacle is detected, the robot uses algorithms to determine the type of obstacle (e.g., static or dynamic) and adjusts its path planning accordingly to avoid collisions. Simultaneously, the robot analyzes the operating status of equipment to determine whether intensive inspection is required. Based on the real-time map and environmental analysis data, the robot uses the A* algorithm for path planning. The robot selects the shortest path while avoiding obstacles and areas that need to be avoided. In dynamic environments, the robot updates its path based on real-time perception data to ensure that the inspection task is completed efficiently and safely. Based on the path planning results, the robot performs the inspection task along the predetermined route. During this process, LiDAR and IMU continue to provide data to ensure that the robot accurately follows the planned path. The robot will continuously perceive its environment and adjust its position and posture in real time to adapt to possible obstacles and environmental changes.

[0111] As an example, specifically, after each point cloud matching, the robot merges the new point cloud data into the existing map, thus performing real-time map building and updating. Using a weighted NDT algorithm, the robot updates obstacles, environmental information, and other key features in the map with each matching. The core objective of map building is to maintain a dynamic and accurate environmental model to facilitate robot navigation and localization. During map updates, the robot's position and angle are corrected using an optimization matching algorithm to ensure that the map does not become distorted due to the accumulation of small errors over long-term use. The real-time update formula is: ;

[0112] in, This is the updated map. It is the original map. This refers to newly added point cloud data or map regions. Each time it updates, the robot uses a weighted NDT algorithm to match the new point cloud data with the current map, ensuring map continuity and accuracy. The robot performs environmental analysis and obstacle detection using LiDAR and camera data. Through LiDAR scans, the robot can identify and classify obstacles on its path, such as walls, equipment, or other static obstacles. Additionally, the robot uses camera data to identify dynamic obstacles, such as people or vehicles. When an obstacle is detected, the robot adjusts its path planning based on its type (static or dynamic) to avoid collisions. Obstacle detection is based on clustering analysis of point cloud data. Assuming the obstacle's shape is spherical or cubic, the obstacle detection formula is:

[0113] ;

[0114] in, It is the first in point cloud One point, This is the robot's current position. This is the preset obstacle detection radius. When an obstacle is detected, the robot updates its path planning based on the new environmental information. Path planning is one of the core tasks of a robot, aiming to calculate the shortest path based on real-time maps and environmental data. Robots typically use the A* algorithm for path planning, which can calculate the optimal path based on the distance between nodes. The A* algorithm is combined with heuristic functions. and path cost To calculate the total cost of each node The specific calculation steps are as follows:

[0115] 1. Initialize the start and end points.

[0116] 2. Calculate the cost of each node and put it into a priority queue.

[0117] 3. Select the next node in the order of least cost to expand until the endpoint is found.

[0118] 4. Return the shortest path from the origin to the destination.

[0119] In dynamic environments, robots continuously update their path planning, detect obstacles in real time, and replan their paths.

[0120] Based on path planning, the robot performs inspection tasks along the planned path. During execution, LiDAR and IMU sensors continuously provide data to ensure the robot accurately follows the planned path. The robot adjusts its position and orientation in real time by comparing real-time point cloud data with the map. The robot uses SLAM algorithms to register and optimize each frame of point cloud data to ensure the accuracy of position estimation. For example, the robot may perform a periodic optimization step to update its current pose. :

[0121] ;

[0122] in This represents the robot's position change, calculated based on the weighted NDT and LIO-SAM algorithms.

[0123] In practical applications, as an example, the positioning method for the mobile robot can be a real-time positioning method for mobile robots based on LiDAR SLAM technology, which can be implemented through the following steps.

[0124] I. Robot startup.

[0125] The robot automatically powers on at a predetermined time, starting the operating system and various hardware drivers. During initialization, the robot self-checks key modules such as the LiDAR, binocular camera, and IMU to ensure the proper functioning of all sensors. After initialization, the robot enters standby mode, ready to collect environmental data and execute tasks.

[0126] II. Real-time data collection of factory environment.

[0127] Once the robot is activated, the LiDAR and binocular cameras begin collecting data about the surrounding environment in real time. The LiDAR collects 3D point cloud data, providing spatial information about the environment; the binocular cameras acquire visual images, providing surface features of the environment. The IMU begins recording acceleration and angular velocity data to assist in the robot's dynamic motion estimation. This sensor data will provide the foundation for subsequent map building and localization.

[0128] Third, combine camera data and LiDAR data into one framework.

[0129] To ensure the accuracy of data fusion, the robot places image data from the camera and point cloud data from the LiDAR within the same reference frame. Through time synchronization and spatial coordinate alignment, the data from both is integrated into a unified framework. This allows the robot to simultaneously utilize spatial information from the LiDAR and visual information from the camera, improving the accuracy of its environmental perception.

[0130] Fourth, tightly couple camera data and lidar data with the IMU.

[0131] With the support of the LIO-SAM algorithm, the robot tightly couples IMU data with camera and LiDAR data. The high-frequency motion information (such as acceleration and angular velocity) provided by the IMU helps the robot better estimate its own dynamic motion and corrects positioning errors from LiDAR and camera data. This tight coupling method improves the system's robustness and accuracy in dynamic and complex environments.

[0132] V. Preprocessing of point cloud data and feature points acquired by radar and cameras.

[0133] Point cloud data acquired by LiDAR and cameras undergoes preprocessing to remove noise and invalid points. For the point cloud data, the robot performs filtering, downsampling, and feature point extraction (such as corner points and planar points). Camera data undergoes distortion correction, feature extraction, and matching. The preprocessed data provides high-quality information for subsequent weighted NDT matching and map construction.

[0134] VI. Weighted NDT Calculation.

[0135] The robot uses a weighted NDT algorithm to match preprocessed point cloud data. Unlike traditional NDT, weighted NDT assigns different weights to points based on their distance and surface features. Points that are farther away and have stronger surface features have a greater impact on the matching results, thus improving the accuracy of point cloud matching. During each scan, the algorithm calculates the optimal transformation between the current frame and historical frames.

[0136] VII. Optimal matching.

[0137] Based on the weighted NDT calculation, the robot uses an optimization algorithm to calculate the optimal registration between the current scan data and the existing map. This process uses the maximum likelihood estimation (MLE) optimization method to minimize the matching error and obtain the robot's precise pose. This step ensures high accuracy of the robot's position information and reduces positioning drift.

[0138] 8. Real-time map construction and updating.

[0139] After each point cloud matching, the robot merges the new point cloud data into the existing map. During real-time map building, the robot gradually expands the map of known areas while optimizing previous map sections to reduce error accumulation. This continuous updating process enables the robot to perform accurate positioning and navigation in uncertain factory environments.

[0140] IX. Environmental Analysis and Obstacle Detection.

[0141] The robot uses data from LiDAR and cameras to analyze the environment and identify obstacles in its path. When an obstacle is detected, the robot uses algorithms to determine the type of obstacle (e.g., static or dynamic) and adjusts its path planning accordingly to avoid collisions. Simultaneously, the robot analyzes the operational status of equipment to determine whether intensive inspection is necessary.

[0142] 10. Path planning.

[0143] Based on real-time maps and environmental analysis data, the robot uses the A* algorithm for path planning. The robot selects the shortest path while avoiding obstacles and areas that need to be avoided. In dynamic environments, the robot updates its path based on real-time perception data, ensuring that the inspection task can be completed efficiently and safely.

[0144] 11. Carry out inspection tasks along the planned route.

[0145] Based on the path planning results, the robot performs inspection tasks along a predetermined route. During this process, LiDAR and IMU continue to provide data to ensure that the robot accurately follows the planned path. The robot continuously perceives its environment and adjusts its position and posture in real time to adapt to possible obstacles and environmental changes.

[0146] 12. Equipment Inspection and Data Acquisition.

[0147] Upon reaching the inspection target, the robot will conduct a detailed inspection of the equipment. It uses LiDAR to scan the equipment surface and binocular cameras to capture images and collect relevant data. The robot also uses temperature and pressure sensors to monitor the equipment's status, ensuring its healthy operation. If any abnormalities are detected, the robot will promptly record them and report them to the central control system.

[0148] Thirteen, Data Upload and Synchronization.

[0149] Data collected in real time by the robot during inspections (point cloud data, image data, equipment status, etc.) will be uploaded to the central control platform via a wireless network. The platform will store and analyze the data to facilitate subsequent decision support and troubleshooting. All data upload and synchronization processes must ensure the integrity and accuracy of the information to promptly identify potential problems.

[0150] XIV. Location and Map Optimization.

[0151] During task execution, the robot periodically optimizes its localization and map. Using weighted NDT and LIO-SAM algorithms, the robot optimizes the previous map and its current location information, reducing error accumulation. The optimized map provides more accurate environmental information, thereby improving the robot's inspection accuracy and efficiency.

[0152] 15. Status Feedback and Adjustment.

[0153] During the inspection process, the robot monitors its own status in real time, such as battery level, sensor operating status, and task progress. If the robot encounters any problems during execution (such as insufficient battery or sensor malfunction), it will immediately report back to the central control system and make corresponding adjustments (such as returning to the charging station or switching to a backup sensor).

[0154] XVI. Report Generation and Uploading.

[0155] After completing the inspection task, the robot generates a detailed inspection report. The report includes the equipment's operating status, the areas inspected, any anomalies found, and the robot's inspection path and location data. The report is uploaded wirelessly to the company's management platform for human maintenance personnel to view and process. Simultaneously, the report is archived for future analysis and optimization.

[0156] 17. Return to the charging station.

[0157] After completing its task, the robot will automatically return to the charging station based on its battery level. During the return journey, the robot will recalculate its path to avoid any potential obstacles. If the battery is low, the robot will prioritize returning to the charging station and prepare for charging. The charging station will automatically adjust the charging process based on the robot's status to ensure charging efficiency and safety.

[0158] 18. System maintenance and updates.

[0159] After the robot completes a certain number of tasks, the system undergoes regular maintenance and updates. This includes sensor calibration, algorithm optimization, and hardware checks. By analyzing historical task data, the R&D team can adjust the algorithm to improve the robot's accuracy and efficiency. Regular maintenance and updates ensure that the robot is always in optimal working condition.

[0160] As an example, specific implementation methods include the following:

[0161] I. Robot startup.

[0162] Upon startup, the robot first loads and initializes the operating system and hardware drivers. It then initiates a self-test to ensure the proper functioning of critical sensors such as the LiDAR, binocular camera, and IMU. After startup, the robot performs a hardware self-test to verify the sensor status, including the LiDAR's scanning range, the camera's image quality, and the IMU's acceleration and angular velocity data. Once the sensor self-test is complete, the robot enters standby mode, awaiting the start signal for the inspection task. Upon receiving the task start command, the robot activates the data acquisition and path planning modules, preparing to collect environmental data and execute the task.

[0163] II. Real-time data collection of factory environment.

[0164] After the robot starts, sensors begin collecting data about the surrounding environment in real time. The LiDAR (Light Detection and Ranging) system begins generating 3D point cloud data through 360-degree scanning. This process measures the distance from each point to the sensor using laser scanning, generating the 3D coordinates of each point. The binocular cameras acquire high-resolution images to capture visual features of the environment, assisting the robot's recognition and localization. The IMU (Integrated Mutor Unit) begins recording the robot's acceleration and angular velocity data, which are used for subsequent dynamic motion estimation. The IMU's output is typically an acceleration vector. and angular velocity vector These sensors serve as motion state information for the robot, supporting subsequent algorithms. All the data from these sensors is fed into the robot's control system for real-time fusion and processing, ensuring data integrity and accuracy.

[0165] Third, combine camera data and LiDAR data into one framework.

[0166] To achieve accurate data fusion, the robot integrates image data from the camera and point cloud data from a LiDAR scanner into the same coordinate frame. First, the camera and LiDAR data are aligned through time synchronization to ensure that data from both sensors are used under the same time reference. Next, a spatial transformation matrix is ​​used to align the camera coordinate system and the LiDAR coordinate system. If the LiDAR coordinates are... The coordinates of the camera are Then through the rotation matrix Translation vector Align these two coordinate systems; the transformation formula is: In this way, the robot can integrate data from LiDAR and cameras into the same three-dimensional coordinate framework, ensuring accurate fusion of environmental information and thus improving the robot's positioning and navigation capabilities.

[0167] Fourth, tightly couple camera data and lidar data with the IMU.

[0168] With the support of the LIO-SAM algorithm, IMU data is tightly coupled with camera and LiDAR data. The IMU provides high-frequency acceleration... and angular velocity Data helps robots correct positioning errors from LiDAR and camera data in highly dynamic environments. Through a tightly coupled approach, IMU data can directly influence the pose estimation process. The LIO-SAM algorithm achieves the fusion of IMU and sensor data by optimizing an objective function.

[0169] ;

[0170] in, Indicates the robot's state. and It contains data from LiDAR and cameras. and It is the corresponding motion model. It is the motion estimation model of the IMU. These are weighting coefficients. This method of fusing IMU data with LiDAR and camera data significantly improves positioning accuracy, especially in dynamic environments.

[0171] V. Preprocessing of point cloud data and feature points acquired by radar and cameras.

[0172] Point cloud data acquired by LiDAR and cameras undergo preprocessing to improve the accuracy and efficiency of subsequent processing. For LiDAR data, filtering and downsampling are first performed to remove outliers and noise points. Then, feature extraction algorithms, such as SIFT or SURF, are used to extract key points or feature points. For camera data, distortion correction is first performed to eliminate the effects of lens distortion. Then, key feature points in the image are extracted using feature matching methods for matching with the point cloud data. The preprocessed data is then fed into a weighted NDT algorithm, providing accurate input for subsequent point cloud matching and map construction.

[0173] VI. Weighted NDT Calculation.

[0174] Weighted NDT (Normal Distributions Transform) is used for point cloud data matching. Traditional NDT methods treat all point cloud cells equally, while weighted NDT assigns different weights to each point based on its distance and surface features. This is achieved through weighting coefficients. Adjusting the degree of influence of the point cloud, the calculation process is as follows: ;

[0175] in, It is the first in point cloud data One point, It is the transformed point position. This is the weight of that point. Weight The settings are based on factors such as point distance and surface features. Weighted NDT makes important points (such as nearby points and points with obvious surface features) have a greater impact on the matching results, thereby improving matching accuracy.

[0176] VII. Optimal matching.

[0177] The goal of optimal matching is to calculate the optimal registration between the current point cloud data and the historical map using an optimization algorithm. The maximum likelihood estimation (MLE) method is used to minimize the point cloud matching error. Assume the current point cloud is... Historical map is The optimization objective function is: ;

[0178] in, It is a transformation matrix, and the goal is to find the optimal transformation matrix. This minimizes the error between the current point cloud and the map. Through iterative optimization, the optimal robot pose is finally obtained, reducing localization drift and ensuring accurate robot localization.

[0179] 8. Real-time map construction and updating.

[0180] After each point cloud matching, the robot merges the new point cloud data into the existing map, thus performing real-time map construction and updates. Using a weighted NDT algorithm, the robot updates obstacles, environmental information, and other key features in the map with each matching. The core objective of map construction is to maintain a dynamic and accurate environmental model to facilitate robot navigation and localization. During map updates, the robot's position and angle are corrected using an optimization matching algorithm to ensure that the map does not become distorted due to the accumulation of small errors over long-term use. The real-time update formula is: ;

[0181] in, This is the updated map. It is the original map. This refers to newly added point cloud data or map regions. Each time an update is performed, the robot uses a weighted NDT algorithm to match the new point cloud data with the current map, ensuring map continuity and accuracy.

[0182] IX. Environmental Analysis and Obstacle Detection.

[0183] The robot performs environmental analysis and obstacle detection using LiDAR and camera data. Through LiDAR scans, the robot can identify and classify obstacles along its path, such as walls, equipment, or other static obstacles. Additionally, the robot uses camera data to identify dynamic obstacles, such as people or vehicles. When an obstacle is detected, the robot adjusts its path planning based on its type (static or dynamic) to avoid collisions. Obstacle detection is based on clustering analysis of point cloud data. Assuming the obstacle's shape is spherical or cubic, the obstacle detection formula is: ;

[0184] in, It is the first in point cloud One point, This is the robot's current position. This is the preset obstacle detection radius. When an obstacle is detected, the robot updates its path planning based on the new environmental information.

[0185] 10. Path planning.

[0186] Path planning is one of the core tasks of robotics, aiming to calculate the shortest path based on real-time maps and environmental data. Robots typically use the A* algorithm for path planning, which calculates the optimal path based on the distances between nodes. The A* algorithm is combined with heuristic functions. and path cost To calculate the total cost of each node The specific calculation steps are as follows:

[0187] 1. Initialize the start and end points.

[0188] 2. Calculate the cost of each node and put it into a priority queue.

[0189] 3. Select the next node in the order of least cost to expand until the endpoint is found.

[0190] 4. Return the shortest path from the origin to the destination.

[0191] In dynamic environments, robots continuously update their path planning, detect obstacles in real time, and replan their paths.

[0192] 11. Carry out inspection tasks along the planned route.

[0193] Based on path planning, the robot performs inspection tasks along the planned path. During execution, LiDAR and IMU sensors continuously provide data to ensure the robot accurately follows the planned path. The robot adjusts its position and orientation in real time by comparing real-time point cloud data with the map. The robot uses SLAM algorithms to register and optimize each frame of point cloud data to ensure the accuracy of position estimation. For example, the robot may perform a periodic optimization step to update its current pose. : ;

[0194] in, This represents the robot's position change, calculated based on the weighted NDT and LIO-SAM algorithms.

[0195] 12. Equipment Inspection and Data Acquisition.

[0196] When the robot arrives at the inspection target, it begins its equipment inspection task. The robot uses LiDAR to scan the equipment surface and extract its features; simultaneously, a binocular camera captures images of the equipment and analyzes its status. The robot also uses sensors (such as temperature and pressure sensors) for real-time monitoring, detecting any abnormalities (such as overheating or damage). The robot compares the equipment status information with preset normal operating conditions and generates a report. If an anomaly is detected, the robot records the anomaly information and feeds it back to the central control system for further processing by human maintenance personnel.

[0197] Thirteen, Data Upload and Synchronization.

[0198] Data collected in real time by the robot during inspections, including point cloud data, image data, and sensor information, is uploaded to a central server via a wireless network. The data upload employs a distributed synchronization mechanism to ensure data integrity and real-time performance. During the upload process, the robot performs data compression and encryption to ensure data security and accuracy. The uploaded data is stored on a cloud platform for subsequent analysis and fault diagnosis. The specific synchronization formula is as follows: ;

[0199] in, It is the synchronized data. and These represent the robot's local data and the data on the server, respectively.

[0200] XIV. Location and Map Optimization.

[0201] During the inspection process, the robot periodically optimizes its localization and map to reduce error accumulation. Using weighted NDT and LIO-SAM algorithms, the robot performs backend optimization on the previously generated map, correcting localization drift that has occurred over long-term. The optimization process minimizes the error between the map and sensor data; the objective function is: ;

[0202] in, It is point cloud data. It's IMU data. It is a pose estimation function. It is a balancing weighting factor. Through this optimization, the robot can maintain high-precision positioning and map updates.

[0203] 15. Status Feedback and Adjustment.

[0204] The robot monitors its operational status in real time, including battery level, sensor status, and task progress. When the robot detects an anomaly (such as low battery or sensor malfunction), it immediately sends a feedback report to the central control system. The system then adjusts its actions based on the feedback, such as returning to a charging station or replacing sensors with backups. The feedback process utilizes a state estimation method. .

[0205] XVI. Report Generation and Uploading.

[0206] Once the inspection task is completed, the robot automatically generates a detailed inspection report, including equipment status, task completion details, and anomaly detection results. The report is uploaded to the management platform via wireless network for viewing and analysis by manual maintenance personnel. When generating the report, the robot summarizes all collected data and generates a report file based on a preset template.

[0207] 17. Return to the charging station.

[0208] Once the task is completed, the robot automatically returns to the charging station based on its battery level. If the battery is low, the robot will prioritize returning to the charging station and calculate the shortest path to recharge. The charging station will automatically adjust the charging process based on the robot's status to ensure charging efficiency and safety.

[0209] 18. System maintenance and updates.

[0210] The system undergoes regular maintenance and updates, including sensor calibration, algorithm optimization, and hardware checks. These maintenance and updates help improve the robot's accuracy and efficiency, ensuring long-term stable operation. Algorithm optimization, based on historical task data, adjusts the parameters of weighted NDT and LIO-SAM to improve positioning accuracy.

[0211] This application proposes a novel scheme based on the fusion of weighted NDT (Normal Distributions Transform) and LIO-SAM (Lidar Inertial Odometry and Mapping) algorithms. This scheme improves the self-localization and mapping accuracy of inspection robots in factory environments by combining weighted NDT with LIO-SAM. Weighted NDT, as an advanced point cloud matching method, improves matching accuracy compared to traditional NDT by weighting each cell in the point cloud and adjusting it according to different distances and surface features. In the dynamically changing environment of a factory, using weighted NDT can effectively reduce point cloud matching errors, especially during long-term operation, significantly reducing the risk of localization drift and improving the robot's navigation stability.

[0212] Furthermore, the LIO-SAM algorithm enhances the localization accuracy and robustness of the robot system by tightly integrating IMU (Inertial Measurement Unit) data with LiDAR data. The IMU provides high-frequency motion information, helping to correct errors in the LiDAR point cloud caused by environmental changes or dynamic obstacles. The combination of LIO-SAM enables the system to maintain low drift in dynamic environments and perform real-time self-localization and mapping, making it particularly suitable for rapidly changing factory environments. Through this fusion approach, the robot can not only perform efficient autonomous navigation but also adapt to various challenges in complex environments, such as obstacle avoidance, rapid movement, and real-time localization correction.

[0213] The proposed solution, based on a weighted NDT and LIO-SAM fusion algorithm, overcomes the shortcomings of traditional inspection robots in terms of accuracy, real-time performance, and robustness in factory environments. Weighted NDT refines the point cloud, improving matching accuracy and reducing positioning drift, while LIO-SAM further enhances system stability and robustness through tight coupling between the IMU and LiDAR. This novel approach not only improves the efficiency of factory inspection robots and reduces labor costs but also ensures high accuracy and safety during the inspection process, demonstrating broad application prospects.

[0214] Based on the same inventive concept as described above, Figure 2 This is a schematic diagram of the structure of a positioning device for a mobile robot provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the device 200 includes:

[0215] The acquisition unit 201 is used to acquire first data collected by the mobile robot; the first data includes at least spatial data, image data, and motion data.

[0216] The first processing unit 202 is used to perform time alignment processing on the spatial data and the image data to obtain the second data;

[0217] The second processing unit 203 is used to couple the second data and the motion data to obtain the third data;

[0218] The third processing unit 204 is used to preprocess the spatial data and the image data respectively to obtain the first target data corresponding to the spatial data and the second target data corresponding to the image data;

[0219] The determining unit 205 is used to determine the pose information of the mobile robot based on the third data, the first target data, and the second target data; the pose information is used to locate the mobile robot.

[0220] In some embodiments, the mobile robot includes a lidar, a binocular camera, and an inertial measurement unit (IMU); the acquisition unit 201 is further configured to acquire the spatial data collected by the lidar, the image data collected by the binocular camera, and the motion data collected by the IMU.

[0221] In some embodiments, the first processing unit 202 is further configured to perform time synchronization alignment processing on the spatial data and the image data to obtain fourth data under the same time reference; and perform spatial coordinate alignment processing on the fourth data to obtain the second data.

[0222] In some embodiments, the second processing unit 203 is further configured to determine a motion estimation model of the mobile robot based on the motion data; determine a motion model of the mobile robot based on the second data; and process the motion data, the second data, the motion estimation model, and the motion model using a preset coupled lidar-inertial odometry (LIO-SAM) method to obtain the third data.

[0223] In some embodiments, the spatial data includes first point cloud data; the image data includes second point cloud data; the third processing unit 204 is further configured to perform filtering and downsampling processing on the first point cloud data to obtain third point cloud data from the first point cloud data after removing outliers and noise points; extract the third point cloud data using a preset feature extraction algorithm to obtain the first target data; perform distortion correction processing on the second point cloud data to obtain fourth point cloud data from the second point cloud data after removing invalid points; and perform matching processing on the fourth point cloud data using a preset feature matching algorithm to obtain the second target data.

[0224] In some embodiments, the determining unit 205 is further configured to perform weighted processing on the first target data and the second target data using a preset point cloud matching algorithm NDT to obtain fifth data; and determine the pose information of the mobile robot based on the third data and the fifth data.

[0225] In some embodiments, the apparatus 200 further includes an updating unit and a planning unit; wherein,

[0226] The updating unit is used to update the first map stored in the mobile robot based on the pose information to obtain a second map.

[0227] The acquisition unit 201 is further configured to acquire environmental information of the obstacle when the spatial data and the image data indicate the presence of an obstacle;

[0228] The planning unit is used to plan the path information of the mobile robot based on the environmental information and the second map.

[0229] It should be noted that the positioning device for the mobile robot provided in this embodiment of the invention and the configuration method provided in the aforementioned embodiment of the invention belong to the same inventive concept. The meanings of the terms used here have been explained in detail above and will not be repeated here.

[0230] This invention also provides a storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0231] This invention also provides a positioning device for a mobile robot, the positioning device for the mobile robot comprising: a processor and a memory for storing a computer program capable of running on the processor, wherein, when the processor runs the computer program, it executes the steps of the above-described method embodiments stored in the memory.

[0232] Figure 3This is a schematic diagram of a hardware structure of a positioning device for a mobile robot according to an embodiment of the present invention. The positioning device 300 for the mobile robot includes at least one processor 301 and a memory 302. Optionally, the positioning device 300 may further include at least one communication interface 303. The various components in the positioning device 300 are coupled together through a bus system 304. It can be understood that the bus system 304 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 304 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 3 The general designated all buses as Bus System 304.

[0233] It is understood that memory 302 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Sync Link Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 302 described in this embodiment of the invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0234] In this embodiment of the invention, the memory 302 is used to store various types of data to support the operation of the positioning device 300 of the mobile robot. Examples of such data include any computer program for operation on the positioning device 300 of the mobile robot, and programs implementing the methods of this embodiment of the invention may be included in the memory 302.

[0235] The methods disclosed in the above embodiments of the present invention can be applied to processor 301, or implemented by processor 301. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of the present invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory. The processor reads information from the memory and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0236] In an exemplary embodiment, the positioning device 300 of the mobile robot may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the methods described above.

[0237] In the several embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units; some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. In addition, all functional units in the various embodiments of this invention can be integrated into one processing module, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated units can be implemented in hardware or in the form of hardware plus software functional units.

[0238] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. A positioning method for a mobile robot, characterized in that, The method includes: acquiring first data collected by the mobile robot; the first data includes at least spatial data, image data, and motion data; The spatial data and the image data are time-aligned to obtain the second data; The second data and the motion data are coupled to obtain the third data; The spatial data and the image data are preprocessed respectively to obtain first target data corresponding to the spatial data and second target data corresponding to the image data; The pose information of the mobile robot is determined based on the third data, the first target data, and the second target data; the pose information is used to locate the mobile robot. The mobile robot includes a lidar, a binocular camera, and an inertial measurement unit (IMU); acquiring the first data collected by the mobile robot includes: acquiring the spatial data collected by the lidar, the image data collected by the binocular camera, and the motion data collected by the IMU; The step of performing time alignment processing on the spatial data and the image data to obtain the second data includes: performing time synchronization alignment processing on the spatial data and the image data to obtain the fourth data of the spatial data and the image data under the same time base; The fourth data is subjected to spatial coordinate alignment processing to obtain the second data; The process of coupling the second data and the motion data to obtain the third data includes: determining the motion estimation model of the mobile robot based on the motion data; The motion model of the mobile robot is determined based on the second data; The motion data, the second data, the motion estimation model, and the motion model are processed by a preset coupled lidar-inertial odometry method (LIO-SAM) to obtain the third data.

2. The method according to claim 1, characterized in that, The spatial data includes first point cloud data; the image data includes second point cloud data; the preprocessing of the spatial data and the image data to obtain first target data corresponding to the spatial data and second target data corresponding to the image data includes: filtering and downsampling the first point cloud data to obtain third point cloud data in the first point cloud data after removing outliers and noise points; The third point cloud data is processed by a preset feature extraction algorithm to obtain the first target data; The second point cloud data is subjected to distortion correction processing to obtain the fourth point cloud data in the second point cloud data after removing invalid points; The fourth point cloud data is matched using a preset feature matching algorithm to obtain the second target data.

3. The method according to claim 1, characterized in that, The step of determining the pose information of the mobile robot based on the third data, the first target data, and the second target data includes: weighting the first target data and the second target data using a preset point cloud matching algorithm NDT to obtain the fifth data; The pose information of the mobile robot is determined based on the third and fifth data.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: updating a first map stored in the mobile robot based on the pose information to obtain a second map; When the spatial data and the image data indicate the presence of an obstacle, environmental information about the obstacle is obtained; The path information of the mobile robot is planned based on the environmental information and the second map.

5. A positioning device for a mobile robot, characterized in that, The device includes: an acquisition unit, configured to acquire first data collected by the mobile robot; the first data includes at least spatial data, image data, and motion data; The first processing unit is used to perform time alignment processing on the spatial data and the image data to obtain the second data; The second processing unit is used to couple the second data and the motion data to obtain the third data. The third processing unit is used to preprocess the spatial data and the image data respectively to obtain the first target data corresponding to the spatial data and the second target data corresponding to the image data; The determining unit is configured to determine the pose information of the mobile robot based on the third data, the first target data, and the second target data; the pose information is used to locate the mobile robot. The mobile robot includes a lidar, a binocular camera, and an inertial measurement unit (IMU); acquiring the first data collected by the mobile robot includes: acquiring the spatial data collected by the lidar, the image data collected by the binocular camera, and the motion data collected by the IMU; The step of performing time alignment processing on the spatial data and the image data to obtain the second data includes: performing time synchronization alignment processing on the spatial data and the image data to obtain the fourth data of the spatial data and the image data under the same time base; The fourth data is subjected to spatial coordinate alignment processing to obtain the second data; The process of coupling the second data and the motion data to obtain the third data includes: determining the motion estimation model of the mobile robot based on the motion data; The motion model of the mobile robot is determined based on the second data; The motion data, the second data, the motion estimation model, and the motion model are processed by a preset coupled lidar-inertial odometry method (LIO-SAM) to obtain the third data.

6. A positioning device for a mobile robot, characterized in that, The device includes: a processor and a memory for storing a computer program capable of running on the processor, wherein, when the processor runs the computer program, it performs the steps of the method according to any one of claims 1 to 4.

7. A storage medium, characterized in that, The storage medium stores a computer program; when the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images