Positioning method and device of mobile robot, equipment and storage medium

Abstract

The invention provides a positioning method and device of a mobile robot, equipment and a storage medium. The method comprises the steps of obtaining first data collected by the mobile robot; the first data at least comprises spatial data, image data and motion data; performing time alignment processing on the spatial data and the image data to obtain second data; performing coupling processing on the second data and the motion data to obtain third data; respectively preprocessing the spatial data and the image data to obtain first target data corresponding to the spatial data and second target data corresponding to the image data; determining pose information of the mobile robot based on the third data, the first target data and the second target data; the pose information is used for positioning the mobile robot.

Description

Technical Field

[0001] The present invention relates to the technical field of system positioning, and in particular to a positioning method, device, equipment, and storage medium for a mobile robot. Background Art

[0002] With the continuous advancement of industrial automation and intelligence, more and more companies are adopting robots to replace human workers in factory inspections. While traditional manual inspections have long been used, they have exposed numerous issues with the expansion of production scale and the increasing complexity of factory environments. First, manual inspections are inefficient and require significant manpower and time. Especially in large-scale production environments, manual inspections often fail to cover every corner and may even miss potential faults. Second, certain areas within factories, such as those with high temperatures, high pressures, or toxic gases, pose safety risks to inspectors, making manual inspections in these hazardous areas inherently risky. Furthermore, manual inspections are susceptible to fatigue and subjective factors, making consistency and accuracy in each inspection impossible, leading to potential problems not being discovered in a timely manner. Therefore, factory inspections are gradually moving towards automation and intelligence, and the use of robots to replace manual inspections has become a key means of improving efficiency, ensuring safety, and enhancing accuracy.

[0003] However, traditional inspection robots still face several technical challenges in practical application. While existing inspection robots based on Lidar (laser radar) or vision can achieve self-localization and environmental perception, their performance in complex factory environments remains problematic. First, traditional Lidar SLAM (Simultaneous Localization and Mapping) technology is prone to positioning drift, especially in dynamic environments. Because factory environments often feature constantly changing obstacles and irregular spatial layouts, traditional methods are unable to effectively cope with these changes, leading to the accumulation of positioning errors and impacting subsequent inspection tasks. Second, traditional inspection robots typically rely on a single sensor for self-localization. This approach is susceptible to sensor noise and environmental interference in complex scenarios, resulting in insufficient positioning accuracy. Furthermore, existing methods often fail to effectively address real-time requirements. In fast-moving environments, robots must process large amounts of sensor data in real time, but the computational efficiency of traditional methods often falls short, hindering real-time and efficient inspection. Summary of the Invention

[0004] The present invention provides a positioning method, device, equipment and storage medium for a mobile robot, which are used to solve technical problems such as positioning drift and insufficient positioning accuracy of the mobile robot.

[0005] An embodiment of the present invention provides a positioning method for a mobile robot, the method comprising: Acquiring first data collected by the mobile robot; the first data includes at least spatial data, image data, and motion data; performing time alignment processing on the spatial data and the image data to obtain second data; performing coupling processing on the second data and the motion data to obtain third data; Preprocessing the spatial data and the image data respectively to obtain first target data corresponding to the spatial data and second target data corresponding to the image data; The position information of the mobile robot is determined based on the third data, the first target data and the second target data; the position information is used to locate the mobile robot.

[0006] In some embodiments, the mobile robot includes a laser radar, a binocular camera, and an inertial measurement unit (IMU); and obtaining the first data collected by the mobile robot includes: Acquire the spatial data collected by the lidar, the image data collected by the binocular camera, and the motion data collected by the IMU.

[0007] In some embodiments, performing time alignment processing on the spatial data and the image data to obtain the second data includes: performing alignment processing on the spatial data and the image data through time synchronization to obtain fourth data of the spatial data and the image data under the same time reference; Perform spatial coordinate alignment processing on the fourth data to obtain the second data.

[0008] In some embodiments, coupling processing the second data and the motion data to obtain third data includes: determining a motion estimation model of the mobile robot based on the motion data; determining a motion model of the mobile robot according to the second data; The motion data, the second data, the motion estimation model, and the motion model are processed by a preset coupled laser radar-inertial odometry method LIO-SAM to obtain the third data.

[0009] In some embodiments, the spatial data includes first point cloud data; the image data includes second point cloud data; and preprocessing the spatial data and the image data to obtain first target data corresponding to the spatial data and second target data corresponding to the image data includes: Filtering and downsampling the first point cloud data to obtain third point cloud data in which outliers and noise points are removed from the first point cloud data; Extracting and processing the third point cloud data using a preset feature extraction algorithm to obtain the first target data; performing a dedistortion process on the second point cloud data to obtain fourth point cloud data from which invalid points are removed; The fourth point cloud data is matched using a preset feature matching algorithm to obtain the second target data.

[0010] In some embodiments, determining the pose information of the mobile robot based on the third data, the first target data, and the second target data includes: Performing weighted processing on the first target data and the second target data using a preset point cloud matching algorithm NDT to obtain fifth data; The posture information of the mobile robot is determined based on the third data and the fifth data.

[0011] In some embodiments, the method further comprises: updating a first map stored in the mobile robot based on the posture information to obtain a second map; When the spatial data and the image data indicate that an obstacle exists, obtaining environmental information of the obstacle; Plan the path information of the mobile robot according to the environmental information and the second map.

[0012] An embodiment of the present invention further provides a positioning device for a mobile robot, the device comprising: an acquisition unit, configured to acquire first data collected by the mobile robot; the first data at least including spatial data, image data, and motion data; a first processing unit, configured to perform time alignment processing on the spatial data and the image data to obtain second data; a second processing unit, configured to perform coupling processing on the second data and the motion data to obtain third data; a third processing unit, configured to pre-process the spatial data and the image data respectively to obtain first target data corresponding to the spatial data and second target data corresponding to the image data; A determination unit is used to determine the position information of the mobile robot based on the third data, the first target data and the second target data; the position information is used to locate the mobile robot.

[0013] An embodiment of the present invention provides a positioning device for a mobile robot, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor executes the steps of any one of the above methods when running the computer program.

[0014] An embodiment of the present invention provides a storage medium having a computer program stored thereon; when the computer program is executed by a processor, the steps of any one of the above methods are implemented.

[0015] An embodiment of the present invention provides a method for positioning a mobile robot, the method comprising: obtaining first data collected by the mobile robot; the first data including at least spatial data, image data, and motion data; performing time alignment processing on the spatial data and the image data to obtain second data; performing coupling processing on the second data and the motion data to obtain third data; preprocessing the spatial data and the image data separately to obtain first target data corresponding to the spatial data and second target data corresponding to the image data; determining position information of the mobile robot based on the third data, the first target data, and the second target data; the position information is used to position the mobile robot. Using the technical solution of the present application, the spatial data and the image data collected by the mobile robot are time aligned to obtain second data; the second data are coupled with the motion data collected by the mobile robot to obtain third data; the spatial data and the image data are preprocessed separately to obtain first target data corresponding to the spatial data and second target data corresponding to the image data; and the position information used to position the mobile robot is determined based on the third data, the first target data, and the second target data. That is, by processing the spatial data, image data and motion data collected by the mobile robot and determining the posture information used to position the mobile robot, technical problems such as mobile robot positioning drift and insufficient positioning accuracy are solved, and the positioning accuracy is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic flow chart of a method for positioning a mobile robot provided by an embodiment of the present invention; Figure 2 A schematic structural diagram of a positioning device for a mobile robot provided by an embodiment of the present invention; Figure 3 The figure is a schematic diagram of the hardware structure of the positioning device of the mobile robot according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] The various specific technical features in the various embodiments described in the specific implementation methods can be combined in various ways without contradiction. For example, different implementation methods can be formed by combining different specific technical features. In order to avoid unnecessary repetition, the various possible combinations of the specific technical features in the present invention will not be described separately.

[0019] It should also be noted here that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions of the present invention are shown in the drawings, while other details that are not closely related to the present invention are omitted.

[0020] In addition, it should be noted that the terms "include", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the following description, the terms "first\second\..." involved are merely used to distinguish different objects and do not indicate that there is any similarity or connection between the objects. It should be understood that the directions described by the directional nouns such as "above", "below", "inside" and "outside" are all directions in normal use.

[0021] To make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the specific technical solutions of the invention will be described in further detail below in conjunction with the accompanying drawings in the embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0022] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] The present invention provides a positioning method for a mobile robot, such as Figure 1 As shown, Figure 1 A flowchart of a method for positioning a mobile robot provided by an embodiment of the present invention; the method comprises: Step S101: Acquire first data collected by the mobile robot; the first data at least includes spatial data, image data, and motion data.

[0024] Step S102: Perform time alignment processing on the spatial data and the image data to obtain second data.

[0025] Step S103: perform coupling processing on the second data and the motion data to obtain third data.

[0026] Step S104 : Preprocess the spatial data and the image data respectively to obtain first target data corresponding to the spatial data and second target data corresponding to the image data.

[0027] Step S105 : determining the position information of the mobile robot based on the third data, the first target data, and the second target data; the position information is used to locate the mobile robot.

[0028] In this embodiment, the positioning method of the mobile robot can be determined according to actual conditions and is not limited here. As an example, the positioning method of the mobile robot can be a real-time positioning method of the mobile robot based on laser radar SLAM technology.

[0029] The mobile robot can be determined according to actual conditions and is not limited here. As an example, the mobile robot can include an inspection robot.

[0030] In step S101, the first data includes at least spatial data, image data, and motion data. The spatial data, image data, and motion data can all be determined based on actual circumstances and are not limited herein. As an example, the spatial data may include three-dimensional point cloud data; the image data may include visual images; and the motion data may include acceleration and angular velocity data.

[0031] The specific acquisition process for obtaining the first data collected by the mobile robot can be determined based on actual circumstances and is not limited herein. As an example, the mobile robot includes a laser radar, a binocular camera, and an inertial measurement unit (IMU); obtaining the first data collected by the mobile robot may include: obtaining the spatial data collected by the laser radar, the image data collected by the binocular camera, and the motion data collected by the IMU.

[0032] In step S102, the specific processing process of performing time alignment processing on the spatial data and the image data to obtain the second data can be determined based on actual circumstances and is not limited herein. As an example, performing time alignment processing on the spatial data and the image data to obtain the second data may include: aligning the spatial data and the image data through time synchronization to obtain fourth data containing the spatial data and the image data at the same time base; and performing spatial coordinate alignment processing on the fourth data to obtain the second data.

[0033] In step S103, the specific processing steps for coupling the second data and the motion data to obtain the third data can be determined based on actual circumstances and are not limited herein. As an example, coupling the second data and the motion data to obtain the third data may include: determining a motion estimation model of the mobile robot based on the motion data; determining the motion model of the mobile robot based on the second data; and processing the motion data, the second data, the motion estimation model, and the motion model using a preset coupled LiDAR-Inertial Odometry (LIO-SAM) method to obtain the third data.

[0034] In step S104, the specific preprocessing process of preprocessing the spatial data and the image data to obtain the first target data corresponding to the spatial data and the second target data corresponding to the image data can be determined according to actual conditions and is not limited here. As an example, the spatial data includes first point cloud data; the image data includes second point cloud data; the preprocessing of the spatial data and the image data to obtain the first target data corresponding to the spatial data and the second target data corresponding to the image data can include: filtering and downsampling the first point cloud data to obtain third point cloud data from which outliers and noise points are removed from the first point cloud data; extracting the third point cloud data using a preset feature extraction algorithm to obtain the first target data; dedistorting the second point cloud data to obtain fourth point cloud data from which invalid points are removed from the second point cloud data; and matching the fourth point cloud data using a preset feature matching algorithm to obtain the second target data.

[0035] In step S105, the specific process of determining the position and pose information of the mobile robot based on the third data, the first target data, and the second target data can be determined based on actual conditions and is not limited here. As an example, determining the position and pose information of the mobile robot based on the third data, the first target data, and the second target data may include: performing weighted processing on the first target data and the second target data using a preset point cloud matching algorithm (NDT) to obtain fifth data; and determining the position and pose information of the mobile robot based on the third data and the fifth data.

[0036] An embodiment of the present invention provides a method for positioning a mobile robot. The method comprises: performing time alignment processing on spatial data and image data collected by the mobile robot to obtain second data; coupling processing on the second data and motion data collected by the mobile robot to obtain third data; preprocessing the spatial data and image data separately to obtain first target data corresponding to the spatial data and second target data corresponding to the image data; and determining pose information used for positioning the mobile robot based on the third data, the first target data, and the second target data. Specifically, by processing the spatial data, image data, and motion data collected by the mobile robot and determining the pose information used for positioning the mobile robot, technical problems such as mobile robot positioning drift and insufficient positioning accuracy are resolved, significantly improving positioning accuracy.

[0037] In some embodiments, the mobile robot includes a laser radar, a binocular camera, and an inertial measurement unit (IMU); and obtaining the first data collected by the mobile robot includes: Acquire the spatial data collected by the lidar, the image data collected by the binocular camera, and the motion data collected by the IMU.

[0038] In this embodiment, the spatial data, the image data, and the motion data can all be determined according to actual conditions and are not limited here. As an example, the spatial data may include three-dimensional point cloud data; the image data may include visual images; and the motion data may include acceleration and angular velocity data. Among them, the acceleration data may include an acceleration vector, which can be expressed as ; The angular velocity data may include an angular velocity vector, which can be expressed as .

[0039] The laser radar may also be referred to as LiDAR.

[0040] In actual use, the robot automatically powers on at a predetermined time, launching the operating system and hardware drivers. During initialization, the robot performs self-checks of key modules, such as the LiDAR, binocular camera, and IMU, to ensure the proper functioning of each sensor. After initialization is complete, the robot enters a standby state, ready to collect environmental data and execute tasks. After the robot powers on, the LiDAR and binocular camera begin collecting real-time data about the surrounding environment. The LiDAR collects 3D point cloud data, providing spatial information about the environment; the binocular camera captures visual images, providing surface features. The IMU begins recording acceleration and angular velocity data to assist in estimating the robot's dynamic motion. These sensor data provide the foundation for subsequent map construction and positioning.

[0041] As an example, when the robot boots up, it first loads and initializes the operating system and hardware drivers. The robot then initiates a self-test to ensure the proper functioning of key sensors, including the LiDAR, binocular camera, and IMU. After booting up, the robot performs a hardware self-test to verify sensor status, including the LiDAR's scanning range, the camera's image quality, and the IMU's acceleration and angular velocity data. Once the sensor self-test is complete, the robot enters standby mode, awaiting a start signal for an inspection task. Upon receiving a task start command, the robot activates its data acquisition and path planning modules, preparing to collect environmental data and execute the task. After the robot boots up, its sensors begin collecting real-time data about the surrounding environment. The LiDAR begins generating 360-degree point cloud data through a 360-degree scan. This process uses laser scanning to measure the distance from the sensor to the point and generate the 3D coordinates of each point. The binocular camera captures high-resolution images, capturing visual features of the environment to assist with robot recognition and positioning. The IMU begins recording the robot's acceleration and angular velocity data, which are used for subsequent dynamic motion estimation. The IMU's output is typically an acceleration vector. and the angular velocity vector , which serve as the robot's motion state information and provide support for subsequent algorithms. All of these sensor data will be fed into the robot control system for real-time fusion and processing to ensure data integrity and accuracy.

[0042] In some embodiments, performing time alignment processing on the spatial data and the image data to obtain the second data includes: performing alignment processing on the spatial data and the image data through time synchronization to obtain fourth data of the spatial data and the image data under the same time reference; Perform spatial coordinate alignment processing on the fourth data to obtain the second data.

[0043] In this embodiment, the spatial data may be point cloud data, which may be understood as point cloud data collected by a laser radar (LiDAR), and the image data may be understood as image data collected by a camera.

[0044] The specific processing process of aligning the spatial data and the image data through time synchronization to obtain the fourth data of the spatial data and the image data under the same time reference can be determined based on actual conditions and is not limited here. As an example, aligning the spatial data and the image data through time synchronization to obtain the fourth data of the spatial data and the image data under the same time reference can be aligning the point cloud data and the image data through time synchronization to obtain the fourth data of the image data under the same time reference.

[0045] The specific processing process of performing spatial coordinate alignment processing on the fourth data to obtain the second data can be determined according to actual conditions and is not limited here. As an example, performing spatial coordinate alignment processing on the fourth data to obtain the second data can be performing spatial coordinate alignment processing on the fourth data using a spatial transformation matrix to obtain the second data. The spatial transformation matrix may include a rotation matrix and a translation vector; wherein the rotation matrix can be expressed as ; The translation vector can be recorded as .

[0046] In practical applications, the point cloud data collected by the lidar and camera are preprocessed to remove noise points and invalid points. For the point cloud data, the robot performs filtering, downsampling and feature point extraction (such as corner points, plane points, etc.). The camera data is dedistorted, feature extracted and matched. The preprocessed data provides high-quality information for subsequent weighted NDT matching and map construction. Specifically, as an example, in order to achieve accurate data fusion, the robot integrates the camera's image data and the LiDAR's point cloud data into the same coordinate frame. First, the camera and lidar data are aligned through time synchronization to ensure that the data of the two sensors are used under the same time reference. Then, the camera coordinate system and the lidar coordinate system are aligned using the spatial transformation matrix. If the coordinates of the lidar are and the camera coordinates are , then through the rotation matrix and translation vectors Aligning these two coordinate systems, the transformation formula is: In this way, the robot can put the data provided by the lidar and camera into the same three-dimensional spatial coordinate frame, ensuring the accurate integration of environmental information, thereby improving the robot's positioning and navigation capabilities.

[0047] In some embodiments, the coupling processing of the second data and the motion data to obtain third data includes: determining a motion estimation model of the mobile robot based on the motion data; determining a motion model of the mobile robot according to the second data; The motion data, the second data, the motion estimation model, and the motion model are processed by a preset coupled laser radar-inertial odometry method LIO-SAM to obtain the third data.

[0048] In this embodiment, the motion estimation model of the mobile robot can be expressed as , which can also be understood as the motion estimation model of IMU.

[0049] The second data can be understood as the data of the laser radar and the camera, which can be recorded as and The motion model of the mobile robot can be understood as the motion model corresponding to the data of the laser radar and the camera, which can be recorded as and .

[0050] The third data can be determined according to actual conditions and is not limited here. As an example, the third data can be the status data of the robot, which can be recorded as .

[0051] In practical applications, with the support of the LIO-SAM algorithm, the robot tightly couples the IMU data with the camera and lidar data. The high-frequency motion information (such as acceleration and angular velocity) provided by the IMU helps the robot better estimate its own dynamic motion and correct the positioning errors of the lidar and camera data. The tight coupling method improves the robustness and accuracy of the system in dynamic and complex environments. As an example, specifically, with the support of the LIO-SAM algorithm, the IMU data is tightly coupled with the camera and lidar data. The high-frequency acceleration provided by the IMU and angular velocity The data helps the robot correct positioning errors in lidar and camera data in highly dynamic environments. Through a tight coupling method, IMU data can directly influence the pose estimation process. The LIO-SAM algorithm achieves the fusion of IMU data and sensor data by optimizing the objective function: ;in Indicates the status of the robot. and It is the data from the lidar and camera, and is the corresponding motion model, is the motion estimation model of the IMU, In this way, the fusion of IMU data with lidar and camera data significantly improves positioning accuracy, especially in dynamic environments.

[0052] In some embodiments, the spatial data includes first point cloud data; the image data includes second point cloud data; and preprocessing the spatial data and the image data to obtain first target data corresponding to the spatial data and second target data corresponding to the image data includes: Filtering and downsampling the first point cloud data to obtain third point cloud data in which outliers and noise points are removed from the first point cloud data; Extracting and processing the third point cloud data using a preset feature extraction algorithm to obtain the first target data; performing a dedistortion process on the second point cloud data to obtain fourth point cloud data from which invalid points are removed; The fourth point cloud data is matched using a preset feature matching algorithm to obtain the second target data.

[0053] In this embodiment, the feature extraction algorithm can be determined according to actual conditions and is not limited here. As an example, the feature extraction algorithm can include SIFT or SURF.

[0054] The first target data and the second target data can be determined according to actual conditions and are not limited here. As an example, the first target data and the second target data can be understood as key points, feature points, or key feature points.

[0055] In practical applications, point cloud data collected by lidar and cameras undergoes preprocessing to remove noise and invalid points. The robot then performs filtering, downsampling, and feature point extraction (such as corner points and planar points) on the point cloud data. Camera data undergoes dedistortion, feature extraction, and matching. This preprocessed data provides high-quality information for subsequent weighted NDT matching and map construction. Specifically, as an example, point cloud data collected by lidar and cameras undergoes preprocessing to improve the accuracy and efficiency of subsequent processing. Lidar data is first filtered and downsampled to remove outliers and noise. Feature extraction algorithms, such as SIFT or SURF, are then used to extract key points or feature points. Camera data is first dedistorted to eliminate the effects of lens distortion. Feature matching methods are then used to extract key feature points from the image for matching with the point cloud data. This preprocessed data is then fed into the weighted NDT algorithm, providing accurate input for subsequent point cloud matching and map construction.

[0056] In some embodiments, determining the position and posture information of the mobile robot based on the third data, the first target data, and the second target data includes: Performing weighted processing on the first target data and the second target data using a preset point cloud matching algorithm NDT to obtain fifth data; The posture information of the mobile robot is determined based on the third data and the fifth data.

[0057] In this embodiment, the robot uses a weighted NDT algorithm to match preprocessed point cloud data. Unlike traditional NDT, weighted NDT assigns different weights based on point distance and surface characteristics. Distant points and points with stronger surface features have a greater impact on the matching results, thereby improving point cloud matching accuracy. During each scan, the algorithm calculates the optimal transformation between the current frame and previous frames.

[0058] The first target data and the second target data are weighted by the preset point cloud matching algorithm NDT to obtain the fifth data. ;in, It is the first Points, is the position of the point after transformation, is the weight of the point. It is set based on factors such as point distance, surface features, etc. Weighted NDT makes important points (such as close points and points with obvious surface features) have a greater impact on the matching results, thereby improving matching accuracy.

[0059] The specific determination process of determining the position and posture information of the mobile robot based on the third data and the fifth data can be determined according to actual conditions and is not limited here. As an example, the determination of the position and posture information of the mobile robot based on the third data and the fifth data can be to determine the position and posture information of the mobile robot based on the third data and the fifth data through the maximum likelihood estimation (MLE) optimization method. In actual applications, based on the weighted NDT calculation, the robot calculates the optimal alignment of the current scan data and the existing map through an optimization algorithm. This process uses the maximum likelihood estimation (MLE) optimization method to minimize the matching error and obtain the precise position and posture of the robot. This step ensures that the robot's position information has high accuracy and reduces positioning drift. Specifically, the purpose of the optimal matching is to calculate the optimal alignment between the current point cloud data and the historical map through an optimization algorithm. Use the maximum likelihood estimation (MLE) method to minimize the point cloud matching error. Assume that the current point cloud is , the historical map is , the optimization objective function is: ;in is the transformation matrix, and the goal is to find the optimal transformation matrix , so that the error between the current point cloud and the map is minimized. Through iterative optimization, the optimal robot posture is finally obtained, positioning drift is reduced, and the robot's precise positioning is guaranteed.

[0060] In some embodiments, the method further comprises: updating a first map stored in the mobile robot based on the posture information to obtain a second map; When the spatial data and the image data indicate that an obstacle exists, obtaining environmental information of the obstacle; Plan the path information of the mobile robot according to the environmental information and the second map.

[0061] In this embodiment, the first map can be determined according to actual conditions and is not limited here. As an example, the first map can be understood as an existing map, an existing map, or a historical map.

[0062] The obstacles can be determined according to actual conditions and are not limited here. As an example, the obstacles can include walls, equipment or other static obstacles.

[0063] The environmental information can be determined according to actual conditions and is not limited here. As an example, the environmental information can be understood as new environmental information.

[0064] The specific planning process for planning the mobile robot's path information based on the environmental information and the second map can be determined based on actual circumstances and is not limited here. As an example, planning the mobile robot's path information based on the environmental information and the second map can be understood as calculating the shortest path based on the real-time map and environmental data.

[0065] In actual application, after completing each point cloud match, the robot merges the new point cloud data into the existing map. During the real-time map building process, the robot gradually expands the map of known areas while optimizing existing map sections to reduce error accumulation. This continuous updating process enables the robot to accurately locate and navigate in uncertain factory environments. The robot uses data from lidar and cameras to analyze the environment and identify obstacles along its path. When an obstacle is detected, the robot uses an algorithm to determine the obstacle type (e.g., static or dynamic) and adjusts its path planning to avoid collisions. The robot also analyzes the operating status of the equipment to determine whether a key inspection is required. Based on the real-time map and environmental analysis data, the robot uses the A* algorithm for path planning. The robot chooses the shortest path while avoiding obstacles and areas that require avoidance. In dynamic environments, the robot updates its path based on real-time perception data to ensure efficient and safe inspections. Based on the results of path planning, the robot performs its inspection along the predetermined route. During this process, the lidar and IMU continue to provide data to ensure the robot accurately follows the planned path. The robot will continuously perceive the environment and adjust its position and posture in real time to adapt to possible obstacles and environmental changes.

[0066] As an example, specifically, after each point cloud match, the robot will merge the new point cloud data into the existing map, thereby performing real-time map construction and updates. Through the weighted NDT algorithm, the robot will update the obstacles, environmental information, and other key features in the map at each match. The core goal of map construction is to maintain a dynamic and accurate environmental model to facilitate robot navigation and positioning. When the map is updated, the robot's position and angle will be corrected through the optimization matching algorithm to ensure that the map will not be distorted due to the accumulation of small errors during long-term use. The real-time update formula is: ;in, It's an updated map. It's the original map. It is newly added point cloud data or map area. Each time it is updated, the robot uses a weighted NDT algorithm to match the new point cloud data with the current map to ensure the continuity and accuracy of the map. The robot performs environmental analysis and obstacle detection through lidar and camera data. Through the data scanned by the lidar, the robot can identify and classify obstacles on the path, such as walls, equipment or other static obstacles. In addition, the robot also identifies dynamic obstacles such as people or vehicles through camera data. When an obstacle is detected, the robot adjusts the path planning according to its type (static or dynamic) to avoid collision. Obstacle detection is based on cluster analysis of point cloud data. Assuming that the shape of the obstacle is spherical or cubic, the obstacle detection formula is: ;in, The first Points, is the current position of the robot, is the preset obstacle detection radius. When an obstacle is detected, the robot will update the path planning based on the new environmental information. Path planning is one of the core tasks of the robot, and its purpose is to calculate the shortest path based on real-time map and environmental data. Robots usually use the A* algorithm for path planning, which can calculate the optimal path based on the distance between nodes. The A* algorithm is combined with the heuristic function and path cost To calculate the total cost of each node The specific calculation steps are as follows: 1. Initialize the start and end points.

[0067] 2. Calculate the cost of each node and put it into the priority queue.

[0068] 3. Select the next node to expand in the order of minimum cost until the end is found.

[0069] 4. Return the shortest path from the starting point to the end point.

[0070] In a dynamic environment, the robot continuously updates its path planning, detects obstacles in real time, and replans its path.

[0071] Based on the results of path planning, the robot performs inspection tasks along the planned path. During the execution process, the LiDAR and IMU sensors continuously provide data to ensure that the robot accurately follows the planned path. The robot adjusts its position and posture in real time by comparing real-time point cloud data with the map. The robot uses the SLAM algorithm to align and optimize each frame of point cloud data to ensure the accuracy of position estimation. For example, the robot may perform a periodic optimization step to update the current robot posture. : in Represents the position change of the robot, which is the result of weighted NDT and LIO-SAM algorithm optimization calculation.

[0072] In practical applications, as an example, the positioning method of the mobile robot may specifically be a real-time positioning method of the mobile robot based on the laser radar SLAM technology, which may be implemented by the following steps.

[0073] 1. Start the robot.

[0074] The robot automatically powers on at the scheduled time, launching the operating system and hardware drivers. During initialization, the robot performs self-checks of key modules, such as the LiDAR, binocular camera, and IMU, to ensure the proper functioning of all sensors. After initialization is complete, the robot enters a standby state, ready to collect environmental data and execute tasks.

[0075] 2. Collection of real-time data of factory environment.

[0076] After the robot starts, the lidar and binocular cameras begin collecting real-time data about the surrounding environment. The lidar collects 3D point cloud data, providing spatial information about the environment; the binocular cameras capture visual images, providing surface features. The inertial measurement unit (IMU) begins recording acceleration and angular velocity data to assist in estimating the robot's dynamic motion. These sensor data provide the foundation for subsequent map construction and positioning.

[0077] 3. Put camera data and lidar data into one framework.

[0078] To ensure accurate data fusion, the robot places the image data collected by the camera and the point cloud data collected by the lidar in the same reference frame. Through time synchronization and spatial coordinate alignment, the two data are integrated into a unified framework. This allows the robot to simultaneously utilize the spatial information from the lidar and the visual information from the camera, improving the accuracy of environmental perception.

[0079] 4. Tightly couple camera data and lidar data with the IMU.

[0080] Powered by the LIO-SAM algorithm, the robot tightly couples IMU data with camera and lidar data. The high-frequency motion information (such as acceleration and angular velocity) provided by the IMU helps the robot better estimate its dynamic motion and correct positioning errors in the lidar and camera data. This tightly coupled approach improves the system's robustness and accuracy in dynamic and complex environments.

[0081] 5. Preprocessing of point cloud data and feature points collected by radar and camera.

[0082] The point cloud data collected by the lidar and camera undergoes preprocessing to remove noise and invalid points. The robot then performs filtering, downsampling, and feature point extraction (such as corners and planar points) on the point cloud data. The camera data undergoes dedistortion, feature extraction, and matching. This preprocessed data provides high-quality information for subsequent weighted NDT matching and map construction.

[0083] 6. Weighted NDT calculation.

[0084] The robot uses a weighted NDT algorithm to match preprocessed point cloud data. Unlike traditional NDT, weighted NDT assigns different weights to points based on their distance and surface characteristics. Points farther away and with stronger surface features have a greater impact on the matching results, thereby improving point cloud matching accuracy. During each scan, the algorithm calculates the optimal transformation between the current frame and previous frames.

[0085] 7. Optimize matching.

[0086] Based on the weighted NDT calculations, the robot uses an optimization algorithm to calculate the optimal registration between the current scan data and the existing map. This process uses the Maximum Likelihood Estimation (MLE) optimization method to minimize the matching error and obtain the robot's precise position. This step ensures high accuracy of the robot's position information and reduces positioning drift.

[0087] 8. Real-time map construction and update.

[0088] After completing each point cloud match, the robot merges the new point cloud data into its existing map. During real-time map building, the robot gradually expands the map of known areas while simultaneously optimizing previously mapped areas to reduce the accumulation of errors. This continuous updating process enables the robot to accurately locate and navigate in uncertain factory environments.

[0089] 9. Environmental analysis and obstacle detection.

[0090] The robot uses data from lidar and cameras to analyze the environment and identify obstacles along its path. Upon detecting an obstacle, the robot uses an algorithm to determine its type (static or dynamic) and adjusts its path planning to avoid collisions. The robot also analyzes the operating status of the equipment and determines whether additional inspections are necessary.

[0091] 10. Path planning.

[0092] Based on real-time maps and environmental analysis data, the robot uses the A* algorithm for path planning. The robot chooses the shortest path while avoiding obstacles and areas that require avoidance. In dynamic environments, the robot updates its path based on real-time perception data, ensuring that inspection tasks are completed efficiently and safely.

[0093] 11. Carry out inspection tasks along the planned route.

[0094] Based on the results of path planning, the robot performs inspections along a predetermined route. During this process, the LiDAR and IMU provide continuous data to ensure the robot accurately follows the planned path. The robot continuously senses its environment and adjusts its position and posture in real time to adapt to potential obstacles and environmental changes.

[0095] 12. Equipment inspection and data collection.

[0096] Upon reaching the inspection target, the robot conducts a detailed inspection of the equipment. LiDAR scans the equipment surface, while a binocular camera captures images and collects relevant data. The robot also uses temperature and pressure sensors to monitor the equipment's status and ensure its health. If any anomalies are detected, the robot promptly records and reports them to the central control system.

[0097] 13. Data upload and synchronization.

[0098] The data collected by the robot during its inspection (point cloud data, image data, device status, etc.) is uploaded to the central control platform via a wireless network. The platform stores and analyzes the data for subsequent decision support and troubleshooting. All data upload and synchronization processes must ensure the integrity and accuracy of the information to promptly identify potential problems.

[0099] 14. Positioning and map optimization.

[0100] During mission execution, the robot regularly optimizes its positioning and map. Using weighted NDT and LIO-SAM algorithms, the robot optimizes its previous map and current position information to reduce error accumulation. This optimized map provides more accurate environmental information, improving the robot's inspection accuracy and efficiency.

[0101] 15. Status feedback and adjustment.

[0102] During the inspection process, the robot monitors its status in real time, including battery life, sensor status, and task progress. If the robot encounters any problems (e.g., low battery, sensor failure, etc.), it immediately reports these to the central control system, which then makes appropriate adjustments (e.g., returning to a charging station or switching to a backup sensor).

[0103] 16. Report generation and upload.

[0104] After completing an inspection, the robot generates a detailed inspection report. This report includes the equipment's operating status, the inspected area, any anomalies detected, and the robot's inspection path and location data. The report is then uploaded to the company's management platform via the wireless network for manual maintenance personnel to review and process. The report is also archived for subsequent analysis and optimization.

[0105] 17. Return to the charging station.

[0106] After completing a mission, the robot automatically returns to the charging station based on its battery level. During the return journey, the robot recalculates its path to avoid any obstacles. If the battery is low, the robot prioritizes returning to the charging station and preparing for recharging. The charging station automatically adjusts the charging process based on the robot's status to ensure efficient and safe charging.

[0107] 18. System maintenance and update.

[0108] After the robot completes a certain period of work, the system undergoes regular maintenance and updates. This includes sensor calibration, algorithm optimization, and hardware inspection. By analyzing historical mission data, the R&D team can adjust the algorithm to improve the robot's accuracy and efficiency. Regular maintenance and updates ensure that the robot is always in optimal working condition.

[0109] As an example, the specific implementation includes the following: 1. Start the robot.

[0110] When the robot boots up, it first loads and initializes the operating system and hardware drivers. The robot then initiates a self-test to ensure the proper functioning of key sensors, including the LiDAR, binocular camera, and IMU. After startup, the robot performs a hardware self-test to verify sensor status, including the LiDAR's scanning range, the camera's image quality, and the IMU's acceleration and angular velocity data. Once the sensor self-test is complete, the robot enters standby mode, awaiting a start signal for an inspection task. Upon receiving the task start command, the robot activates its data acquisition and path planning modules, preparing to collect environmental data and execute the task.

[0111] 2. Collection of real-time data of factory environment.

[0112] After the robot starts, the sensors begin to collect data about the surrounding environment in real time. The LiDAR begins to generate three-dimensional point cloud data through 360-degree scanning. This process measures the distance from the point to the sensor through laser scanning and generates the three-dimensional coordinates of each point. The binocular camera obtains high-resolution images to capture the visual features of the environment and assist in the recognition and positioning of the robot. The IMU begins to record the acceleration and angular velocity data of the robot, which are used for subsequent dynamic motion estimation. The output of the IMU is usually the acceleration vector and the angular velocity vector , which serve as the robot's motion state information and provide support for subsequent algorithms. All of these sensor data will be fed into the robot control system for real-time fusion and processing to ensure data integrity and accuracy.

[0113] 3. Put camera data and lidar data into one framework.

[0114] To achieve accurate data fusion, the robot integrates the camera's image data and the LiDAR's point cloud data into the same coordinate frame. First, the camera and LiDAR data are aligned through time synchronization to ensure that the data from the two sensors are used under the same time reference. Then, the spatial transformation matrix is ​​used to align the camera coordinate system with the LiDAR coordinate system. If the LiDAR coordinates are and the camera coordinates are , then through the rotation matrix and translation vectors Aligning these two coordinate systems, the transformation formula is: In this way, the robot can put the data provided by the lidar and camera into the same three-dimensional spatial coordinate frame, ensuring the accurate integration of environmental information, thereby improving the robot's positioning and navigation capabilities.

[0115] 4. Tightly couple camera data and lidar data with the IMU.

[0116] With the support of LIO-SAM algorithm, IMU data is tightly coupled with camera and lidar data. IMU provides high-frequency acceleration and angular velocity The data helps the robot correct positioning errors in lidar and camera data in highly dynamic environments. Through a tight coupling method, IMU data can directly influence the pose estimation process. The LIO-SAM algorithm achieves the fusion of IMU data and sensor data by optimizing the objective function: ;in Indicates the status of the robot. and It is the data from the lidar and camera, and is the corresponding motion model, is the motion estimation model of the IMU, In this way, the fusion of IMU data with lidar and camera data significantly improves positioning accuracy, especially in dynamic environments.

[0117] 5. Preprocessing of point cloud data and feature points collected by radar and camera.

[0118] Point cloud data collected by lidar and cameras undergoes preprocessing to improve the accuracy and efficiency of subsequent processing. Lidar data is first filtered and downsampled to remove outliers and noise points. Feature extraction algorithms, such as SIFT or SURF, are then used to extract key points or feature points. Camera data is first dedistorted to eliminate the effects of lens distortion. Feature matching methods are then used to extract key feature points from the image for matching with the point cloud data. This preprocessed data is then fed into a weighted NDT algorithm, providing accurate input for subsequent point cloud matching and map construction.

[0119] 6. Weighted NDT calculation.

[0120] Weighted NDT (Normal Distributions Transform) is used for point cloud data matching. The traditional NDT method treats all point cloud units equally, while the weighted NDT method assigns different weights to each point according to the distance and surface features of the point cloud. Adjust the influence of the point cloud. The calculation process is: ;in, It is the first Points, is the position of the point after transformation, is the weight of the point. It is set based on factors such as point distance, surface features, etc. Weighted NDT makes important points (such as close points and points with obvious surface features) have a greater impact on the matching results, thereby improving matching accuracy.

[0121] 7. Optimize matching.

[0122] The purpose of optimal matching is to calculate the optimal registration between the current point cloud data and the historical map through the optimization algorithm. The maximum likelihood estimation (MLE) method is used to minimize the point cloud matching error. Assume that the current point cloud is , the historical map is , the optimization objective function is: ;in is the transformation matrix, and the goal is to find the optimal transformation matrix , so that the error between the current point cloud and the map is minimized. Through iterative optimization, the optimal robot posture is finally obtained, positioning drift is reduced, and the robot's precise positioning is guaranteed.

[0123] 8. Real-time map construction and update.

[0124] After each point cloud match, the robot will merge the new point cloud data into the existing map, thereby performing real-time map construction and updates. Through the weighted NDT algorithm, the robot will update the obstacles, environmental information, and other key features in the map at each match. The core goal of map construction is to maintain a dynamic and accurate environmental model to facilitate robot navigation and positioning. When the map is updated, the robot's position and angle will be corrected through the optimization matching algorithm to ensure that the map will not be distorted due to the accumulation of small errors during long-term use. The real-time update formula is: ; in, It's an updated map. It's the original map. It is the newly added point cloud data or map area. During each update, the robot uses the weighted NDT algorithm to match the new point cloud data with the current map to ensure the continuity and accuracy of the map.

[0125] 9. Environmental analysis and obstacle detection.

[0126] The robot uses lidar and camera data to analyze the environment and detect obstacles. Using lidar scan data, the robot can identify and classify obstacles on its path, such as walls, equipment, or other static obstacles. Furthermore, the robot uses camera data to identify dynamic obstacles, such as people or vehicles. When an obstacle is detected, the robot adjusts its path planning based on its type (static or dynamic) to avoid collision. Obstacle detection is based on cluster analysis of point cloud data. Assuming the obstacle is spherical or cubic, the obstacle detection formula is: ;in, The first Points, is the current position of the robot, is the preset obstacle detection radius. When an obstacle is detected, the robot will update its path planning based on the new environment information.

[0127] 10. Path planning.

[0128] Path planning is one of the core tasks of robots. Its purpose is to calculate the shortest path based on real-time map and environment data. Robots usually use A* algorithm for path planning, which can calculate the optimal path based on the distance between nodes. A* algorithm is combined with heuristic function and path cost To calculate the total cost of each node The specific calculation steps are as follows: 1. Initialize the start and end points.

[0129] 2. Calculate the cost of each node and put it into the priority queue.

[0130] 3. Select the next node to expand in the order of minimum cost until the end is found.

[0131] 4. Return the shortest path from the starting point to the end point.

[0132] In a dynamic environment, the robot continuously updates its path planning, detects obstacles in real time, and replans its path.

[0133] 11. Carry out inspection tasks along the planned route.

[0134] Based on the results of path planning, the robot performs inspection tasks along the planned path. During the execution process, the LiDAR and IMU sensors continuously provide data to ensure that the robot accurately follows the planned path. The robot adjusts its position and posture in real time by comparing real-time point cloud data with the map. The robot uses the SLAM algorithm to align and optimize each frame of point cloud data to ensure the accuracy of position estimation. For example, the robot may perform a periodic optimization step to update the current robot posture. : ;in Represents the position change of the robot, which is the result of weighted NDT and LIO-SAM algorithm optimization calculation.

[0135] 12. Equipment inspection and data collection.

[0136] When the robot reaches its inspection target, it begins its equipment inspection. It uses lidar to scan the equipment surface and extract its features. Simultaneously, a binocular camera captures images of the equipment and analyzes its status. The robot also uses sensors (such as temperature and pressure sensors) for real-time monitoring, detecting any anomalies (such as overheating or damage). The robot compares this equipment status information with pre-set normal operating conditions and generates a report. If an anomaly is detected, the robot records the information and sends it back to the central control system for further processing by maintenance personnel.

[0137] 13. Data upload and synchronization.

[0138] The data collected by the robot during the inspection process, including point cloud data, image data, sensor information, etc., is uploaded to the central server via the wireless network. Data upload uses a distributed synchronization mechanism to ensure data integrity and real-time performance. During the upload process, the robot will perform data compression and encryption to ensure data security and accuracy. The uploaded data will be stored on the cloud platform for subsequent analysis and fault diagnosis. The specific synchronization formula is: ;in It is the synchronized data. and Represents the robot's local data and the data on the server respectively.

[0139] 14. Positioning and map optimization.

[0140] During inspections, the robot regularly optimizes its positioning and maps to reduce error accumulation. Using weighted NDT and the LIO-SAM algorithm, the robot performs back-end optimization on previously generated maps to correct for positioning drift during long-term operation. The optimization process minimizes the error between the map and sensor data, and the optimization objective function is: ;in It is point cloud data, is the IMU data, is the pose estimation function, is the balancing weighting factor. Through this optimization, the robot is able to maintain high-precision positioning and map updates.

[0141] 15. Status feedback and adjustment.

[0142] The robot monitors its operating status in real time, including battery charge, sensor status, and task progress. When the robot detects an anomaly (such as low battery or sensor failure), it immediately reports this to the central control system. The system then makes adjustments based on this feedback, such as returning to a charging station or replacing a spare sensor. This feedback process uses state estimation methods: .

[0143] 16. Report generation and upload.

[0144] Upon completing an inspection, the robot automatically generates a detailed inspection report, including device status, task completion status, and anomaly detection results. The report is uploaded to the management platform via the wireless network for review and analysis by maintenance personnel. When generating a report, the robot aggregates all collected data and generates a report file based on a pre-set template.

[0145] 17. Return to the charging station.

[0146] When the mission is completed, the robot automatically returns to the charging station based on the battery level. If the battery is low, the robot will prioritize returning to the charging station and calculate the shortest path to recharge. The charging station automatically adjusts the charging process based on the robot's status to ensure charging efficiency and safety.

[0147] 18. System maintenance and update.

[0148] The system undergoes regular maintenance and updates, including sensor calibration, algorithm optimization, and hardware inspections. These maintenance and updates help improve the robot's accuracy and efficiency, maintaining long-term stable system operation. Algorithm optimization adjusts weighted NDT and LIO-SAM parameters based on historical mission data to improve positioning accuracy.

[0149] This application proposes a new solution based on the fusion of weighted NDT (Normal Distributions Transform) and LIO-SAM (Lidar Inertial Odometry and Mapping) algorithms. By combining weighted NDT with LIO-SAM, this solution improves the self-localization and mapping accuracy of inspection robots in factory environments. As an advanced point cloud matching method, weighted NDT improves the accuracy of point cloud matching compared to traditional NDT by weighting each unit in the point cloud and adjusting it according to different distances and surface features. In a dynamically changing environment like a factory, the use of weighted NDT can effectively reduce point cloud matching errors, especially during long-term operation, greatly reducing the risk of positioning drift and improving the robot's navigation stability.

[0150] Furthermore, the LIO-SAM algorithm enhances the positioning accuracy and robustness of the robot system by tightly integrating IMU (Inertial Measurement Unit) data with LiDAR data. The IMU provides high-frequency motion information about the robot, helping to correct errors in the LiDAR point cloud caused by environmental changes or dynamic obstacles. The LIO-SAM combination enables the system to maintain low drift in dynamic environments and perform real-time self-localization and mapping, making it particularly suitable for use in rapidly changing factory environments. This fusion approach enables the robot to not only efficiently navigate autonomously but also adapt to various challenges in complex environments, such as obstacle avoidance, rapid motion, and real-time positioning corrections.

[0151] This application proposes a solution based on a fusion algorithm of weighted NDT and LIO-SAM, addressing the shortcomings of traditional inspection robots in terms of accuracy, real-time performance, and robustness in factory environments. Weighted NDT refines point cloud processing, improving matching accuracy and reducing positioning drift. LIO-SAM further enhances system stability and robustness through the tight coupling of IMU and LiDAR. This new approach not only improves the efficiency of factory inspection robots and reduces labor costs, but also ensures high accuracy and safety during the inspection process, promising broad application prospects.

[0152] Based on the same inventive concept as above, Figure 2 A schematic structural diagram of a positioning device for a mobile robot provided by an embodiment of the present invention, such as Figure 2As shown, the device 200 includes: An acquisition unit 201 is configured to acquire first data collected by the mobile robot; the first data includes at least spatial data, image data, and motion data; A first processing unit 202 is configured to perform time alignment processing on the spatial data and the image data to obtain second data; A second processing unit 203 is configured to perform coupling processing on the second data and the motion data to obtain third data; A third processing unit 204 is configured to pre-process the spatial data and the image data to obtain first target data corresponding to the spatial data and second target data corresponding to the image data; The determination unit 205 is configured to determine the position and posture information of the mobile robot based on the third data, the first target data, and the second target data; the position and posture information is used to locate the mobile robot.

[0153] In some embodiments, the mobile robot includes a laser radar, a binocular camera and an inertial measurement unit IMU; the acquisition unit 201 is also used to acquire the spatial data collected by the laser radar, the image data collected by the binocular camera and the motion data collected by the IMU.

[0154] In some embodiments, the first processing unit 202 is further used to align the spatial data and the image data through time synchronization to obtain fourth data of the spatial data and the image data under the same time reference; and to perform spatial coordinate alignment on the fourth data to obtain the second data.

[0155] In some embodiments, the second processing unit 203 is further used to determine the motion estimation model of the mobile robot based on the motion data; determine the motion model of the mobile robot according to the second data; and process the motion data, the second data, the motion estimation model, and the motion model through a preset coupled lidar-inertial odometry method LIO-SAM to obtain the third data.

[0156] In some embodiments, the spatial data includes first point cloud data; the image data includes second point cloud data; the third processing unit 204 is further used to filter and downsample the first point cloud data to obtain third point cloud data from the first point cloud data by removing outliers and noise points; extract and process the third point cloud data through a preset feature extraction algorithm to obtain the first target data; dedistort the second point cloud data to obtain fourth point cloud data from the second point cloud data by removing invalid points; and match the fourth point cloud data through a preset feature matching algorithm to obtain the second target data.

[0157] In some embodiments, the determination unit 205 is further used to perform weighted processing on the first target data and the second target data through a preset point cloud matching algorithm NDT to obtain fifth data; and determine the posture information of the mobile robot based on the third data and the fifth data.

[0158] In some embodiments, the apparatus 200 further includes an updating unit and a planning unit; wherein, The updating unit is configured to update the first map stored in the mobile robot based on the posture information to obtain a second map; The acquisition unit 201 is further configured to acquire environmental information of an obstacle when the spatial data and the image data indicate that an obstacle exists; The planning unit is used to plan the path information of the mobile robot according to the environmental information and the second map.

[0159] It should be noted that the positioning device for a mobile robot provided in an embodiment of the present invention and the configuration method provided in the aforementioned embodiment of the present invention belong to the same inventive concept. The meanings of the terms appearing here have been explained in detail above and will not be repeated here.

[0160] An embodiment of the present invention further provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned method embodiment are implemented. The aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program code.

[0161] An embodiment of the present invention also provides a positioning device for a mobile robot, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein when the processor is used to run the computer program, it executes the steps of the above-mentioned method embodiment stored in the memory.

[0162] Figure 3 This is a hardware structure diagram of the positioning device of the mobile robot according to an embodiment of the present invention. The positioning device 300 of the mobile robot includes: at least one processor 301, a memory 302. Optionally, the positioning device 300 of the mobile robot may further include at least one communication interface 303. The various components in the positioning device 300 of the mobile robot are coupled together through a bus system 304. It can be understood that the bus system 304 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 304 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 3 Various buses are labeled as bus system 304 .

[0163] It is understood that memory 302 can be volatile memory or non-volatile memory, or can include both volatile and non-volatile memory. Non-volatile memory can include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disk, or compact disc read-only memory (CD-ROM); magnetic surface memory can include magnetic disk storage or magnetic tape storage. Volatile memory can include random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memory 302 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0164] The memory 302 in the embodiment of the present invention is used to store various types of data to support the operation of the positioning device 300 of the mobile robot. Examples of such data include any computer program for operating on the positioning device 300 of the mobile robot. The program for implementing the method of the embodiment of the present invention may be included in the memory 302.

[0165] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 301. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor or by software instructions. The above processor may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. The processor can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium located in a memory. The processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0166] In an exemplary embodiment, the positioning device 300 of the mobile robot can be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the above method.

[0167] In the several embodiments provided herein, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is merely a logical functional division. In actual implementation, other divisions may be employed, such as combining multiple units or components, integrating them into another system, or omitting or disabling certain features. Furthermore, the coupling, direct coupling, or communication connection between the components shown or discussed may be through interfaces. The indirect coupling or communication connection between devices or units may be electrical, mechanical, or other. The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of these units may be selected to achieve the objectives of the present embodiments based on actual needs. Furthermore, the functional units in the various embodiments of the present invention may all be integrated into a single processing module, each unit may be a separate unit, or two or more units may be integrated into a single unit. These integrated units may be implemented in hardware or as hardware plus software functional units.

[0168] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. A positioning method for a mobile robot, characterized in that: The method comprises: Acquiring first data collected by the mobile robot; the first data includes at least spatial data, image data, and motion data; performing time alignment processing on the spatial data and the image data to obtain second data; performing coupling processing on the second data and the motion data to obtain third data; Preprocessing the spatial data and the image data respectively to obtain first target data corresponding to the spatial data and second target data corresponding to the image data; The position information of the mobile robot is determined based on the third data, the first target data and the second target data; the position information is used to locate the mobile robot.

2. The method according to claim 1, characterized in that The mobile robot includes a laser radar, a binocular camera, and an inertial measurement unit (IMU); and obtaining the first data collected by the mobile robot includes: Acquire the spatial data collected by the lidar, the image data collected by the binocular camera, and the motion data collected by the IMU.

3. The method according to claim 1, characterized in that The performing time alignment processing on the spatial data and the image data to obtain second data includes: performing alignment processing on the spatial data and the image data through time synchronization to obtain fourth data of the spatial data and the image data under the same time reference; Perform spatial coordinate alignment processing on the fourth data to obtain the second data.

4. The method according to claim 1, wherein The coupling processing of the second data and the motion data to obtain third data includes: determining a motion estimation model of the mobile robot based on the motion data; determining a motion model of the mobile robot according to the second data; The motion data, the second data, the motion estimation model, and the motion model are processed by a preset coupled laser radar-inertial odometry method LIO-SAM to obtain the third data.

5. The method according to claim 1, wherein The spatial data includes first point cloud data; the image data includes second point cloud data; and the preprocessing of the spatial data and the image data to obtain first target data corresponding to the spatial data and second target data corresponding to the image data includes: Filtering and downsampling the first point cloud data to obtain third point cloud data in which outliers and noise points are removed from the first point cloud data; Extracting and processing the third point cloud data using a preset feature extraction algorithm to obtain the first target data; performing a dedistortion process on the second point cloud data to obtain fourth point cloud data from which invalid points are removed; The fourth point cloud data is matched using a preset feature matching algorithm to obtain the second target data.

6. The method according to claim 1, wherein The determining the position and posture information of the mobile robot based on the third data, the first target data, and the second target data includes: Performing weighted processing on the first target data and the second target data using a preset point cloud matching algorithm NDT to obtain fifth data; The posture information of the mobile robot is determined based on the third data and the fifth data.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: updating a first map stored in the mobile robot based on the posture information to obtain a second map; When the spatial data and the image data indicate that an obstacle exists, obtaining environmental information of the obstacle; Plan the path information of the mobile robot according to the environmental information and the second map.

8. A positioning device for a mobile robot, characterized in that: The device comprises: an acquisition unit, configured to acquire first data collected by the mobile robot; the first data at least including spatial data, image data, and motion data; a first processing unit, configured to perform time alignment processing on the spatial data and the image data to obtain second data; a second processing unit, configured to perform coupling processing on the second data and the motion data to obtain third data; a third processing unit, configured to pre-process the spatial data and the image data respectively to obtain first target data corresponding to the spatial data and second target data corresponding to the image data; A determination unit is used to determine the position information of the mobile robot based on the third data, the first target data and the second target data; the position information is used to locate the mobile robot.

9. A positioning device for a mobile robot, characterized in that: The device comprises: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor executes the steps of the method according to any one of claims 1 to 7 when running the computer program.

10. A storage medium, characterized in that: The storage medium stores a computer program; when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images