Robot positioning method, device and non-volatile storage medium

Abstract

The present application discloses a robot positioning method, device and non-volatile storage medium. The method includes: obtaining first positioning data corresponding to an absolute positioning sensor provided on the robot, and second positioning data corresponding to a relative positioning sensor, wherein the absolute positioning sensor is used to provide positioning data in a world coordinate system, and the relative positioning sensor is used to determine positioning data based on historical displacement; using the first positioning data, correcting the second positioning data to obtain the corrected second positioning data; determining a sensor data fusion strategy based on the environmental information of the robot; using the sensor data fusion strategy, fusing the first positioning data and the corrected second positioning data to obtain the positioning information of the robot. The present application solves the technical problem of unsatisfactory robot positioning accuracy existing in the related art.

Description

Technical Field

[0001] The present application relates to the field of robot navigation and autonomous driving technology, and more specifically, to a robot positioning method, device and non-volatile storage medium. Background Art

[0002] In the field of robot navigation and autonomous driving, accurate positioning information, obstacle avoidance and path planning are key technologies for achieving autonomous navigation. Navigation methods based on a single sensor, such as using odometers, GPS or lidar, cannot provide sufficiently accurate location information, especially in complex environments. GPS signal instability, excessive sensor noise, and sensor failure can lead to inaccurate positioning or even navigation failure. Existing obstacle avoidance technologies mainly rely on lidar and ultrasonic sensors to detect obstacles, but each has obvious limitations. Lidar has blind spots in three-dimensional obstacle detection and cannot effectively sense suspended objects or steps. Ultrasonic sensors have low accuracy due to their detection principles, especially in complex scenes, and are prone to false alarms or missed alarms, making it difficult to meet the robot's needs for long-distance obstacle detection. The technical problem of unsatisfactory robot positioning accuracy exists in related technologies.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a robot positioning method, device and non-volatile storage medium to at least solve the technical problem of unsatisfactory robot positioning accuracy existing in the related art.

[0005] According to one aspect of an embodiment of the present application, a robot positioning method is provided, including: obtaining first positioning data corresponding to an absolute positioning sensor provided on the robot, and second positioning data corresponding to a relative positioning sensor, wherein the absolute positioning sensor is used to provide positioning data in a world coordinate system, and the relative positioning sensor is used to determine positioning data based on historical displacement; using the first positioning data, correcting the second positioning data to obtain corrected second positioning data; determining a sensor data fusion strategy based on environmental information of the robot; using the sensor data fusion strategy, fusing the first positioning data and the corrected second positioning data to obtain the positioning information of the robot.

[0006] Optionally, the absolute positioning sensor includes at least a global positioning system GPS sensor and a laser radar, and the use of the first positioning data to correct the second positioning data to obtain the corrected second positioning data includes: performing data fusion based on the first positioning data corresponding to the GPS sensor and the laser radar respectively to obtain global position information; according to a predetermined compensation period, using the global position information to correct the second positioning data to obtain the corrected second positioning data.

[0007] Optionally, the absolute positioning sensor includes at least a laser radar, the relative positioning sensor includes at least an inertial measurement unit, and the first positioning data is used to correct the second positioning data to obtain corrected second positioning data, including: determining the current posture information of the robot based on the second positioning data corresponding to the inertial measurement unit; determining the current position map based on the first positioning data corresponding to the laser radar; and correcting the current posture information based on the current position map to obtain corrected posture information as the corrected second positioning data.

[0008] Optionally, the sensor data fusion strategy is adopted to perform data fusion on the first positioning data and the corrected second positioning data to obtain the positioning information of the robot, including: determining a first weight corresponding to the first positioning data and a second weight corresponding to the corrected second positioning data based on the sensor data fusion strategy; determining a first timestamp of the first positioning data and a second timestamp of the corrected second positioning data; based on the first weight, the second weight, the first timestamp, and the second timestamp, performing data fusion on the first positioning data and the corrected second positioning data to obtain the positioning information.

[0009] Optionally, the absolute positioning sensor includes at least: a global positioning system GPS sensor, a laser radar, and the sensor data fusion strategy is determined based on the environmental information of the robot, including: when the environmental information indicates an indoor environment, the sensor data fusion strategy is to reduce the third weight corresponding to the GPS sensor, increase the fourth weight corresponding to the laser radar, and increase the second weight corresponding to the relative positioning sensor; when the environmental information indicates an optical acquisition restricted environment, the sensor data fusion strategy is to increase the fourth weight corresponding to the laser radar and increase the second weight corresponding to the relative positioning sensor.

[0010] Optionally, the robot is provided with a heterogeneous sensor fusion interface, and the heterogeneous sensor fusion interface at least allows access to a visual sensor. The method further includes: when the visual sensor is a depth camera, using the depth camera to obtain a color image and depth data around the robot; performing registration based on the color image and the depth data to obtain environmental point cloud data; performing identification based on the environmental point cloud data to determine obstacle information; performing path planning based on the obstacle information and the positioning information to determine the movement control strategy of the robot.

[0011] Alternatively, although lidar can provide highly accurate distance information, it is essentially a sensor based on laser reflection, which can only detect the intensity and return time of the reflected signal, so it cannot identify the color or texture of the object. In contrast, the depth camera can not only provide depth information, but also capture the color and texture of the object at the same time. This enables the depth camera to distinguish between stagnant water and other surfaces on the ground through image recognition technology. Stagnant water will present different colors and reflection characteristics from the surrounding dry land in the image. The depth camera can use this information to identify the location of stagnant water, thereby helping the robot avoid potential risks of slipping or equipment damage.

[0012] Optionally, the identifying and determining obstacle information based on the environmental point cloud data includes: determining obstacle shape, obstacle size, and obstacle position based on the environmental point cloud data; classifying based on the obstacle shape, obstacle size, and obstacle position to determine static obstacles and dynamic obstacles as the obstacle information.

[0013] Optionally, the path planning based on the obstacle information and the positioning information to determine the movement control strategy of the robot includes: determining an electronic map based on the alignment of the color image and the depth data and the positioning information; and determining the movement control strategy based on the obstacle information and the electronic map.

[0014] According to another aspect of an embodiment of the present application, a robot positioning device is provided, including: a positioning data determination module, used to obtain first positioning data corresponding to an absolute positioning sensor provided on the robot, and second positioning data corresponding to a relative positioning sensor, wherein the absolute positioning sensor is used to provide positioning data in a world coordinate system, and the relative positioning sensor is used to determine positioning data based on historical displacement; a positioning data correction module, used to use the first positioning data to correct the second positioning data to obtain corrected second positioning data; a data fusion strategy determination module, used to determine a sensor data fusion strategy based on environmental information of the robot; and a positioning information determination module, used to use the sensor data fusion strategy to perform data fusion on the first positioning data and the corrected second positioning data to obtain the positioning information of the robot.

[0015] According to another aspect of an embodiment of the present application, a non-volatile storage medium is provided, wherein the non-volatile storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing any one of the robot positioning methods described above.

[0016] In the embodiment of the present application, by obtaining the first positioning data corresponding to the absolute positioning sensor set on the robot and the second positioning data corresponding to the relative positioning sensor, wherein the absolute positioning sensor is used to provide positioning data in the world coordinate system, and the relative positioning sensor is used to determine the positioning data based on the historical displacement; using the first positioning data, the second positioning data is corrected to obtain the corrected second positioning data; according to the environmental information of the robot, the sensor data fusion strategy is determined; using the sensor data fusion strategy, the first positioning data and the corrected second positioning data are fused to obtain the positioning information of the robot. The purpose of improving the robot's navigation performance and path planning efficiency is achieved, and the technical effect of improving the robot's positioning accuracy and obstacle avoidance ability is achieved, thereby solving the technical problem of unsatisfactory robot positioning accuracy existing in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a flow chart of an optional robot positioning method provided according to an embodiment of the present application;

[0019] Figure 2 It is a schematic diagram of an optional robot positioning device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0023] Extended Kalman Filter (EKF) is a state estimation method for nonlinear systems. It handles the state estimation problem in nonlinear systems by local linearization. In robot navigation, EKF is used to fuse data from multiple sensors to improve the accuracy and stability of position and attitude estimation.

[0024] Particle filter (PF) generates a large number of particles to represent the posterior probability distribution of the system state. As sensor data is continuously input, the weights of the particles will be updated, and the most likely state estimate will eventually be obtained.

[0025] Inertial Measurement Unit (IMU), IMU can measure the acceleration and angular velocity of an object and provide attitude information in some cases, such as pitch angle, roll angle, yaw angle, etc.

[0026] The Global Positioning System (GPS) determines the precise location of a point by receiving signals from multiple satellites. In robot navigation, GPS provides global position information.

[0027] Two-dimensional (2D), 2D refers to the representation in a plane coordinate system, such as X and Y axes. 2D sensors can provide obstacle outline information in the horizontal plane, but cannot provide depth information.

[0028] Three-dimensional (3D), 3D refers to the representation in the spatial coordinate system, including three axes of X, Y and Z. 3D sensors can provide three-dimensional information of the environment.

[0029] Dynamic A* Algorithm: Dynamic A* Algorithm is an improvement on the classic A* search algorithm. It is used to handle dynamic changes in the environment. It can update obstacle information in the map in real time to ensure that the robot can still move safely and efficiently in a dynamic environment.

[0030] Convolutional Neural Network (CNN), CNN can process input data with a grid structure, such as images or videos. CNN identifies local features in the input data by sliding a series of small filters. The filters can detect information such as edges, textures, and colors in the image, and extract features by stacking multiple convolutional layers.

[0031] According to an embodiment of the present application, an embodiment of a method for robot positioning is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] Figure 1 is a flow chart of an optional robot positioning method provided according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0033] Step S102, obtaining first positioning data corresponding to an absolute positioning sensor and second positioning data corresponding to a relative positioning sensor provided on the robot, wherein the absolute positioning sensor is used to provide positioning data in a world coordinate system, and the relative positioning sensor is used to determine positioning data based on historical displacement;

[0034] It can be understood that the first positioning data from the absolute positioning sensor and the second positioning data from the relative positioning sensor are comprehensively collected. The absolute positioning sensor can provide accurate global position information of the robot in the world coordinate system, while the relative positioning sensor determines the current position and posture of the robot by measuring the change relative to the historical displacement. By combining these two types of positioning data, the loss of absolute positioning information caused by unstable signals or indoor environmental restrictions can be effectively compensated, while the high-frequency update characteristics of the relative positioning sensor can be utilized to improve the continuity and real-time performance of positioning.

[0035] Optionally, the absolute positioning sensor includes at least: a global positioning system GPS sensor, a laser radar, and the relative positioning sensor includes at least: an odometer, an inertial measurement unit.

[0036] Optionally, there can be multiple sensor data acquisition methods, for example, dynamic switching of sensor priorities based on environmental changes, intelligent detection of environmental changes, and dynamic switching of sensor priorities in specific environments. For example: when it is detected that the lidar cannot obtain clear environmental information, such as in smoke or rainy environments, the priority of the lidar is automatically reduced, and the reliance on the IMU and odometer is increased; in indoor environments with weak GPS signals, it automatically switches to relying on the lidar and IMU for positioning. The dynamic switching mechanism enhances the robustness of the system in diverse environments.

[0037] Optionally, the heterogeneous sensor data fusion framework can be multiple, for example, the planar information of 2D lidar is fused with the spatial information of 3D inertial sensor to build a more complete 3D environment perception model. The framework allows the addition of other sensors for expansion, such as ultrasonic and visual sensors, to enhance the perception capability of the system and process data from different types of sensors.

[0038] Step S104, using the first positioning data to correct the second positioning data to obtain corrected second positioning data;

[0039] It can be understood that the first positioning data provided by the absolute positioning sensor is used as a reference to correct the second positioning data of the relative positioning sensor. The relative positioning is corrected using the information of absolute positioning to prevent inaccurate positioning during long-term operation or in complex environments, improve the navigation stability and accuracy of the robot, and ensure the reliability of the robot's navigation in various environments.

[0040] Optionally, the correction method can be multiple, for example, through a dynamic error compensation mechanism, based on the changing trend of real-time sensor data and historical data, the accumulated error can be intelligently detected and corrected. For example: by fusing the global position information of the lidar or GPS, the accumulated error of the odometer or IMU is regularly corrected to prevent the error from growing over time. The short-term attitude information of the IMU is matched with the map of the lidar to correct the attitude drift during long-term operation. The system can maintain high-precision positioning during long-term operation, which is suitable for automatic navigation systems that need to run continuously. The IMU is deeply integrated with the odometer, and the instantaneous jitter error of the odometer is compensated by the IMU data; the short-term accuracy of the odometer is combined with the long-term global positioning of the lidar to ensure the provision of continuous and stable position information during long-term driving.

[0041] In an optional embodiment, the absolute positioning sensor includes at least a global positioning system GPS sensor and a laser radar, and uses the first positioning data to correct the second positioning data to obtain the corrected second positioning data, including: performing data fusion based on the first positioning data corresponding to the GPS sensor and the laser radar respectively to obtain global position information; according to a predetermined compensation period, using the global position information to correct the second positioning data to obtain the corrected second positioning data.

[0042] It can be understood that the position information in the earth coordinate system provided by the GPS sensor is combined with the environmental map constructed by the laser radar, and the two data are fused through an algorithm to obtain a more accurate first positioning data. Subsequently, according to the pre-set compensation cycle, the first positioning data is used to correct the cumulative error of the relative positioning sensor to obtain the second positioning data. By utilizing the globality and high precision of the absolute positioning sensor, and the continuity and high-frequency update characteristics of the relative positioning sensor, the positioning accuracy of the robot in long-term operation or complex environments is improved through periodic calibration and error compensation.

[0043] In an optional embodiment, the absolute positioning sensor includes at least a laser radar, and the relative positioning sensor includes at least an inertial measurement unit. The first positioning data is used to correct the second positioning data to obtain the corrected second positioning data, including: determining the current posture information of the robot based on the second positioning data corresponding to the inertial measurement unit; determining the current position map based on the first positioning data corresponding to the laser radar; and correcting the current posture information based on the current position map to obtain the corrected posture information as the corrected second positioning data.

[0044] It can be understood that the environmental map constructed by the lidar is used as the first positioning data, providing a global position reference based on the characteristics of the surrounding environment. The inertial measurement unit (IMU) is used to obtain the current posture information of the robot, including acceleration, angular velocity, etc. The system intelligently adjusts the posture information by comparing and fusing the current posture information of the IMU with the current position map constructed by the lidar to compensate for the cumulative error caused by the IMU due to the integration process, thereby obtaining the corrected posture information as the corrected second positioning data. In terms of IMU cumulative error correction, the map generated by the lidar provides a reliable external reference that can help the system identify and correct posture deviations caused by sensor drift, integration error or motion uncertainty.

[0045] Step S106, determining a sensor data fusion strategy based on the environment information of the robot;

[0046] It can be understood that the sensor data fusion strategy is determined based on the specific information of the robot's environment. By analyzing environmental data, such as terrain features, lighting conditions, signal interference levels, etc., the weights and fusion algorithms of different sensors are intelligently selected and adjusted to optimize positioning accuracy.

[0047] Optionally, the weight calculation method can be multiple, for example, an adaptive weight adjustment mechanism is introduced to adjust the weight of each sensor in the fusion algorithm in real time according to the reliability of different sensors. For example: in an environment with weak GPS signals, the weight of GPS data is automatically reduced, and the weight of IMU and odometer is increased; when the odometer cumulative error is detected to be large, the system automatically increases the weight of LiDAR to ensure the overall positioning accuracy. This adaptive weight adjustment strategy improves the system's ability to adapt to complex environments.

[0048] Optionally, the above fusion algorithm can be multiple, for example, using an algorithm architecture that combines an extended Kalman filter (EKF) and a particle filter (PF). Under different sensor combinations, the system can flexibly select a suitable filtering algorithm: the extended Kalman filter is used for efficient fusion in linear and approximately linear scenarios; the particle filter provides more accurate pose estimation in nonlinear scenarios (such as complex map matching).

[0049] In an optional embodiment, the absolute positioning sensor includes at least: a global positioning system GPS sensor, a laser radar, and a sensor data fusion strategy is determined based on the environmental information of the robot, including: when the environmental information indicates an indoor environment, the sensor data fusion strategy is to reduce the third weight corresponding to the GPS sensor, increase the fourth weight corresponding to the laser radar, and increase the second weight corresponding to the relative positioning sensor; when the environmental information indicates an optical acquisition restricted environment, the sensor data fusion strategy is the fourth weight corresponding to the laser radar, and increase the second weight corresponding to the relative positioning sensor.

[0050] It can be understood that in indoor environments, due to the weak or completely unavailable GPS signal, the third weight corresponding to the GPS sensor is automatically reduced, while the fourth weight corresponding to the lidar is correspondingly increased, and the second weight corresponding to the relative positioning sensor is increased. This strategy adjustment ensures that in indoor environments, the system relies more on the high-precision positioning capability of the lidar and the continuous attitude update of the relative positioning sensor to achieve stable and accurate positioning. In optical acquisition restricted environments, such as strong light, low light or highly reflective surfaces, the performance of the lidar may decline, reducing the fourth weight corresponding to the lidar and increasing the second weight corresponding to the relative positioning sensor to ensure that under adverse optical conditions, the system can still maintain the continuity and reliability of positioning through sensors such as IMU and odometer.

[0051] Step S108, adopting a sensor data fusion strategy to fuse the first positioning data and the corrected second positioning data to obtain the positioning information of the robot.

[0052] It can be understood that according to the determined sensor data fusion strategy, the first positioning data from the absolute positioning sensor and the corrected second positioning data are comprehensively analyzed and fused to obtain the positioning information of the robot. By adopting an intelligent data fusion strategy, the data fusion method and ratio are dynamically adjusted according to real-time environmental changes and sensor characteristics to ensure that the optimal positioning information can be provided in various scenarios.

[0053] In an optional embodiment, a sensor data fusion strategy is adopted to fuse the first positioning data and the corrected second positioning data to obtain the positioning information of the robot, including: determining a first weight corresponding to the first positioning data and a second weight corresponding to the corrected second positioning data based on the sensor data fusion strategy; determining a first timestamp of the first positioning data and a second timestamp of the corrected second positioning data; based on the first weight, the second weight, the first timestamp, and the second timestamp, fusing the first positioning data and the corrected second positioning data to obtain positioning information.

[0054] It can be understood that the sensor data fusion strategy is used to fuse the first positioning data from different positioning sensors and the corrected second positioning data, and the positioning information is obtained by determining the weights and timestamps of the first positioning data and the second positioning data. The data fusion method based on weights and timestamps avoids the limitations of a single sensor and realizes the complementarity and enhancement of positioning information.

[0055] Optionally, the above data fusion method at least includes timestamp synchronization. Since the sampling frequency and delay of each sensor are different, the timestamp synchronization management of different sensor data is carried out, and the seamless fusion of high-frequency (such as IMU) and low-frequency (such as LiDAR, GPS) data is achieved through precise interpolation and alignment algorithms. The fusion accuracy of sensor data is improved, and data distortion caused by time series asynchrony is avoided.

[0056] In an optional embodiment, the robot is provided with a heterogeneous sensor fusion interface, which at least allows access to a visual sensor. The method further includes: when the visual sensor is a depth camera, using the depth camera to obtain color images and depth data around the robot; performing registration based on the color image and depth data to obtain environmental point cloud data; performing identification based on the environmental point cloud data to determine obstacle information; performing path planning based on the obstacle information and positioning information to determine the robot's mobile control strategy.

[0057] It can be understood that the robot has a built-in heterogeneous sensor fusion interface, which at least allows access to a visual sensor. When the visual sensor is a depth camera, it obtains the color image and depth data of the environment in real time, and generates environmental point cloud data of the environment through precise registration of the image and depth information. Obstacles are identified based on the environmental point cloud data, and the shape, size and distance of the obstacles are determined, which greatly improves the robot's obstacle avoidance ability in complex environments. Subsequently, the obstacle information is combined with the robot's precise positioning information to perform path planning, determine the robot's mobile control strategy, avoid obstacles, and reach the destination safely and efficiently, improving the accuracy and flexibility of the robot's navigation.

[0058] Optionally, the depth camera is used to collect color images and depth information of the environment in real time. The depth camera can capture the shape and distance of objects in complex 3D environments, overcoming the limitations of laser radar and ultrasonic sensors in three-dimensional obstacle detection.

[0059] Optionally, the above-mentioned dynamic path planning at least includes: using an improved dynamic A* algorithm, combined with real-time obstacle data, to dynamically update the robot's forward path. The algorithm supports rapid adjustment of local path planning, ensuring that the robot can quickly respond and adjust the direction of travel when the environment changes. The path planning results are converted into motion control instructions to control the robot's movement in real time, including forward movement, steering, obstacle avoidance and other operations. Through wireless communication technology, the system is connected to the remote control center to detect the robot's operating status and sensor data in real time.

[0060] In an optional embodiment, obstacle information is determined based on environmental point cloud data, including: determining obstacle shape, obstacle size, and obstacle position based on environmental point cloud data; classifying based on obstacle shape, obstacle size, and obstacle position to determine static obstacles and dynamic obstacles as obstacle information.

[0061] It can be understood that through deep information analysis, the shape, size and position coordinates of obstacles can be accurately determined, and then the obstacles can be effectively classified to distinguish between static obstacles and dynamic obstacles. The robot can respond quickly and accurately to obstacles in the environment, especially for real-time detection and identification of dynamic obstacles, which improves the robot's obstacle avoidance ability and safety in complex dynamic environments.

[0062] In an optional embodiment, path planning is performed based on obstacle information and positioning information to determine the robot's mobile control strategy, including: determining an electronic map based on color images and depth data and positioning information for alignment; and determining a mobile control strategy based on obstacle information and the electronic map.

[0063] It can be understood that the color image and depth data obtained by the depth camera are combined with positioning information to build an electronic map. Based on real-time electronic map data and obstacle information, the strategy is quickly adjusted according to the real-time changes in the environment, and the robot's mobile control strategy is determined, which improves the robot's autonomous navigation ability and makes it more adaptable and flexible in complex and changing environments.

[0064] Through the above step S102, the first positioning data corresponding to the absolute positioning sensor set on the robot and the second positioning data corresponding to the relative positioning sensor are obtained, wherein the absolute positioning sensor is used to provide positioning data in the world coordinate system, and the relative positioning sensor is used to determine the positioning data based on the historical displacement; step S104, using the first positioning data, correcting the second positioning data to obtain the corrected second positioning data; step S106, according to the environmental information of the robot, determining the sensor data fusion strategy; step S108, using the sensor data fusion strategy, data fusion of the first positioning data and the corrected second positioning data, to obtain the positioning information of the robot. The purpose of improving the navigation performance and path planning efficiency of the robot can be achieved, and the technical effect of improving the positioning accuracy and obstacle avoidance ability of the robot is achieved, thereby solving the technical problem of unsatisfactory robot positioning accuracy existing in the related technology.

[0065] Based on the above embodiments and optional embodiments, the present application proposes an optional implementation manner.

[0066] Step S1, collects the first positioning data from the absolute positioning sensor and the second positioning data from the relative positioning sensor. That is, real-time data is obtained from sensors such as odometers, IMUs, lidars and GPS, and the data of each sensor is preprocessed. The absolute positioning sensor can provide accurate global position information of the robot in the world coordinate system, while the relative positioning sensor determines the current position and posture of the robot by measuring the changes relative to the historical displacement. By combining these two types of positioning data, the loss of absolute positioning information caused by unstable signals or indoor environmental restrictions can be effectively compensated, and the high-frequency update characteristics of the relative positioning sensor can be used to improve the continuity and real-time performance of positioning.

[0067] Optionally, the absolute positioning sensor includes at least: a global positioning system GPS sensor, a laser radar, and the relative positioning sensor includes at least: an odometer, an inertial measurement unit. The odometer estimates the relative position and posture of the robot by measuring the rotation speed and displacement of the wheels. It has high short-term accuracy, but is prone to cumulative errors during long-term operation. The IMU can provide the robot's acceleration and angular velocity information for estimating posture changes. The IMU responds quickly in a short period of time, but its errors will also accumulate over time. The laser radar (LiDAR, Laser Imaging Detection And Ranging) is used to scan the surrounding environment, build a two-dimensional or three-dimensional environmental map, and achieve positioning by comparing it with a pre-built map. The laser radar has a good positioning effect in complex environments, but its accuracy will decrease when the environment changes greatly or there are fewer obstacles. The GPS module provides global position information and is suitable for outdoor environments, but in indoor environments or environments with weak GPS signals, the GPS positioning accuracy is poor or even fails.

[0068] Step S2, the pre-processed sensor data is time-aligned through the data synchronization module to ensure the consistency of the timeliness of each sensor data. Through Kalman filtering or other advanced fusion algorithms, the data of each sensor is weighted to form accurate location information. If it is detected that the error accumulation of the odometer or IMU is large, the system will perform error correction through the data of the lidar or GPS to maintain the stability of the global positioning. That is, the first positioning data provided by the absolute positioning sensor is used as a reference benchmark to correct the second positioning data of the relative positioning sensor. Use the information of absolute positioning to correct the relative positioning to prevent inaccurate positioning during long-term operation or in complex environments, improve the navigation stability and accuracy of the robot, and ensure the reliability of the robot navigation in various environments.

[0069] The data of odometer, IMU, lidar and GPS are fused through algorithms such as Kalman filter and particle filter. Combining the advantages of each sensor, the data weights of different sensors are flexibly adjusted in different working scenarios to provide the best positioning estimate. For the accumulated errors of odometer and IMU, the absolute position information of lidar and GPS is fused to regularly correct the errors to ensure the positioning accuracy of the robot for long-term operation. Each sensor collects data at different frequencies, so the sensor data needs to be synchronized in time. The data of each sensor is aligned through timestamps to ensure the accuracy of data fusion.

[0070] The system adopts a modular design architecture, and users can flexibly choose different sensor combinations and fusion strategies according to their needs. It can be customized according to different application scenarios (such as autonomous driving, drones, robots, etc.), which improves the versatility of the system.

[0071] At the hardware level, the system is designed with fault tolerance and redundancy mechanisms. Even if a sensor fails or the data quality is poor, the data from other sensors can still maintain the normal operation of the system and ensure the stability of positioning and navigation. For example, when the IMU fails, the system can still achieve relatively accurate positioning through lidar and odometer; when the GPS signal fails, the system automatically switches to a positioning solution based on lidar and odometer to ensure that the robot continues to operate.

[0072] Step S3, determines the sensor data fusion strategy based on the specific information of the robot's environment, and intelligently selects and adjusts the weights and fusion algorithms of different sensors by analyzing environmental data, such as terrain features, lighting conditions, signal interference levels, etc., to optimize positioning accuracy.

[0073] Step S4, by adopting an intelligent data fusion strategy, dynamically adjust the data fusion mode and ratio according to real-time environmental changes and sensor characteristics to ensure that the best positioning information can be provided in various scenarios. The fused positioning information is fed back to the robot control system to guide the robot to complete the autonomous navigation task.

[0074] The robot has a built-in heterogeneous sensor fusion interface, which allows at least the access of visual sensors. When the visual sensor is a depth camera, the color image and depth data of the environment are obtained in real time through infrared ranging technology. Through the precise registration of image and depth information, the environmental point cloud data of the environment is generated. Obstacles are identified based on the environmental point cloud data, and the shape, size and distance of obstacles are determined, which greatly improves the robot's obstacle avoidance ability in complex environments. Subsequently, the obstacle information is combined with the robot's precise positioning information to perform path planning, determine the robot's mobile control strategy, avoid obstacles, and reach the destination safely and efficiently, improving the accuracy and flexibility of the robot's navigation.

[0075] Sensor fusion at least includes fusing depth camera data, IMU (inertial measurement unit) sensor data, and odometer data to generate a real-time environmental map. Through time synchronization and spatial calibration, the effective combination of different sensor data is ensured, thereby improving the accuracy and stability of environmental information.

[0076] Obstacle detection receives the fused depth information and uses algorithms to classify and identify objects in the scene, including static obstacles (such as walls, furniture) and dynamic obstacles (such as pedestrians, vehicles). The algorithm can accurately locate obstacles of different sizes and distances. Based on the obstacle detection results, the improved A* dynamic path planning algorithm is used to generate the optimal path from the current robot position to the target position. The algorithm will take into account the current dynamic changes in the environment and continuously update the path to ensure that the robot reaches its destination smoothly. The planned path is converted into a control signal and sent to the robot's actuator to control its forward movement, obstacle avoidance or steering. The real-time operating status and environmental information are uploaded to the remote control center, allowing the operator to intervene or adjust the robot's tasks at any time.

[0077] The above optional implementation methods achieve at least the following effects: by fusing data from multiple sensors and adopting high-precision time synchronization and space alignment technology, data from different sensors can be seamlessly integrated to provide more comprehensive environmental perception, make up for the shortcomings of a single sensor in complex environments, and improve the positioning accuracy of the robot; based on the three-dimensional point cloud data of the depth camera and the improved A* path planning algorithm, the robot can accurately detect various obstacles in the three-dimensional environment, especially difficult-to-detect obstacles such as suspended objects and steps, and respond quickly to changes in the environment to ensure safe and efficient driving in dynamic and complex environments; even if some sensors fail or the data is unreliable, it can still maintain stable positioning capabilities and enhance robustness through the valid data provided by other sensors.

[0078] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0079] In this embodiment, a robot positioning device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the terms "module" and "device" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0080] According to an embodiment of the present application, a device embodiment for implementing a robot positioning method is also provided. Figure 2 is a schematic diagram of a robot positioning device according to an embodiment of the present application, such as Figure 2As shown, the robot positioning device includes a positioning data determination module 202, a positioning data correction module 204, a data fusion strategy determination module 206, and a positioning information determination module 208. The device is described below.

[0081] A positioning data determination module 202 is used to obtain first positioning data corresponding to an absolute positioning sensor and second positioning data corresponding to a relative positioning sensor provided on the robot, wherein the absolute positioning sensor is used to provide positioning data in a world coordinate system, and the relative positioning sensor is used to determine positioning data based on historical displacement;

[0082] The positioning data correction module 204 is connected to the positioning data determination module 202 and is used to correct the second positioning data using the first positioning data to obtain the corrected second positioning data;

[0083] A data fusion strategy determination module 206 is connected to the positioning data correction module 204 and is used to determine a sensor data fusion strategy based on the environment information of the robot;

[0084] The positioning information determination module 208 is connected to the data fusion strategy determination module 206 and is used to adopt a sensor data fusion strategy to perform data fusion on the first positioning data and the corrected second positioning data to obtain the positioning information of the robot.

[0085] In a robot positioning device provided by an embodiment of the present application, a positioning data determination module 202 is set to obtain the first positioning data corresponding to the absolute positioning sensor set on the robot, and the second positioning data corresponding to the relative positioning sensor, wherein the absolute positioning sensor is used to provide positioning data in the world coordinate system, and the relative positioning sensor is used to determine the positioning data based on the historical displacement; the positioning data correction module 204 is connected to the positioning data determination module 202, and is used to use the first positioning data to correct the second positioning data to obtain the corrected second positioning data; the data fusion strategy determination module 206 is connected to the positioning data correction module 204, and is used to determine the sensor data fusion strategy according to the environmental information of the robot; the positioning information determination module 208 is connected to the data fusion strategy determination module 206, and is used to use the sensor data fusion strategy to fuse the first positioning data and the corrected second positioning data to obtain the positioning information of the robot. The purpose of improving the robot navigation performance and path planning efficiency is achieved, and the technical effect of improving the robot positioning accuracy and obstacle avoidance ability is achieved, thereby solving the technical problem of unsatisfactory robot positioning accuracy existing in the related technology.

[0086] It should be noted that the above modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following ways: the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0087] It should be noted that the above-mentioned module for determining positioning data 202, the module for correcting positioning data 204, the module for determining data fusion strategy 206, and the module for determining positioning information 208 correspond to steps S102 to S108 in the embodiment, and the examples and application scenarios implemented by the above-mentioned modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiment. It should be noted that the above-mentioned modules can be run in a computer terminal as part of the device.

[0088] It should be noted that the optional or preferred implementation of this embodiment can refer to the relevant description in the embodiment, which will not be repeated here.

[0089] The above-mentioned robot positioning device may also include a processor and a memory. The positioning data determination module 202, the positioning data correction module 204, the data fusion strategy determination module 206, the positioning information determination module 208, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0090] The processor includes a kernel, which retrieves the corresponding program unit from the memory. There can be one or more kernels. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0091] An embodiment of the present application provides a non-volatile storage medium on which a program is stored, and when the program is executed by a processor, a robot positioning method is implemented.

[0092] The embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program stored in the memory and can be run on the processor. When the processor executes the program, the following steps are implemented: obtaining the first positioning data corresponding to the absolute positioning sensor set on the robot, and the second positioning data corresponding to the relative positioning sensor, wherein the absolute positioning sensor is used to provide positioning data in the world coordinate system, and the relative positioning sensor is used to determine the positioning data based on the historical displacement; using the first positioning data, correcting the second positioning data to obtain the corrected second positioning data; determining the sensor data fusion strategy based on the environmental information of the robot; using the sensor data fusion strategy, data fusion of the first positioning data and the corrected second positioning data to obtain the positioning information of the robot. The device in this article can be a server, a PC, etc.

[0093] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: obtaining first positioning data corresponding to an absolute positioning sensor provided on the robot, and second positioning data corresponding to a relative positioning sensor, wherein the absolute positioning sensor is used to provide positioning data in a world coordinate system, and the relative positioning sensor is used to determine positioning data based on historical displacement; using the first positioning data, correcting the second positioning data to obtain corrected second positioning data; determining a sensor data fusion strategy based on environmental information of the robot; using the sensor data fusion strategy, performing data fusion on the first positioning data and the corrected second positioning data to obtain the positioning information of the robot.

[0094] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0095] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0096] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0097] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0098] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0099] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0100] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0101] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0102] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0103] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A robot positioning method, characterized in that: include: Acquire first positioning data corresponding to an absolute positioning sensor and second positioning data corresponding to a relative positioning sensor provided on the robot, wherein the absolute positioning sensor is used to provide positioning data in a world coordinate system, and the relative positioning sensor is used to determine positioning data based on historical displacement; Using the first positioning data, correcting the second positioning data to obtain corrected second positioning data; Determining a sensor data fusion strategy based on the environment information of the robot; The sensor data fusion strategy is adopted to fuse the first positioning data and the corrected second positioning data to obtain the positioning information of the robot.

2. The method according to claim 1, characterized in that The absolute positioning sensor at least includes a global positioning system GPS sensor and a laser radar, and the first positioning data is used to correct the second positioning data to obtain the corrected second positioning data, including: Perform data fusion based on the first positioning data corresponding to the GPS sensor and the laser radar respectively to obtain global position information; According to a predetermined compensation period, the second positioning data is corrected using the global position information to obtain the corrected second positioning data.

3. The method according to claim 1, characterized in that The absolute positioning sensor includes at least a laser radar, the relative positioning sensor includes at least an inertial measurement unit, and the first positioning data is used to correct the second positioning data to obtain the corrected second positioning data, including: Determining current posture information of the robot based on second positioning data corresponding to the inertial measurement unit; Determine a current location map based on the first positioning data corresponding to the laser radar; Based on the current position map, the current posture information is corrected to obtain corrected posture information as the corrected second positioning data.

4. The method according to claim 1, characterized in that: The adopting the sensor data fusion strategy to fuse the first positioning data and the corrected second positioning data to obtain the positioning information of the robot includes: Based on the sensor data fusion strategy, determine a first weight corresponding to the first positioning data and a second weight corresponding to the corrected second positioning data; Determining a first timestamp of the first positioning data and a second timestamp of the corrected second positioning data; Based on the first weight, the second weight, the first timestamp, and the second timestamp, data fusion is performed on the first positioning data and the corrected second positioning data to obtain the positioning information.

5. The method according to claim 1, characterized in that The absolute positioning sensor includes at least: a global positioning system GPS sensor, a laser radar, and the sensor data fusion strategy is determined based on the environmental information of the robot, including: When the environmental information indicates an indoor environment, the sensor data fusion strategy is to reduce the third weight corresponding to the GPS sensor, increase the fourth weight corresponding to the laser radar, and increase the second weight corresponding to the relative positioning sensor; When the environmental information indicates an optical acquisition restricted environment, the sensor data fusion strategy is to use the fourth weight corresponding to the laser radar to increase the second weight corresponding to the relative positioning sensor.

6. The method according to any one of claims 1 to 5, characterized in that The robot is provided with a heterogeneous sensor fusion interface, and the heterogeneous sensor fusion interface at least allows access to a visual sensor, and the method further includes: In the case where the visual sensor is a depth camera, the depth camera is used to obtain a color image and depth data of the robot's surroundings; Performing registration based on the color image and the depth data to obtain environmental point cloud data; Identify and determine obstacle information based on the environmental point cloud data; Path planning is performed based on the obstacle information and the positioning information to determine the movement control strategy of the robot.

7. The method according to claim 6, characterized in that The step of identifying and determining obstacle information based on the environmental point cloud data includes: Determine the shape, size, and position of an obstacle based on the environmental point cloud data; Classification is performed based on the obstacle shape, the obstacle size, and the obstacle position, and static obstacles and dynamic obstacles are determined as the obstacle information.

8. The method according to claim 6, characterized in that The performing of path planning based on the obstacle information and the positioning information to determine the movement control strategy of the robot includes: Performing registration based on the color image, the depth data, and the positioning information to determine an electronic map; The movement control strategy is determined based on the obstacle information and the electronic map.

9. A robot positioning device, characterized in that: include: A positioning data determination module is used to obtain first positioning data corresponding to an absolute positioning sensor provided on the robot, and second positioning data corresponding to a relative positioning sensor, wherein the absolute positioning sensor is used to provide positioning data in a world coordinate system, and the relative positioning sensor is used to determine positioning data based on historical displacement; a positioning data correction module, configured to correct the second positioning data using the first positioning data to obtain corrected second positioning data; A data fusion strategy module is used to determine the sensor data fusion strategy according to the environment information of the robot; A positioning information determination module is used to adopt the sensor data fusion strategy to perform data fusion on the first positioning data and the corrected second positioning data to obtain the positioning information of the robot.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the robot positioning method according to any one of claims 1 to 8.

Images