ROS-based robot environment sensing and positioning system

Abstract

The invention discloses a robot environment sensing and positioning system based on an ROS. Effective data of multiple sensors can be efficiently processed, and the timestamps are unified for data fusion. According to the invention, an NUC11 industrial personal computer receives sensor data of an inertial sensor, a laser radar and an industrial camera and transmits the sensor data to a corresponding algorithm module for processing; a grid map is established after the surrounding environment is sensed; the speed and the direction are sent to the STM32 single-chip microcomputer through serial port communication, the STM32 single-chip microcomputer calculates the speed and the direction and then converts the speed and the direction into current values, current value data are sent to the motor through CAN communication, the Mecanum wheels are driven to move in all directions, and the navigation robot capable of achieving self-positioning and omni-directional movement is achieved. According to the omni-directional moving efficient wheeled robot, Mecanum wheels are innovatively combined to achieve omni-directional moving, a universal positioning and navigation robot chassis frame system is constructed, and the omni-directional moving efficient wheeled robot can become a universal solution for a cleaning robot, a meal delivery robot, an express delivery robot and the like in combination with different application scenes.

Description

Technical Field

[0001] The present invention belongs to the field of simultaneous positioning and mapping in robot three-dimensional vision. The method is a robot environment perception and positioning system based on ROS. Specifically, it is a method that utilizes the ROS communication architecture to fuse multi-sensor information. It can comprehensively improve the positioning accuracy of the SLAM system and enhance the robot's autonomous motion capability. It can be actually applied to the control and autonomous motion of wheeled robots. Background Art

[0002] The Robot Operating System (ROS) is an open source framework that provides a set of tools, libraries, and rules to simplify the development of robotics software. It is widely used in the robotics research community due to its modular and flexible design, which allows developers to easily integrate sensors, algorithms, and control systems.

[0003] A ROS-based robot environmental perception and localization system focuses on the robot's ability to perceive and understand its environment and accurately locate itself within it. This involves integrating various sensor modalities, such as cameras, lidar, and inertial measurement units (IMUs), to generate a comprehensive map of the robot's surroundings. This map can then be used to determine the robot's position and plan its path within the environment.

[0004] SLAM (Simultaneous Localization and Mapping) systems are systems that simultaneously perform mapping and positioning. Commonly used in robotics and drones, SLAM systems can map the robot's surroundings and simultaneously localize the robot.

[0005] There are many different approaches and technologies for SLAM systems, including those based on sensors such as LiDAR, vision, and inertial measurement units (IMUs). LiDAR is a very useful technology with widespread application in a variety of real-world scenarios. For example, it can be used in devices such as robot vacuums to help them perceive their surroundings and plan paths. However, LiDAR is expensive, making it unsuitable for all applications. Furthermore, LiDAR can be affected by factors such as interference, which can affect its performance. Consequently, people are looking for alternative solutions to these issues. Visual SLAM is a widely researched alternative that has begun to gain public attention and find increasing application. This approach, based on computer vision technology, can perform functions such as environmental perception and path planning by analyzing image and video data.

[0006] In the real-time positioning of robots, since the robot's position information obtained through motion estimation often has large errors, we also need to use the surrounding environment information obtained by the ranging unit to correct the robot's position. Currently, common ranging units include laser ranging, ultrasonic ranging, and image ranging. Among them, lidar has become the core sensor of mobile robots due to its excellent directivity and high focusing ability. It is also the most reliable and stable positioning technology.

[0007] However, it also has some shortcomings, such as easy matching errors in complex environments such as strong light and dynamic scenes, and slow processing speed due to large computational workload. Therefore, how to select appropriate sensor information sources and efficiently fuse them is a very important and urgent problem to be solved. Summary of the Invention

[0008] The technical problems to be solved by the present invention are: insufficient positioning and mapping accuracy of robots based on Mecanum wheeled chassis; and the failure of positioning and mapping in complex scenes.

[0009] In response to the shortcomings of the existing technology, the present invention provides a robot environment perception and positioning system based on ROS. The present invention can efficiently process the valid data of multiple sensors (cameras, lidars, inertial sensors) and unify the timestamps for data fusion, ultimately improving the positioning and mapping accuracy of the robot. The present invention uses lidars, industrial cameras, and inertial sensors as the main sources of sensor data, communicates through the ROS architecture, stores map information through grid maps, and locates the robot position through corresponding algorithms; and innovatively combines Mecanum wheels to achieve an efficient wheeled robot with omnidirectional mobility, constructing a universal positioning and navigation robot chassis frame system, which can be combined with different application scenarios to become a universal solution for cleaning robots, food delivery robots, express delivery robots, etc.

[0010] A robot environment perception and positioning system based on ROS, mainly including the following designs:

[0011] The hardware includes a chassis frame, motor, Mecanum wheels, inertial measurement units (IMUs), an STM32 microcontroller, a NUC11 industrial computer, a Silam A2 LiDAR sensor, and an industrial camera. The system's functionality is implemented by the NUC11 industrial computer receiving sensor data from the inertial sensors, LiDAR, and industrial camera and processing it through corresponding algorithm modules to perceive the surrounding environment and build a grid map. The speed and direction are then transmitted to the STM32 microcontroller via serial communication. The STM32 microcontroller calculates the speed and direction and converts them into current values. This current value is then transmitted to the motor via CAN communication, driving the Mecanum wheels for omnidirectional movement, achieving autonomous positioning and omnidirectional navigation.

[0012] The software modules include: IMU driver node, camera driver node, lidar driver node, costmap node, positioning node, communication node, path planning node, and decision node. The positioning node receives and processes sensor information sent by the IMU, camera, and lidar to determine their respective positions. After the decision node determines the target position, the path planning node dynamically plans the optimal path in real time based on the costmap nodes. This information is then sent to the STM32 microcontroller via the communication node to drive the motors.

[0013] The beneficial effects of the present invention are as follows:

[0014] (1) Multi-sensor synchronization and data fusion technology: The integration of multiple sensors such as cameras, lidar, and IMU is crucial for generating a comprehensive map of the robot's surroundings. However, synchronizing and fusing data from different sensors is challenging, especially in dynamic and unstructured environments. This paper implements data synchronization and an efficient fusion algorithm.

[0015] (2) Accurate positioning and mapping technology: Accurate positioning and mapping are crucial for successful robot operation in complex environments. This invention implements advanced SLAM technology that can handle large-scale environments and achieve long-term autonomous operation.

[0016] (3) Algorithm deployment and application value: The multi-sensor SLAM algorithm based on ROS communication is deployed on a wheeled robot, and a universal positioning and navigation robot chassis framework system is finally realized through environmental perception and autonomous decision-making. Combined with different application scenarios, it can become a universal solution for cleaning robots, food delivery robots, express delivery robots, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a diagram of the system hardware structure of the present invention.

[0018] Figure 2 This is a schematic diagram of the installation method of the Mecanum wheel chassis of the present invention.

[0019] Figure 3 It is the algorithm flow chart of the present invention.

[0020] Figure 4 It is a schematic diagram of positioning correction of the present invention.

[0021] Figure 5 Schematic diagram of Gaussian distribution in the nonlinear system of the present invention.

[0022] Figure 6 This is an approximate diagram of the Kalman filter in a nonlinear system. DETAILED DESCRIPTION

[0023] The invention patent is further described in detail below with reference to the accompanying drawings and embodiments:

[0024] This paper provides a robot environment perception and positioning system based on ROS, which efficiently processes valid data from multiple sensors (cameras, lidars, inertial sensors) and integrates data with unified timestamps, ultimately improving the robot's positioning and mapping accuracy. The method described in this paper includes the following parts:

[0025] The hardware includes a chassis frame, motor, Mecanum wheels, inertial measurement units (IMUs), an STM32 microcontroller, a NUC11 industrial computer, a Silam A2 LiDAR sensor, and an industrial camera. The system's functionality is implemented by the NUC11 industrial computer receiving sensor data from the inertial sensors, LiDAR, and industrial camera and processing it through corresponding algorithm modules to perceive the surrounding environment and build a grid map. The speed and direction are then transmitted to the STM32 microcontroller via serial communication. The STM32 microcontroller calculates the speed and direction and converts them into current values. This current value is then transmitted to the motor via CAN communication, driving the Mecanum wheels for omnidirectional movement, achieving autonomous positioning and omnidirectional navigation.

[0026] The software modules include: IMU driver node, camera driver node, lidar driver node, costmap node, positioning node, communication node, path planning node, and decision node. The positioning node receives and processes sensor information sent by the IMU, camera, and lidar to determine their respective positions. After the decision node determines the target position, the path planning node dynamically plans the optimal path in real time based on the costmap nodes. This information is then sent to the STM32 microcontroller via the communication node to drive the motors.

[0027] The specific implementation of the hardware device includes the following parts:

[0028] (1) System architecture module: The system function is realized by the NUC11 industrial computer receiving the sensor data of the inertial sensor, laser radar, and industrial camera and handing it over to the corresponding algorithm module for processing: the speed variable and acceleration variable collected by the inertial sensor are processed by the mean filter algorithm; the point cloud collected by the laser radar is processed by denoising and dedistortion; the image collected by the industrial camera is processed by dedistortion; the surrounding environment is perceived and a grid map is established; the speed variable and direction variable of the path planning are sent to the STM32 microcontroller through serial communication, and the microcontroller converts them into current values, and sends the data to the motor through the CAN communication protocol to drive the Mecanum wheel to move omnidirectionally; at the same time, the angle of the motor encoder is calculated to construct a wheel odometer to assist the laser radar in positioning; the functions of autonomous positioning and omnidirectional movement are realized, and the system hardware structure is as follows Figure 1 shown.

[0029] (2) Chassis structure module: The chassis is the main structure of the wheeled robot, equipped with motion actuators and sensors. In order to achieve omnidirectional movement, the present invention uses four Mecanum wheels. Wheel A is a left-handed wheel, and wheel B is a right-handed wheel. They are installed in an O-rectangle manner (the diagonal installation method is the same). The specific installation method is as follows: Figure 2 As shown in Figure 2, a Mecanum wheel consists of a hub and rollers. The hub is the main frame of the wheel, while the rollers are barrel-shaped components mounted on the hub. The hub axis of the Mecanum wheel is at a 45-degree angle to the roller axis. The combined force generated by the four wheels enables omnidirectional movement of the robot.

[0030] The chassis is equipped with a variety of sensors, so its shock-absorbing design is crucial. This invention utilizes a terrain-adaptive suspension design. When the wheels become suspended due to environmental factors, a torque is applied to the suspension supports, causing them to contact the ground. This ensures that all four Mecanum wheels are in contact, ensuring the robot's normal omnidirectional operation. Furthermore, the excellent shock-absorbing design effectively reduces the interference of bumpy roads on the wheel odometer and the impact of other sensor data, thereby achieving high-quality positioning accuracy. Furthermore, the chassis design's high scalability facilitates the integration of multiple sensors and their integration with specific application scenarios to achieve corresponding business functions.

[0031] (3) Sensor module: Based on the expected functions of the system design and combined with market research, in order to achieve the expected design functions, the sensor models selected for this system are inertial measurement unit, STM32 single-chip microcomputer, NUC11 industrial computer, Silan A2 laser radar, and industrial camera. The system function is realized by the NUC11 industrial computer receiving the sensor data of the inertial sensor, laser radar, and industrial camera and passing it to the corresponding algorithm for processing: the two variables (speed and acceleration) collected by the inertial sensor are processed by the mean filter algorithm. The point cloud collected by the laser radar is processed by denoising and dedistortion. Specifically, the sampling statistical filter identifies and removes abnormal points by calculating the distance distribution of points in the neighborhood of each point. The specific steps are: ① For each point, calculate the average distance and standard deviation of its neighborhood (usually k nearest neighbors or points within a certain radius). ② Set a threshold. If the average distance of a point's neighborhood exceeds this threshold (usually twice the global average distance), the point is considered a noise point and removed. Images captured by the industrial camera undergo dedistortion processing. Specifically, a correction transformation matrix is ​​calculated using camera calibration parameters and then applied to the captured image. This algorithm senses the surrounding environment and creates a grid map. The two variables used in path planning (speed and direction) are then transmitted to the STM32 microcontroller via serial communication. The microcontroller then converts the calculated values ​​into current values, which are then transmitted to the motor via CAN communication.

[0032] (4) Algorithm module

[0033] This system is based on the ROS framework because ROS is a distributed architecture with low module coupling, allowing for the arbitrary addition of new driver modules or functional modules. At the same time, topic-based communication facilitates efficient development, treating each module as a black box. When calling open-source algorithms, one only needs to understand the module's input and output types and parameters to connect the entire system. The algorithm module implements the following functions:

[0034] Step 1, map construction node (Gmapping);

[0035] Step 2: Matching and positioning nodes (AMCL);

[0036] Step 3, path planning control node (Move_base);

[0037] Furthermore, step 1 is specifically implemented as follows:

[0038] Mapping is done using the open-source Gmapping algorithm. Because mapping isn't always necessary, a grid map is only needed after the initial build. If the environment doesn't change much, this map can be used over time. Therefore, dashed lines are used in the system architecture diagram.

[0039] Furthermore, step 2 is implemented as follows:

[0040] Using the improved Adaptive AMCL algorithm, mobile robots can be positioned based on a constructed map and auxiliary positioning corrections using wheel odometry and inertial odometry. Adaptive AMCL employs a specially designed adaptive particle filter, adaptively weighting the filter based on the degree of drift. When the error between the wheel odometry and inertial odometry exceeds 5%, the inertial odometry data is used with a weight of 0.6. It also integrates the lidar point cloud topic (scan) and the odom coordinate system to base_link coordinate system transformation described below, primarily publishing the base_link coordinate system to map coordinate system coordinate transformation to the TF tree. Specifically, the newly added odom_frame to map_frame transformation calculates accumulated drift, offsetting the accumulated error caused by slippage in real-world situations to improve final positioning accuracy. Compared to the ORB-SLAM algorithm, positioning accuracy is improved by nearly 5%.

[0041] The core parts of this system architecture are mapping, positioning and path planning, which correspond to Figure 3 The map construction node (Gmapping), matching positioning node (AMCL) and path planning control node (Move_base) nodes in.

[0042] The implementation of the mapping algorithm focuses on the extended Kalman filter positioning (robot_pose_ekf) node. The extended Kalman filter positioning node uses the extended Kalman filter to fuse the drive wheel positioning topic (odom), the inertial odometer topic (imu_data), and the encoder positioning topic (vo) to achieve a more accurate odometer prior. Specifically:

[0043] (1.1) Prediction step: Based on the state and control input (from / odom) of the previous time step, predict the state of the current time step I. The control input comes from the driving wheel positioning topic odom;

[0044] (1.2) Update the state covariance matrix to reflect the uncertainty of the state;

[0045] (2.1) Update step: Update state II using sensor observation data, which comes from the inertial odometry topic imu_data and the encoder positioning topic vo;

[0046] (2.2) Calculate the residuals of the observed state I and the predicted state II, and use the Kalman gain to adjust the state and covariance matrix.

[0047] Finally, the transformation from the odom coordinate system to the base_link coordinate system is published. In our actual robot architecture, the / odom topic refers to the odometry data calculated using open-source algorithms for the four-wheel Mecanum wheels and odometry data generated using the measuring wheels and inertial navigation. We also distinguish between the odom coordinate system and the / odom topic; the former is the coordinate system, while the latter is the odometry data calculated using encoders (or vision, etc.).

[0048] like Figure 4 As shown in the figure, the / map to / odom coordinate transformation is a new coordinate system transformation designed to reduce odometry drift. This means that when the system is initially started, the odometry drift is zero, and the / map coordinate system and the / odom coordinate system coincide. However, as the robot continues to move and slips, the drift increases. This is because the odometry is based on relative positioning and has cumulative errors. Therefore, the improved Adaptive AMCL particle filter algorithm is used to process the lidar data to correct the cumulative error from / map to / odom.

[0049] In addition, expanding the Kalman filter positioning node is also an important improvement to improve accuracy. Based on the Bayesian filter, the Kalman filter uses Gaussian distribution to describe the state quantity, so that only the mean and variance need to be iteratively calculated to fully describe the robot state. Therefore, this requires that the state at any time obeys the Gaussian distribution. For this reason, three assumptions are made in the Kalman filter to construct a linear Gaussian system. Unfortunately, most actual systems are nonlinear. For example, we use three quantities to describe the robot's posture state in a plane;

[0050] Trigonometric functions are likely to appear in its motion equations and observation equations, which makes it nonlinear. The result of Gaussian distribution in nonlinear system will no longer be Gaussian distribution, such as Figure 5 As shown in the figure. Here, x follows a Gaussian distribution, and y = g(x) is a nonlinear function. The resulting distribution of y is severely distorted (the distribution of y here is obtained through Monte Carlo sampling). Calculating the mean and variance of y also yields the Gaussian function curve shown in the figure, but this seriously deviates from the actual distribution. Therefore, the Kalman filter's assumptions are often not met when describing the robot's state. To maintain the use of the Kalman filter, we employ methods such as linearizing nonlinear systems to expand its scope.

[0051] The idea of ​​Kalman filtering is actually very simple. Since the system is nonlinear and not a "straight line", we use the "tangent" approximation. We should use the tangent at the point with the largest possible value in the original distribution, that is, the tangent at the mean, such as Figure 6 As shown in Figure 2, the result (dashed line) is close to the mean-variance of the actual result. Therefore, the nonlinear equations of motion and observation equations must be linearized by replacing them with tangents. This is essentially a first-order Taylor expansion at the mean.

[0052] Equations of motion:

[0053]

[0054] Among them, u t represents the robot control input at time t, x t-1 represents the robot position at time t-1, μ t-1 Represents the true value of the sensor at time t-1;

[0055] Observation equation:

[0056]

[0057] Among them, x t represents the robot position at time t, Represents the measurement value collected by the sensor at time t.

[0058] Therefore, the extended Kalman filter can integrate mileage information from wheel speedometers, inertial navigation odometry, and visual odometry to achieve more accurate positioning and reduce cumulative error. In actual testing, the accuracy comparison with ORB-SLAM is shown in the table below: RMSE stands for "Root Mean Squared Error," and accuracy improved by 5%-10% in different scenarios.

[0059] RMSE (cm) ORB-SLAM Ours scene1 5.32 4.82 scene2 2.33 1.98 scene3 6.48 5.88

[0060] In summary, the positioning and mapping accuracy of the robot based on the Mecanum wheeled chassis of the present invention is significantly improved.

Claims

1. A robot environment perception and positioning system based on ROS, characterized in that: include: Hardware: This includes a chassis frame, motor, Mecanum wheels, inertial sensors, an STM32 microcontroller, a NUC11 industrial computer, a Silam A2 LiDAR, and an industrial camera. The NUC11 industrial computer receives sensor data from the inertial sensor, LiDAR, and industrial camera and passes it to the corresponding algorithm module for processing. After sensing the surrounding environment, a grid map is created. The speed and direction are then sent to the STM32 microcontroller via serial communication. The STM32 microcontroller calculates the speed and direction and converts them into current values. This current value is then sent to the motor via CAN communication, driving the Mecanum wheels to move omnidirectionally, achieving autonomous positioning and omnidirectional navigation of the robot. Software modules: including IMU drive node, camera drive node, lidar drive node, cost map node, positioning node, communication node, path planning node, and decision node; The positioning node receives and processes the sensor information sent by the IMU drive node, camera drive node, and lidar drive node to determine their respective positions; At the same time, the decision node determines the target location; The path planning node dynamically plans the optimal path in real time based on the cost map node, and sends the information to the STM32 microcontroller to drive the motor movement through the communication node.

2. A robot environment perception and positioning system based on ROS according to claim 1, characterized in that: The hardware device includes a system architecture module, which is implemented as follows; The NUC11 industrial computer receives sensor data from inertial sensors, lidars, and industrial cameras and passes it to the corresponding algorithm modules for processing: the velocity and acceleration variables collected by the inertial sensors are processed by the mean filtering algorithm; the point cloud collected by the lidar is denoised and dedistorted; and the images collected by the industrial camera are dedistorted.

3. A robot environment perception and positioning system based on ROS according to claim 1, characterized in that: The hardware device includes a chassis architecture module, which is implemented as follows; The chassis, the main structure of the wheeled robot, is equipped with motion actuators and sensors. To achieve omnidirectional mobility, it uses four Mecanum wheels. Wheel A is left-handed and wheel B is right-handed, using an O-rectangle mounting method. The Mecanum wheel hub axis is at a 45-degree angle to the roller axis. The chassis is equipped with a variety of sensors, so it uses terrain-adaptive suspension.

4. The ROS-based robot environment perception and positioning system according to claim 1, characterized in that: The hardware device includes a sensor module, which is implemented as follows; The point cloud collected by the LiDAR is denoised and dedistorted. The specific steps are as follows: ① For each point, the average distance and standard deviation of its neighborhood are calculated; ② A threshold is set. If the average distance of a point's neighborhood exceeds this threshold, the point is considered a noise point and removed; The images captured by the industrial camera are dedistorted by calculating the correction transformation matrix using the camera calibration parameters and applying the correction transformation to the captured image.

5. The ROS-based robot environment perception and positioning system according to claim 1, characterized in that: The hardware device includes an algorithm module, which is implemented as follows; Step 1: Map construction node; Step 2: Match and locate nodes; Step 3: Path planning control node.

6. A robot environment perception and positioning system based on ROS according to claim 5, characterized in that: Step 1: Use the open source Gmapping algorithm to build the map.

7. A robot environment perception and positioning system based on ROS according to claim 6, characterized in that: Step 2 is implemented as follows: The improved Adaptive AMCL algorithm is used to achieve the positioning of the mobile robot based on the constructed map and auxiliary positioning correction based on the wheel odometer and inertial odometer. The improved Adaptive AMCL algorithm uses a designed adaptive particle filter and adaptive weighted filtering based on the degree of drift. When the error between the wheel odometer and the inertial odometer is greater than 5%, the inertial odometer data is used with a weight of 0.

6. Fuse the lidar point cloud topic scan, transform the odom coordinate system to the base_link coordinate system, and publish the coordinate transformation from the base_link coordinate system to the map coordinate system to the tf tree.

8. A robot environment perception and positioning system based on ROS according to claim 7, characterized in that: The core of the map construction is implemented through the extended Kalman filter positioning node. The extended Kalman filter positioning node uses the extended Kalman filter to fuse the driving wheel positioning topic odom, the inertial odometer topic imu_data, and the encoder positioning topic vo to implement the odometer prior. Specifically: (1.1) Prediction step: Based on the state and control input (from / odom) of the previous time step, predict the state of the current time step I. The control input comes from the driving wheel positioning topic odom; (1.2) Update the state covariance matrix to reflect the uncertainty of the state; (2.1) Update step: Update state II using sensor observation data, which comes from the inertial odometry topic imu_data and the encoder positioning topic vo; (2.2) Calculate the residuals of the observed state I and the predicted state II, and use the Kalman gain to adjust the state and covariance matrix.

Images