Method for dynamic obstacle multi-feature description in indoor-outdoor alternating environment
By combining multi-sensor fusion technology of 3D LiDAR, depth camera, IMU and GNSS, dynamic obstacle multi-feature description in changing indoor and outdoor environments is realized, which solves the problem of inaccurate detection in traditional methods and improves detection accuracy and data support capabilities.
Patent Information
- Application Number
- CN202211603609.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-13
- Publication Date
- 2025-12-19
- Estimated Expiration
- 2042-12-13
AI Technical Summary
Traditional dynamic obstacle detection methods lack clear detection mechanisms in changing indoor and outdoor environments, and the multi-sensor signal fusion methods are not specific, resulting in low detection accuracy and efficiency, which cannot meet the high requirements of intelligent mobile robots.
By combining 3D LiDAR, depth camera, IMU and GNSS, a global map is built using the LiDAR SLAM method. Combined with Euclidean clustering, minimax method, Kalman filter algorithm and other technologies, the shape, position and velocity features of dynamic obstacles are distinguished and described, realizing multi-feature fusion description.
This technology enables the effective differentiation between dynamic and static obstacles in changing indoor and outdoor environments, accurately describes the physical and motion characteristics of dynamic obstacles, improves detection accuracy, solves the problem of high false detection rate, and provides reliable data support for obstacle avoidance algorithms.
Smart Images

Figure CN115930946B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of obstacle detection, in particular to a method for multi-feature description of dynamic obstacles in indoor and outdoor alternating environments. BACKGROUND
[0002] With the continuous progress of robot technology, the application of intelligent mobile robots is also more and more widely, and the working environment of mobile robots is relatively complex. In order to ensure the safety of its work, the requirement for the perception of the surrounding environment and dynamic obstacle avoidance of mobile robots is also higher and higher. Therefore, the rapid detection of obstacles, the accurate description of geometric features and motion features are particularly important.
[0003] For example, the dynamic obstacle detection method based on a single sensor clusters data and extracts features, but it cannot obtain sufficient external environment information. The traditional obstacle detection method based on different sensors can present the shape features of moving objects and use them as the motion model of the object, but the false detection rate is high and it is powerless for the detection of fast-moving obstacles.
[0004] Therefore, the traditional dynamic obstacle detection and its description method has the problems of unclear detection mechanism and non-specific multi-sensor signal fusion method, which cannot meet the increasing demand of intelligent mobile robots. SUMMARY
[0005] The purpose of the present application is to overcome the shortcomings of the prior art, such as unclear detection mechanism and non-specific multi-sensor signal fusion method in dynamic obstacle detection and description method, and provide a method for multi-feature description of dynamic obstacles in indoor and outdoor alternating environments.
[0006] In order to achieve the above purpose, the present application adopts the following technical scheme:
[0007] A method for multi-feature description of dynamic obstacles in indoor and outdoor alternating environments, a robot is provided with a three-dimensional laser radar, a depth camera, an IMU and a GNSS, and the processor of the robot is connected with the three-dimensional laser radar, the depth camera, the IMU and the GNSS data respectively;
[0008] The method comprises the following steps:
[0009] Step 1, the three-dimensional laser radar, the depth camera, the IMU and the GNSS detect the environmental data information around the robot, and the processor uses the laser SLAM method to establish a global map in the indoor and outdoor alternating environment, and the processor establishes a semantic map;
[0010] Step 2, if the robot receives valid GNSS data, the processor makes a judgment that the robot is in an outdoor environment;
[0011] Step 2-1, the processor pre-processes the point cloud of the data output by the three-dimensional laser radar: downsamples the original point cloud using a Voxel Grid Filter, separates the ground points from the non-ground points in the point cloud, and clusters the pre-processed point cloud using Euclidean clustering to obtain a plurality of point cloud clusters, the point cloud clusters being obstacle point cloud clusters;
[0012] Step 2-2, the processor projects each obstacle point cloud cluster into a grid map using a maximum and minimum value method: a threshold A is set, when the absolute value of the difference between the maximum height value and the minimum height value of any grid i is greater than A, the processor marks the grid i as an occupied state, otherwise as a non-occupied state;
[0013] An obstacle point cloud cluster projected into a grid map corresponds to an obstacle block, and the processor calculates the occupancy rate of the obstacle grid in the obstacle block;
[0014] Step 2-3, the processor calculates the edge of each obstacle point cloud cluster on the two-dimensional grid map, and fits the corner and edge features of the obstacle block corresponding to the obstacle point cloud cluster;
[0015] Step 2-4, the processor tracks the obstacles and distinguishes between static obstacles and dynamic obstacles;
[0016] Step 2-4-1, for dynamic obstacles, a dynamic obstacle list O is established:
[0017]
[0018] where (x obstacle (t),y obstacle (t)) represents the coordinates of the center position of the obstacle, and represent the speed and acceleration of the obstacle respectively, θ obstacle (t) represents the angle between the long axis of the obstacle and the y-axis of the global map Cartesian coordinate system,
[0019] 2-4-2, for obstacles with obvious edge and corner features, the processor uses a square model to represent the obstacle;
[0020] 2-4-3, for obstacles without obvious edge and corner features, the processor uses an elliptical model to represent the obstacle; after completing the current frame data processing, each obstacle block is obtained;
[0021] Step 2-5, the correlation value between the obstacle blocks is established, the obstacle with the largest correlation value in the current frame and the previous frame is matched together to obtain a final map;
[0022] Step 2-6, the processor obtains the latitude and longitude and the heading angle through GNSS data, projects the latitude and longitude coordinates on the plane coordinates of the final map, obtains the plane coordinates and the heading angle of the robot between continuous frames, calculates the relative motion speed of the dynamic obstacle and the robot, updates the speed, the acceleration, the speed covariance and the acceleration covariance of the obstacle according to the Kalman filtering algorithm, and establishes the dynamic obstacle model in the semantic map by using the square model, the elliptical model and the motion characteristics of each obstacle block.
[0023] Step 3, if the robot does not receive valid GNSS data, it is judged that the robot is in the indoor environment:
[0024] Repeat step 2-1, the processor processes the data output by the three-dimensional laser radar to obtain a plurality of obstacle point cloud clusters, and repeats steps 2-2 to 2-4 to fit the corner and edge features of the obstacle, and uses the square model and the elliptical model to represent each obstacle block; each indoor obstacle is extracted by using a depth camera, and each obstacle block in the current frame is matched with each obstacle in the previous frame by repeating step 2-5.
[0025] The depth camera detects each indoor obstacle, and the processor obtains the average value of the RGB of each obstacle image pixel.
[0026] The IMU is used to obtain the linear speed and angular speed of the robot, and the relative motion speed of the dynamic obstacle and the robot is calculated; the final obstacle model is formed by using each matched obstacle block, the obstacle model is substituted into the Kalman filter, the motion state of the dynamic obstacle is predicted, and a multi-feature fusion dynamic obstacle model is established.
[0027] The intelligent mobile robot in the application integrates perception, decision-making and control, and can detect the shape, size, position, speed, acceleration, speed covariance and acceleration covariance of the dynamic obstacle.
[0028] The application extracts environmental information in indoor and outdoor environments, scans the surrounding environment through multiple source sensors, counts the data of obstacle frequency changes and associates continuous frame data to distinguish dynamic obstacles and static obstacles. The motion state of the dynamic obstacle is fitted and tracked, and a dynamic obstacle model is established in the semantic map and is clustered and segmented.
[0029] The application can realize effective distinction of dynamic obstacles and static obstacles in indoor and outdoor alternating environments, and can accurately describe physical characteristics and motion characteristics of the dynamic obstacles. The application solves the problem of key information loss of the dynamic obstacles in detection, and compared with a traditional method, the application describes dynamic obstacle characteristics from multiple dimensions, and solves a safety problem caused by inaccurate description of the dynamic obstacles when avoiding the dynamic obstacles.
[0030] The application extracts dynamic obstacle information in multi-source signals, proposes a method for rapidly detecting and judging dynamic obstacles in different areas such as indoors and outdoors, realizes accurate detection and tracking of the dynamic obstacles, and provides data support for subsequent obstacle avoidance algorithm path planning.
[0031] As preferred, step 2-4-2 includes the following steps:
[0032] Supposing that the length and width of the obstacle of the rectangle model are 2a obstacle and 2b obstacle , the processor represents points (X obstacle , Y obstacle ) on the edge of the obstacle as follows:
[0033] Left side: X obstacle cosθ obstacle -Y obstacle sinθ obstacle =-a obstacle ;
[0034] Right side: X obstacle cosθ obstacle -Y obstacle sinθ obstacle =a obstacle ;
[0035] Upper side: X obstacle sinθ obstacle +Y obstacle cosθ obstacle =-b obstacle ;
[0036] Lower side: X obstacle sinθ obstacle +Y obstacle cosθ obstacle =b obstacle .
[0037] As preferred, step 2-4-3 includes the following steps:
[0038] Supposing that the length of the long axis and the length of the short axis of the obstacle of the ellipse model are a obstacle and b obstacle respectively, the processor represents points (X obstacle , Yobstacle ) is expressed as:
[0039]
[0040] wherein (x obstacle ,y obstacle ) represents the coordinates of the center position of the obstacle.
[0041] As preferred, the step 2-5 comprises the following specific steps:
[0042] The correlation value between the obstacle blocks is established, and the correlation value of the square model obstacle block is the absolute value of the difference of the length of any two obstacle blocks between adjacent frames, plus the absolute value of the difference of the width of the two obstacle blocks, plus the distance of the center point coordinates of the two obstacle blocks, plus the absolute value of the difference of the occupancy of the obstacle grids in the two obstacle blocks;
[0043] The correlation value of the ellipse model obstacle block is the absolute value of the difference of the major axis of any two obstacle blocks between adjacent frames, plus the absolute value of the difference of the minor axis of the two obstacle blocks, plus the distance of the center point coordinates of the two obstacle blocks, plus the absolute value of the difference of the occupancy of the obstacle grids in the two obstacle blocks;
[0044] After obtaining the latest clustering result, each obstacle block Q of the current frame is compared with each obstacle block stored in the previous frame one by one, to obtain the correlation value of the obstacle block Q and each obstacle block of the previous frame, and the obstacle block B with the maximum correlation value in the previous frame is found, and the correlation value of the obstacle block Q and the obstacle block B is E 最大 , and the minimum correlation value is preset as C, and when E 最大 ≥ C, it indicates that the obstacle block Q and the obstacle block B are matched successfully.
[0045] The initial existence confidence of the obstacle block Q is set as 5, and if the obstacle block Q is not matched successfully with each obstacle block in the previous frame, the existence confidence of the obstacle block Q is reduced by 1.
[0046] If the obstacle block Q is matched successfully with the obstacle block in the previous frame, the existence confidence of the obstacle block Q is increased by 1.
[0047] The maximum existence confidence N 最大 , the minimum existence confidence N 最小 and the motion confidence G of the obstacle block Q are set, when the existence confidence of the obstacle block Q is < N 最小 , the obstacle block Q is deleted from the dynamic obstacle list; when the existence confidence of the obstacle block Q is > N 最大 , the existence confidence of the obstacle block Q is set as N 最大 .
[0048] When the speed of the obstacle block Q is less than 0.5 m / s, the motion confidence of the obstacle block Q is set to 0, the obstacle block with the motion confidence of 0 is regarded as a static obstacle, the obstacle blocks outside the dynamic obstacle list are all regarded as static obstacles, a static obstacle grid map is established for the static obstacles, the static obstacle grid map is fused into the dynamic obstacle grid map, and finally the map is obtained.
[0049] Preferably, when the robot is in the indoor environment, the correlation value of the square model obstacle block in step 2-5 is the absolute value of the difference between the lengths of any two obstacle blocks in adjacent frames, plus the absolute value of the difference between the widths of the two obstacle blocks, plus the distance between the center point coordinates of the two obstacle blocks, plus the absolute value of the difference between the occupancy rates of the obstacle grids in the two obstacle blocks, plus the average RGB value of the obstacle image pixel points.
[0050] The correlation value of the ellipse model obstacle block is the absolute value of the difference between the major axes of any two obstacle blocks in adjacent frames, plus the absolute value of the difference between the minor axes of the two obstacle blocks, plus the distance between the center point coordinates of the two obstacle blocks, plus the absolute value of the difference between the occupancy rates of the obstacle grids in the two obstacle blocks, plus the average RGB value of the obstacle image pixel points.
[0051] Therefore, the present application has the following beneficial effects:
[0052] The present application has a wide range of applications, can obtain sufficient environmental information, improves the detection accuracy, and solves the problem of high false detection rate caused by complex environment and insufficient information acquisition.
[0053] Different sensors and their fusion methods are used indoors and outdoors to quickly detect and judge dynamic obstacles, and provide reliable data basis for accurate tracking of dynamic obstacles. BRIEF DESCRIPTION OF DRAWINGS
[0054] Figure 1 is a flowchart of the present application;
[0055] Figure 2 is a schematic diagram of the robot and obstacle model of the present application. DETAILED DESCRIPTION
[0056] The present application will be further described below in combination with the drawings and specific embodiments.
[0057] As shown in the embodiments shown in Figure 1 , Figure 2 , the present application is a method for multi-feature description of dynamic obstacles in indoor and outdoor alternating environments, a robot is provided with a three-dimensional laser radar, a depth camera, an IMU inertial measurement unit and a GNSS global navigation microsystem, the processor of the robot is connected with the three-dimensional laser radar, the depth camera, the IMU and the GNSS data respectively.
[0058] comprising the steps of:
[0059] Step 1, the three-dimensional laser radar, depth camera, IMU and GNSS detect the environmental data information around the robot, and the processor uses a laser SLAM method to establish a global map in an indoor-outdoor alternating environment, and the processor establishes a semantic map;
[0060] Step 2, if the robot receives valid GNSS data, the processor makes a judgment that the robot is in an outdoor environment;
[0061] Step 2-1, the processor pre-processes the point cloud of the data output by the three-dimensional laser radar: uses a Voxel Grid Filter to down-sample the original point cloud, separates the ground points from the non-ground points in the point cloud, and uses Euclidean clustering to cluster the pre-processed point cloud to obtain a plurality of point cloud clusters, the point cloud clusters being obstacle point cloud clusters;
[0062] Step 2-2, the processor projects each obstacle point cloud cluster into a grid map using a maximum and minimum value method: sets a threshold A, when the absolute value of the difference between the maximum height value and the minimum height value in any grid i is greater than A, the processor marks the grid i as an occupied state, otherwise as a non-occupied state;
[0063] An obstacle point cloud cluster projected into the grid map corresponds to an obstacle block, and the processor calculates the occupancy rate of the obstacle grid in the obstacle block;
[0064] Step 2-3, the processor calculates the edge of each obstacle point cloud cluster on the two-dimensional grid map, and fits the corner and edge features of the obstacle block corresponding to the obstacle point cloud cluster;
[0065] Step 2-4, the processor tracks the obstacles and distinguishes static obstacles and dynamic obstacles;
[0066] Step 2-4-1, for dynamic obstacles, a dynamic obstacle list O is established:
[0067]
[0068] wherein (x obstacle (t),y obstacle (t)) represents the coordinates of the center position of the obstacle, and represent the speed and acceleration of the obstacle respectively, θ obstacle (t) represents the angle between the long axis of the obstacle and the y-axis of the Cartesian coordinate system of the global map,
[0069] 2-4-2, for the obstacle with obvious edge and corner features, the processor uses square model to represent the obstacle;
[0070] Let the length and width of the square model obstacle be: 2a obstacle and 2b obstacle , the processor represents the equation of the point (X obstacle , Y obstacle ) on the edge of the obstacle as:
[0071] Left side: X obstacle cosθ obstacle -Y obstacle sinθ obstacle =-a obstacle ;
[0072] Right side: X obstacle cosθ obstacle -Y obstacle sinθ obstacle =a obstacle ;
[0073] Upper side: X obstacle sinθ obstacle +Y obstacle cosθ obstacle =-b obstacle ;
[0074] Lower side: X obstacle sinθ obstacle +Y obstacle cosθ obstacle =b obstacle .
[0075] 2-4-3, for the obstacle without obvious edge and corner features, the processor uses ellipse model to represent the obstacle; after completing the processing of the current frame data, the processor obtains each obstacle block;
[0076] Let the length of the major axis and the minor axis of the ellipse model obstacle be: a obstacle and b obstacle ; the processor represents the equation of the point (X obstacle , Y obstacle ) as:
[0077]
[0078] Wherein, (x obstacle , y obstacle ) represents the coordinate of the center position of the obstacle.
[0079] Step 2-5, establish the correlation value between the obstacle blocks, and match the obstacle with the largest correlation value in the current frame and the previous frame together;
[0080] The correlation value between the obstacle blocks is established, and the correlation value of the polygon model obstacle block is the absolute value of the difference of the lengths of any two obstacle blocks between adjacent frames, plus the absolute value of the difference of the widths of the two obstacle blocks, plus the distance of the center point coordinates of the two obstacle blocks, plus the absolute value of the difference of the occupancy of the obstacle grids in the two obstacle blocks;
[0081] The correlation value of the ellipse model obstacle block is the absolute value of the difference of the major axes of any two obstacle blocks between adjacent frames, plus the absolute value of the difference of the minor axes of the two obstacle blocks, plus the distance of the center point coordinates of the two obstacle blocks, plus the absolute value of the difference of the occupancy of the obstacle grids in the two obstacle blocks;
[0082] After obtaining the latest clustering result, each obstacle block Q of the current frame is compared with each obstacle block stored in the previous frame one by one, the correlation value of the obstacle block Q and each obstacle block of the previous frame is obtained, the obstacle block B with the maximum correlation value in the previous frame is found, and the correlation value of the obstacle block Q and the obstacle block B is E 最大 , and the minimum correlation value is set as C. When E 最大 ≥ C, it indicates that the obstacle block Q and the obstacle block B are matched successfully.
[0083] The initial existence confidence of the obstacle block Q is set as 5. If the obstacle block Q is not matched successfully with each obstacle block in the previous frame, the existence confidence of the obstacle block Q is reduced by 1.
[0084] If the obstacle block Q is matched successfully with the obstacle block in the previous frame, the existence confidence of the obstacle block Q is increased by 1.
[0085] The maximum existence confidence N 最大 , the minimum existence confidence N 最小 and the motion confidence G of the obstacle block Q are set. When the existence confidence of the obstacle block Q is less than N 最小 , the obstacle block Q is deleted from the dynamic obstacle list. When the existence confidence of the obstacle block Q is greater than N 最大 , the existence confidence of the obstacle block Q is set as N 最大 .
[0086] When the speed of the obstacle block Q is less than 0.5 m / s, the motion confidence of the obstacle block Q is set as 0, and the obstacle block with the motion confidence of 0 is regarded as a static obstacle. The obstacle blocks outside the dynamic obstacle list are all regarded as static obstacles. The static obstacle grid map is established for the static obstacles, the static obstacle grid map is fused into the dynamic obstacle grid map, and the final map is obtained.
[0087] Step 2-6, the processor obtains the latitude and longitude and the heading angle through the GNSS data, projects the latitude and longitude coordinates to the plane coordinates of the final map, obtains the plane coordinates and the heading angle of the robot between the continuous frames, calculates the relative motion speed of the dynamic obstacle and the robot, updates the speed, the acceleration, the speed covariance and the acceleration covariance of the obstacle according to the Kalman filtering algorithm, and establishes the dynamic obstacle model in the semantic map by using the square model, the elliptical model and the motion characteristics of each obstacle block;
[0088] Step 3, if the robot does not receive valid GNSS data, it is judged that the robot is in the indoor environment:
[0089] Repeat step 2-1, the processor processes the data output by the three-dimensional laser radar to obtain a plurality of obstacle point cloud clusters, repeat steps 2-2 to 2-4 to fit the corner and edge features of the obstacle, and use the square model and the elliptical model to represent each obstacle block; use the depth camera to extract the features of each obstacle in each indoor environment, repeat step 2-5, add the average value of the RGB of the obstacle image pixel points to the correlation value, and match each obstacle block in the current frame with each obstacle in the previous frame;
[0090] The IMU is used to obtain the linear speed and angular speed of the robot, the relative motion speed of the dynamic obstacle and the robot is calculated, the final obstacle model is formed by using each matched obstacle block, the obstacle model is substituted into the Kalman filter, the motion state of the dynamic obstacle is predicted, and a multi-feature fusion dynamic obstacle model is established.
[0091] When the robot is in the indoor environment, the correlation value of the square model obstacle block in step 2-5 is the absolute value of the difference between the lengths of any two obstacle blocks between adjacent frames, plus the absolute value of the difference between the widths of the two obstacle blocks, plus the distance between the center point coordinates of the two obstacle blocks, plus the absolute value of the difference between the occupancy rates of the obstacle grids in the two obstacle blocks, plus the average value of the RGB of the obstacle image pixel points.
[0092] The correlation value of the elliptical model obstacle block is the absolute value of the difference between the major axes of any two obstacle blocks between adjacent frames, plus the absolute value of the difference between the minor axes of the two obstacle blocks, plus the distance between the center point coordinates of the two obstacle blocks, plus the absolute value of the difference between the occupancy rates of the obstacle grids in the two obstacle blocks, plus the average value of the RGB of the obstacle image pixel points.
[0093] The above only describes the preferred embodiments of the present application and is not intended to limit the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. A method for multi-feature description of dynamic obstacles in indoor and outdoor changing environments, characterized in that, The robot is equipped with a 3D LiDAR, depth camera, IMU, and GNSS. The robot's processor is connected to the 3D LiDAR, depth camera, IMU, and GNSS data respectively. Includes the following steps: Step 1: The 3D LiDAR, depth camera, IMU and GNSS detect the environmental data information around the robot. The processor uses the laser SLAM method to build a global map of the indoor and outdoor changing environment. The processor builds a semantic map. Step 2: If the robot receives valid GNSS data, the processor determines that the robot is in an outdoor environment. Step 2-1: The processor preprocesses the point cloud data output by the 3D LiDAR: Voxel GridFilter is used to downsample the original point cloud to separate ground points from non-ground points in the point cloud; Euclidean clustering is used to cluster the preprocessed point cloud to obtain a set of multiple point clouds, which is the obstacle point cloud cluster. Step 2-2: The processor uses the maximum and minimum value method to project each obstacle point cloud cluster onto the grid map: Set a threshold A. When the absolute value of the difference between the maximum height value and the minimum height value on any grid i is greater than A, the processor marks grid i as occupied; otherwise, it is unoccupied. An obstacle point cloud cluster projected onto the grid map corresponds to an obstacle block, and the processor calculates the occupancy rate of the obstacle grid within the obstacle block; Steps 2-3: The processor calculates the edge of each obstacle point cloud cluster on the two-dimensional grid map and fits the corner and edge features of the obstacle block corresponding to the obstacle point cloud cluster. Steps 2-4: The processor tracks obstacles and distinguishes between static and dynamic obstacles; Step 2-4-1, for dynamic obstacles, create a dynamic obstacle list O: Among them, (x obstacle (t),y obstacle (t) represents the coordinates of the center position of the obstacle. and Let θ represent the velocity and acceleration of the obstacle, respectively. obstacle (t) represents the angle between the major axis of the obstacle and the y-axis of the Cartesian coordinate system of the global map. 2-4-2, For obstacles with obvious edge and corner features, the processor uses a square model to represent the obstacle; let the length and width of the obstacle in the square model be, and the processor represents the points on the edge of the obstacle with equations; 2-4-3, For obstacles without obvious edge or corner features, the processor uses an elliptical model to represent the obstacle; after completing the processing of the current frame data, each obstacle block is obtained; Steps 2-5: Establish the association values between obstacle blocks, and match the obstacle with the largest association value in the current frame and the previous frame together to obtain the final map; Steps 2-6: The processor obtains latitude, longitude and heading angle through GNSS data, projects the latitude and longitude coordinates onto the plane coordinates of the final map, obtains the plane coordinates and heading angle of the robot between consecutive frames, calculates the relative motion speed between dynamic obstacles and the robot, updates the speed, acceleration, speed covariance and acceleration covariance of obstacles according to the Kalman filter algorithm, and establishes a dynamic obstacle model in the semantic map using the square model, elliptical model and motion features of each obstacle block. Step 3: If the robot does not receive valid GNSS data, then determine that the robot is indoors. Repeat step 2-1, the processor processes the data output by the 3D LiDAR to obtain multiple obstacle point cloud clusters, repeat steps 2-2 to 2-4 to fit the corner and edge features of the obstacles, and represent each obstacle block with square and elliptical models; use a depth camera to extract features from each obstacle in each room, repeat step 2-5, add the RGB average value of the obstacle image pixels to the correlation value, and match each obstacle block in the current frame with each obstacle in the previous frame; The linear velocity and angular velocity of the robot are obtained using an IMU, and the relative motion velocity between the dynamic obstacle and the robot is calculated. The final obstacle model is formed by using the matched obstacle blocks. The obstacle model is then fed into a Kalman filter to predict the motion state of the dynamic obstacle and establish a multi-feature fusion dynamic obstacle model.
2. The method for multi-feature description of dynamic obstacles in indoor and outdoor changing environments according to claim 1, characterized in that, Step 2-4-2 includes the following steps: Let the length and width of the obstacle in the square model be 2a. obstacle and 2b obstacle The processor will select the point (X) on the edge of the obstacle. obstacle Y obstacle The equation for is expressed as: Left side: X obstacle cosθ obstacle -Y obstacle sinθ obstacle =-a obstacle ; Right: X obstacle cosθ obstacle -Y obstacle sinth obstacle =a obstacle ; Upper side: X obstacle sinθ obstacle +Y obstacle cosθ obstacle =-b obstacle ; Below: X obstacle sinθ obstacle +Y obstacle cosθ obstacle =b obstacle .
3. The method for multi-feature description of dynamic obstacles in indoor and outdoor changing environments according to claim 1, characterized in that, Step 2-4-3 includes the following steps: Let the lengths of the major and minor axes of the obstacle in the elliptical model be a, a, and a, respectively. obstacle and b obstacle The processor will point (X) obstacle Y obstacle The equation for is expressed as: Among them, (x obstacle ,y obstacle ) represents the coordinates of the center position of the obstacle.
4. The method for multi-feature description of dynamic obstacles in indoor and outdoor changing environments according to claim 1, 2, or 3, characterized in that, Steps 2-5 include the following specific steps: Establish the association value between obstacle blocks. The association value of the obstacle block in the square model is the absolute value of the difference between the lengths of any two obstacle blocks between adjacent frames, plus the absolute value of the difference between the widths of the two obstacle blocks, plus the distance between the center point coordinates of the two obstacle blocks, plus the absolute value of the difference between the occupancy rates of the obstacle grids in the two obstacle blocks. The associated value of an obstacle block in an elliptical model is the absolute value of the difference between the major axes of any two obstacle blocks between adjacent frames, plus the absolute value of the difference between the minor axes of the two obstacle blocks, plus the distance between the center point coordinates of the two obstacle blocks, plus the absolute value of the difference between the occupancy rates of the obstacle grids in the two obstacle blocks. After obtaining the latest clustering result, each obstacle block Q in the current frame is compared with each obstacle block stored in the previous frame to obtain the association value between obstacle block Q and each obstacle block in the previous frame. The obstacle block B with the largest association value in the previous frame is found, and the association value between obstacle block Q and obstacle block B is E. 最大 The minimum associated value is preset to C, when E 最大 If the value is greater than or equal to C, it means that obstacle block Q and obstacle block B are successfully matched. Set the initial existence confidence of obstacle block Q to 5; if obstacle block Q does not match any of the obstacle blocks in the previous frame, decrease the existence confidence of obstacle block Q by 1. If obstacle block Q matches an obstacle block in the previous frame, increment the confidence level of obstacle block Q by 1. Set the maximum confidence level N for the existence of obstacle block Q. 最大 Minimum existence confidence N 最小 And motion confidence G, when the confidence in the existence of obstacle block Q is < N 最小 When the time comes, remove obstacle block Q from the list of dynamic obstacles; When the confidence level of the existence of obstacle block Q is greater than N 最大 Make the confidence level of the existence of obstacle block Q = N 最大 ; When the velocity of obstacle block Q is less than 0.5 m / s, the motion confidence of obstacle block Q is set to 0. Obstacle blocks with a motion confidence of 0 are considered static obstacles. Obstacle blocks not in the dynamic obstacle list are all considered static obstacles. A static obstacle grid map is created for the static obstacles. The static obstacle grid map is then merged into the dynamic obstacle grid map to obtain the final map.
5. The method for multi-feature description of dynamic obstacles in indoor and outdoor changing environments according to claim 4, characterized in that, When the robot is indoors, the associated value of the obstacle block in the square model in steps 2-5 is the absolute value of the difference in length between any two obstacle blocks between adjacent frames, plus the absolute value of the difference in width between the two obstacle blocks, plus the distance between the center point coordinates of the two obstacle blocks, plus the absolute value of the difference in the occupancy rate of the obstacle grid in the two obstacle blocks, plus the RGB average value of the obstacle image pixels. The correlation value of the obstacle block in the elliptical model is the absolute value of the difference between the major axes of any two obstacle blocks between adjacent frames, plus the absolute value of the difference between the minor axes of the two obstacle blocks, plus the distance between the center point coordinates of the two obstacle blocks, plus the absolute value of the difference between the occupancy rates of the obstacle grid in the two obstacle blocks, plus the RGB average value of the obstacle image pixels.
Citation Information
Patent Citations
Dynamic obstacle tracking method based on sparse laser radar data
CN111337941A
Obstacle detection method and device based on three-dimensional radar, medium and robot
CN112529874A