Wheeled robot autonomous obstacle avoidance method and system based on fusion of multiple sensors, and medium

Abstract

The embodiment of the invention provides a wheeled robot autonomous obstacle avoidance method and system based on fusion of multiple sensors and a medium, and the method comprises the steps: setting a detection region, obtaining collection data in the detection region based on multiple sensors, and obtaining multi-source data; performing fusion processing on the multi-source data based on a fusion strategy, and analyzing barrier distribution information in the detection area; generating an obstacle avoidance strategy based on an obstacle avoidance algorithm, and generating wheeled robot path planning information based on the obstacle avoidance strategy; comparing the moving track information of the wheeled robot with the path planning information of the wheeled robot to obtain path difference information; analyzing the performance index of the obstacle avoidance strategy, and dynamically correcting the obstacle avoidance strategy and the wheeled robot path planning information; the multi-source data is acquired through multiple sensors, the multi-source data is subjected to fusion processing, and the obstacle avoidance strategy is generated, so that the wheeled robot can plan the obstacle avoidance path more quickly and accurately, unnecessary path adjustment is reduced, the obstacle avoidance time is saved, and the overall moving efficiency is improved.

Description

Technical Field

[0001] This application relates to the technical field of autonomous obstacle avoidance, and more particularly, to a method, system, and medium for autonomous obstacle avoidance of a wheeled robot based on multi-sensor fusion. Background Art

[0002] In a complex real-world environment, one of the key technologies for a wheeled robot to achieve autonomous movement is effective obstacle avoidance. A single sensor has limitations in environmental perception and is difficult to comprehensively and accurately obtain surrounding information, making it difficult to effectively avoid obstacles within a region. Existing autonomous obstacle avoidance methods cannot fuse multi-sensor data, making it difficult to leverage the advantages of each sensor, resulting in poor obstacle avoidance capabilities of the robot and affecting its safe and efficient operation. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a method, system, and medium for autonomous obstacle avoidance of a wheeled robot based on multi-sensor fusion. By acquiring multi-source data through multi-sensors, fusing and processing the multi-source data, and generating an obstacle avoidance strategy, the wheeled robot can more quickly and accurately plan an obstacle avoidance path, reduce unnecessary path adjustments, save obstacle avoidance time, and improve the overall movement efficiency.

[0004] The embodiments of this application also provide a method for autonomous obstacle avoidance of a wheeled robot based on multi-sensor fusion, including:

[0005] Set a detection area, and based on multi-sensors, acquire the collected data within the detection area to obtain multi-source data;

[0006] Based on a fusion strategy, fuse and process the multi-source data to obtain fusion data, and analyze the obstacle distribution information within the detection area according to the fusion data;

[0007] Based on an obstacle avoidance algorithm, dynamically analyze the obstacle distribution information to generate an obstacle avoidance strategy, and generate path planning information for the wheeled robot based on the obstacle avoidance strategy;

[0008] Obtain the movement trajectory information of the wheeled robot, compare the movement trajectory information of the wheeled robot with the path planning information of the wheeled robot to obtain path difference information;

[0009] Based on the path difference information, analyze the performance indicators of the obstacle avoidance strategy, based on the performance indicators of the obstacle avoidance strategy, analyze the obstacle avoidance status information of the wheeled robot, and dynamically correct the obstacle avoidance strategy and the path planning information of the wheeled robot based on the obstacle avoidance status information.

[0010] Optionally, in the method for autonomous obstacle avoidance of a wheeled robot based on multi-sensor fusion described in the embodiments of this application, setting a detection area and acquiring the collected data within the detection area based on multi-sensors to obtain multi-source data specifically includes:

[0011] Set a detection area, analyze the terrain and area of the detection area, and set at least one sensor of different types at different positions in the detection area based on the terrain and area. Different types of sensors include lidar, cameras, and ultrasonic sensors;

[0012] Obtain the parameter information of the lidar, obtain a number of discrete distance data points based on the lidar parameter information, and construct a three-dimensional point cloud map of the detection area based on the number of discrete distance data points to obtain three-dimensional point cloud data;

[0013] Based on the camera using the principle of optical imaging, focus the light in the detection area and convert it into a digital image signal. Identify the obstacle contour and features based on the digital image signal to obtain image data;

[0014] Based on the ultrasonic sensor using the piezoelectric effect, convert the electrical signal into ultrasonic waves, and receive the reflected wave based on the ultrasonic sensor to measure the time difference between transmission and reception;

[0015] Calculate the distance between the wheeled robot and the obstacle based on the time difference between transmission and reception to obtain ultrasonic data;

[0016] Obtain multi-source data based on the three-dimensional point cloud data, image data, and ultrasonic data.

[0017] Optionally, in the method for autonomous obstacle avoidance of a wheeled robot based on multi-sensor fusion described in the embodiments of the present application, fuse the multi-source data based on a fusion strategy to obtain fusion data, and analyze the obstacle distribution information in the detection area according to the fusion data. Specifically, it includes:

[0018] Construct a data layer fusion strategy, a feature layer fusion strategy, and a probability statistics fusion strategy based on the fusion strategy;

[0019] Perform coordinate transformation on the three-dimensional point cloud data and the image data based on the data layer fusion strategy, and convert the three-dimensional point cloud data and the image data to the same coordinate system for fusion to obtain data layer fusion information;

[0020] Extract data features from the three-dimensional point cloud data, image data, and ultrasonic data to obtain three-dimensional point cloud features, image features, and ultrasonic features. Based on the feature layer fusion strategy, fuse the three-dimensional point cloud features, image features, and ultrasonic features to obtain feature fusion information;

[0021] Analyze the probability distributions of the ultrasonic data and the three-dimensional point cloud data based on the probability statistics fusion strategy, and fuse the probability distributions of the ultrasonic data and the three-dimensional point cloud data to obtain probability statistics fusion information;

[0022] Fusion is performed based on the information fused at the data layer, the feature fusion information, and the probability statistical fusion information to obtain the fused data. Based on the fused data analysis, the obstacle position and the obstacle geometry are obtained to obtain the obstacle distribution information.

[0023] Optionally, in the method for autonomous obstacle avoidance of a wheeled robot based on multi-sensor fusion described in the embodiments of the present application, dynamic analysis is performed on the obstacle distribution information based on an obstacle avoidance algorithm to generate an obstacle avoidance strategy, and the path planning information of the wheeled robot is generated based on the obstacle avoidance strategy, specifically including:

[0024] Obtain the obstacle position information at different time nodes, compare the obstacle position information of adjacent time nodes of the same obstacle, and obtain the position difference value;

[0025] Determine whether the position difference value is zero;

[0026] If it is zero, it is determined that the current obstacle is a stationary obstacle, and the stationary obstacle distribution information is obtained based on all the stationary obstacles;

[0027] If it is not zero, it is determined that the current obstacle is a moving obstacle, obtain the moving direction and moving speed of the moving obstacle, and generate the moving trend information of the moving obstacle based on the moving parameters and moving speed of the moving obstacle;

[0028] Generate an obstacle avoidance strategy based on the stationary obstacle distribution information and the moving trend information of the moving obstacle, and generate the path planning information of the wheeled robot based on the obstacle avoidance strategy.

[0029] Optionally, in the method for autonomous obstacle avoidance of a wheeled robot based on multi-sensor fusion described in the embodiments of the present application, obtain the moving trajectory information of the wheeled robot, compare the moving trajectory information of the wheeled robot with the path planning information of the wheeled robot, and obtain the path difference information, specifically including:

[0030] Obtain the moving parameter information of the wheeled robot, and the moving parameter information of the wheeled robot includes the moving speed, moving acceleration, and moving direction of the wheeled robot;

[0031] Analyze the moving trajectory information of the wheeled robot based on the moving speed, moving acceleration, and moving direction of the wheeled robot;

[0032] Set multiple acquisition time nodes, and obtain multiple current position information of the wheeled robot based on the multiple acquisition time nodes;

[0033] Obtain the standard position information at the same acquisition time node based on the path planning information of the wheeled robot;

[0034] Compare the current position information with the standard position information, calculate the Euclidean distance, and analyze the path difference information based on the Euclidean distance.

[0035] Optionally, in the method for autonomous obstacle avoidance of a wheeled robot based on fused multi-sensors described in the embodiments of the present application, the performance metrics of the obstacle avoidance strategy are analyzed based on the path difference information, and the obstacle avoidance state information of the wheeled robot is analyzed based on the performance metrics of the obstacle avoidance strategy, specifically including:

[0036] Obtain the path difference information, compare the path difference information with the set condition information, and obtain a path difference value;

[0037] Compare the path difference value with the set difference interval to obtain the performance metrics of the obstacle avoidance strategy, where the performance metrics of the obstacle avoidance strategy include the obstacle avoidance success rate, path planning efficiency, and obstacle avoidance time;

[0038] Analyze the obstacle avoidance state information of the wheeled robot based on the obstacle avoidance success rate, path planning efficiency, and obstacle avoidance time.

[0039] In a second aspect, the embodiments of the present application provide a system for autonomous obstacle avoidance of a wheeled robot based on fused multi-sensors. The system includes: a memory and a processor. The memory includes a program for the method for autonomous obstacle avoidance of a wheeled robot based on fused multi-sensors. When the program for the method for autonomous obstacle avoidance of a wheeled robot based on fused multi-sensors is executed by the processor, the following steps are implemented:

[0040] Set a detection area, and obtain acquisition data within the detection area based on multi-sensors to obtain multi-source data;

[0041] Perform a fusion process on the multi-source data based on a fusion strategy to obtain fusion data, and analyze the obstacle distribution information within the detection area according to the fusion data;

[0042] Dynamically analyze the obstacle distribution information based on an obstacle avoidance algorithm to generate an obstacle avoidance strategy, and generate path planning information for the wheeled robot based on the obstacle avoidance strategy;

[0043] Obtain the movement trajectory information of the wheeled robot, compare the movement trajectory information of the wheeled robot with the path planning information of the wheeled robot, and obtain path difference information;

[0044] Analyze the performance metrics of the obstacle avoidance strategy based on the path difference information, analyze the obstacle avoidance state information of the wheeled robot based on the performance metrics of the obstacle avoidance strategy, and dynamically correct the obstacle avoidance strategy and the path planning information of the wheeled robot based on the obstacle avoidance state information.

[0045] Optionally, in the system for autonomous obstacle avoidance of a wheeled robot based on fused multi-sensors described in the embodiments of the present application, setting a detection area and obtaining acquisition data within the detection area based on multi-sensors, specifically includes:

[0046] Set a detection area, analyze the terrain and area of the detection area, and set at least one sensor of different types at different positions in the detection area based on the terrain and area. The sensors of different types include lidar, camera, and ultrasonic sensor;

[0047] Obtain the parameter information of the lidar, obtain a number of discrete distance data points based on the lidar parameter information, and form a three-dimensional point cloud map of the detection area based on the number of discrete distance data points to obtain three-dimensional point cloud data;

[0048] Based on the camera using the principle of optical imaging, focus the light in the detection area and convert it into a digital image signal. Identify the obstacle contour and features based on the digital image signal to obtain image data;

[0049] Based on the ultrasonic sensor using the piezoelectric effect, convert the electrical signal into ultrasonic waves, receive the reflected wave based on the ultrasonic sensor, and measure the time difference between transmission and reception;

[0050] Calculate the distance between the wheeled robot and the obstacle based on the time difference between transmission and reception to obtain ultrasonic data;

[0051] Obtain multi-source data based on the three-dimensional point cloud data, image data, and ultrasonic data.

[0052] Optionally, in the wheeled robot autonomous obstacle avoidance system based on the fusion of multiple sensors described in the embodiments of the present application, fuse the multi-source data based on the fusion strategy to obtain fusion data, and analyze the obstacle distribution information in the detection area according to the fusion data. Specifically, it includes:

[0053] Construct a data layer fusion strategy, a feature layer fusion strategy, and a probability statistics fusion strategy based on the fusion strategy;

[0054] Based on the data layer fusion strategy, perform coordinate transformation on the three-dimensional point cloud data and the image data, and convert the three-dimensional point cloud data and the image data to the same coordinate system for fusion to obtain data layer fusion information;

[0055] Extract data features from the three-dimensional point cloud data, image data, and ultrasonic data to obtain three-dimensional point cloud features, image features, and ultrasonic features. Based on the feature layer fusion strategy, fuse the three-dimensional point cloud features, image features, and ultrasonic features to obtain feature fusion information;

[0056] Based on the probability statistics fusion strategy, analyze the probability distributions of the ultrasonic data and the three-dimensional point cloud data, and fuse the probability distributions of the ultrasonic data and the three-dimensional point cloud data to obtain probability statistics fusion information;

[0057] Fuse the information based on the data layer, the feature fusion information, and the probability statistics fusion information to obtain the fused data, and analyze the obstacle position and the obstacle geometry based on the fused data to obtain the obstacle distribution information.

[0058] In a third aspect, an embodiment of the present application also provides a computer-readable storage medium, which includes a program for the autonomous obstacle avoidance method of a wheeled robot based on multi-sensor fusion. When the program for the autonomous obstacle avoidance method of a wheeled robot based on multi-sensor fusion is executed by a processor, the steps of the autonomous obstacle avoidance method of a wheeled robot based on multi-sensor fusion as described in any one of the above are implemented.

[0059] As can be seen from the above, an autonomous obstacle avoidance method, system, and medium for a wheeled robot based on multi-sensor fusion provided by an embodiment of the present application set a detection area, obtain acquisition data within the detection area based on multiple sensors to obtain multi-source data; fuse the multi-source data based on a fusion strategy to obtain fused data, and analyze the obstacle distribution information within the detection area according to the fused data; dynamically analyze the obstacle distribution information based on an obstacle avoidance algorithm to generate an obstacle avoidance strategy, and generate path planning information for the wheeled robot based on the obstacle avoidance strategy; obtain the movement trajectory information of the wheeled robot, compare the movement trajectory information of the wheeled robot with the path planning information of the wheeled robot to obtain path difference information; analyze the performance index of the obstacle avoidance strategy based on the path difference information, analyze the obstacle avoidance state information of the wheeled robot based on the performance index of the obstacle avoidance strategy, and dynamically correct the obstacle avoidance strategy and the path planning information of the wheeled robot based on the obstacle avoidance state information; obtain multi-source data through multiple sensors, fuse the multi-source data, and generate an obstacle avoidance strategy, so that the wheeled robot can plan an obstacle avoidance path more quickly and accurately, reduce unnecessary path adjustments, save obstacle avoidance time, and improve the overall movement efficiency. Description of the Drawings

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0061] Figure 1 It is a flowchart of the autonomous obstacle avoidance method for a wheeled robot based on multi-sensor fusion provided by an embodiment of the present application;

[0062] Figure 2 It is a flowchart of the multi-source data acquisition method of the autonomous obstacle avoidance method for a wheeled robot based on multi-sensor fusion provided by an embodiment of the present application;

[0063] Figure 3 This is a flowchart of the fusion data acquisition method for the wheeled robot autonomous obstacle avoidance method based on fused multi-sensors provided by the embodiments of the present application. Specific embodiments

[0064] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application claimed, but only represents the selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0065] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, the terms "first", "second", etc. are only used for descriptive distinction and cannot be understood as indicating or implying relative importance.

[0066] Please refer to Figure 1 , Figure 1 which is a flowchart of a wheeled robot autonomous obstacle avoidance method based on fused multi-sensors in some embodiments of the present application. The wheeled robot autonomous obstacle avoidance method based on fused multi-sensors is used in a terminal device. The wheeled robot autonomous obstacle avoidance method based on fused multi-sensors includes the following steps:

[0067] S101, set a detection area, and obtain acquisition data within the detection area based on multi-sensors to obtain multi-source data;

[0068] S102, perform fusion processing on the multi-source data based on a fusion strategy to obtain fusion data, and analyze the obstacle distribution information within the detection area according to the fusion data;

[0069] S103, perform dynamic analysis on the obstacle distribution information based on an obstacle avoidance algorithm to generate an obstacle avoidance strategy, and generate wheeled robot path planning information based on the obstacle avoidance strategy;

[0070] S104, obtain the moving trajectory information of the wheeled robot, compare the moving trajectory information of the wheeled robot with the wheeled robot path planning information to obtain path difference information;

[0071] S105. Analyze the performance metrics of the obstacle avoidance strategy based on the path difference information, analyze the obstacle avoidance status information of the wheeled robot based on the performance metrics of the obstacle avoidance strategy, and dynamically correct the obstacle avoidance strategy and the path planning information of the wheeled robot based on the obstacle avoidance status information.

[0072] It should be noted that multi-source data is collected through multiple sensors and the multi-source data is fused, so as to accurately analyze the distribution of obstacles, and then the obstacle avoidance can be effectively carried out according to the distribution of obstacles, improving the effect of autonomous obstacle avoidance.

[0073] Please refer to Figure 2 , Figure 2 FIG. is a flowchart of a multi-source data acquisition method for a wheeled robot's autonomous obstacle avoidance method based on the fusion of multiple sensors in some embodiments of the present application. According to the embodiments of the present invention, a detection area is set, and the acquisition data in the detection area is obtained based on multiple sensors to obtain multi-source data, specifically including:

[0074] S201. Set a detection area, analyze the terrain and area of the detection area, and set at least one sensor of different types at different positions in the detection area based on the terrain and area. The different types of sensors include lidar, cameras, and ultrasonic sensors;

[0075] S202. Obtain the parameter information of the lidar, obtain a number of discrete distance data points based on the lidar parameter information, and form a three-dimensional point cloud map of the detection area based on the number of discrete distance data points to obtain three-dimensional point cloud data;

[0076] S203. Based on the camera using the principle of optical imaging, focus the light in the detection area and convert it into a digital image signal, and identify the obstacle contour and features based on the digital image signal to obtain image data;

[0077] S204. Based on the ultrasonic sensor using the piezoelectric effect, convert the electrical signal into ultrasonic waves, receive the reflected waves based on the ultrasonic sensor, measure the time difference between transmission and reception, and calculate the distance between the wheeled robot and the obstacle based on the time difference between transmission and reception to obtain ultrasonic data;

[0078] S205. Obtain multi-source data based on the three-dimensional point cloud data, image data, and ultrasonic data.

[0079] It should be noted that the lidar has extremely high distance measurement accuracy, which can reach the millimeter level in an ideal environment, has a high data refresh rate, usually can perform multiple scans per second, and can reflect the dynamic changes of the environment in real time. The camera can provide rich texture and color information, which is helpful for identifying different types of obstacles, such as distinguishing pedestrians, vehicles, buildings, etc. The ultrasonic sensor has a low cost, a simple structure, is easy to install, and has a high accuracy in short-distance detection.

[0080] Please refer to Figure 3 , Figure 3 which is a flowchart of a fusion data acquisition method for a wheeled robot's autonomous obstacle avoidance method based on multi-sensor fusion in some embodiments of the present application. According to the embodiments of the present invention, multi-source data is fused based on a fusion strategy to obtain fusion data, and the obstacle distribution information in the detection area is analyzed based on the fusion data, specifically including:

[0081] S301, construct a data layer fusion strategy, a feature layer fusion strategy, and a probability statistics fusion strategy based on the fusion strategy;

[0082] S302, perform coordinate transformation on the 3D point cloud data and the image data based on the data layer fusion strategy, and convert the 3D point cloud data and the image data to the same coordinate system for fusion to obtain data layer fusion information;

[0083] S303, extract data features based on the 3D point cloud data, the image data, and the ultrasonic data to obtain 3D point cloud features, image features, and ultrasonic features, and fuse the 3D point cloud features, image features, and ultrasonic features based on the feature layer fusion strategy to obtain feature fusion information;

[0084] S304, analyze the probability distributions of the ultrasonic data and the 3D point cloud data based on the probability statistics fusion strategy, and fuse the probability distributions of the ultrasonic data and the 3D point cloud data to obtain probability statistics fusion information;

[0085] S305, fuse based on the data layer fusion information, the feature fusion information, and the probability statistics fusion information to obtain fusion data, and analyze the obstacle position and the obstacle geometry based on the fusion data to obtain the obstacle distribution information.

[0086] It should be noted that data layer fusion can retain all the information of the original data, make full use of the characteristics of each sensor, and provide rich materials for subsequent processing. Feature layer fusion can reduce the amount of data analysis and lower the computational complexity.

[0087] According to the embodiments of the present invention, the obstacle distribution information is dynamically analyzed based on an obstacle avoidance algorithm to generate an obstacle avoidance strategy, and the wheeled robot path planning information is generated based on the obstacle avoidance strategy, specifically including:

[0088] Obtain the obstacle position information at different time nodes, compare the obstacle position information of the same obstacle at adjacent time nodes, and obtain the position difference value;

[0089] Judge whether the position difference value is zero;

[0090] If it is zero, it is determined that the current obstacle is a stationary obstacle, and the stationary obstacle distribution information is obtained based on all the stationary obstacles;

[0091] If it is not zero, it is determined that the current obstacle is a moving obstacle, the moving direction and moving speed of the moving obstacle are obtained, and the moving trend information of the moving obstacle is generated based on the moving parameters and moving speed of the moving obstacle;

[0092] Based on the static obstacle distribution information and the moving trend information of the moving obstacle, an obstacle avoidance strategy is generated, and the path planning information of the wheeled robot is generated based on the obstacle avoidance strategy.

[0093] It should be noted that the motion trend of the obstacle is monitored through multi-sensor data. The camera uses a target tracking algorithm (such as the Kalman filter tracking algorithm) to track the motion trajectories of pedestrians and vehicles; the lidar analyzes the change rate of the point cloud data to obtain the speed and direction of the obstacle. If it is predicted that the obstacle will collide with the robot's motion path, the robot adjusts its motion direction in advance.

[0094] According to the embodiment of the present invention, the moving trajectory information of the wheeled robot is obtained, and the moving trajectory information of the wheeled robot is compared with the path planning information of the wheeled robot to obtain path difference information, which specifically includes:

[0095] Obtain the moving parameter information of the wheeled robot, where the moving parameter information of the wheeled robot includes the moving speed, moving acceleration, and moving direction of the wheeled robot;

[0096] Analyze the moving trajectory information of the wheeled robot based on the moving speed, moving acceleration, and moving direction of the wheeled robot;

[0097] Set multiple acquisition time nodes, and obtain the current position information of the wheeled robot based on the multiple acquisition time nodes;

[0098] Obtain the standard position information at the same acquisition time node based on the path planning information of the wheeled robot;

[0099] Compare the current position information with the standard position information, calculate the Euclidean distance, and analyze the path difference information based on the Euclidean distance.

[0100] It should be noted that by analyzing the Euclidean distance between the robot's moving trajectory and the path planning information at the same time node, the path difference information of the wheeled robot can be accurately analyzed.

[0101] According to the embodiment of the present invention, based on the path difference information, the performance index of the obstacle avoidance strategy is analyzed, and based on the performance index of the obstacle avoidance strategy, the obstacle avoidance state information of the wheeled robot is analyzed, which specifically includes:

[0102] Obtain the path difference information, compare the path difference information with the set condition information, and obtain the path difference value;

[0103] Compare the path difference value with a set difference range to obtain the performance indicators of the obstacle avoidance strategy. The performance indicators of the obstacle avoidance strategy include the success rate of obstacle avoidance, the path planning efficiency, and the obstacle avoidance time.

[0104] Analyze the obstacle avoidance status information of the wheeled robot based on the success rate of obstacle avoidance, the path planning efficiency, and the obstacle avoidance time.

[0105] It should be noted that by analyzing the performance indicators of the obstacle avoidance strategy, the obstacle avoidance effect of the obstacle avoidance strategy can be analyzed, so as to accurately analyze the obstacle avoidance status of the wheeled robot, and further provide an effective basis for adjusting the movement trajectory and the obstacle avoidance strategy.

[0106] According to an embodiment of the present invention, the obstacle avoidance strategy and the path planning information of the wheeled robot are dynamically corrected based on the obstacle avoidance status information, which specifically includes:

[0107] Obtain the obstacle avoidance status information, compare the obstacle avoidance status information with the set standard status information to obtain the status deviation rate;

[0108] Determine whether the status deviation rate is greater than or equal to the set status deviation rate threshold;

[0109] If it is greater than or equal to the set status deviation rate threshold, generate correction information, and adjust the obstacle avoidance strategy or the path planning information of the wheeled robot based on the correction information;

[0110] If it is less than the set status deviation rate threshold, continuously detect the obstacle avoidance status information of the wheeled robot.

[0111] It should be noted that by analyzing the obstacle avoidance status of the wheeled robot in real time, the obstacle avoidance strategy can be dynamically corrected according to the obstacle avoidance status, so as to improve the autonomous obstacle avoidance effect and ensure the obstacle avoidance accuracy of the wheeled robot.

[0112] In a second aspect, an embodiment of the present application provides a wheeled robot autonomous obstacle avoidance system based on multi-sensor fusion. The system includes: a memory and a processor. The memory includes a program for the wheeled robot autonomous obstacle avoidance method based on multi-sensor fusion. When the program for the wheeled robot autonomous obstacle avoidance method based on multi-sensor fusion is executed by the processor, the following steps are implemented:

[0113] Set a detection area, and obtain the acquisition data in the detection area based on multiple sensors to obtain multi-source data;

[0114] Based on a fusion strategy, fuse and process the multi-source data to obtain fused data, and analyze the obstacle distribution information in the detection area according to the fused data;

[0115] Dynamically analyze the obstacle distribution information based on an obstacle avoidance algorithm to generate an obstacle avoidance strategy, and generate path planning information for the wheeled robot based on the obstacle avoidance strategy;

[0116] Obtain the movement trajectory information of the wheeled robot, compare the movement trajectory information of the wheeled robot with the path planning information of the wheeled robot, and obtain the path difference information;

[0117] Analyze the performance indicators of the obstacle avoidance strategy based on the path difference information, analyze the obstacle avoidance state information of the wheeled robot based on the performance indicators of the obstacle avoidance strategy, and dynamically correct the obstacle avoidance strategy and the path planning information of the wheeled robot based on the obstacle avoidance state information.

[0118] It should be noted that multi-source data is collected through multiple sensors and the multi-source data is fused, so as to accurately analyze the distribution of obstacles, and then the obstacle avoidance can be effectively carried out according to the distribution of obstacles, improving the effect of autonomous obstacle avoidance.

[0119] According to the embodiment of the present invention, a detection area is set, and the acquisition data within the detection area is obtained based on multiple sensors to obtain multi-source data, specifically including:

[0120] Set a detection area, analyze the terrain and area of the detection area, and set at least one sensor of different types at different positions in the detection area based on the terrain and area. The different types of sensors include lidar, camera, and ultrasonic sensor;

[0121] Obtain the parameter information of the lidar, obtain a number of discrete distance data points based on the lidar parameter information, and form a three-dimensional point cloud map of the detection area based on the number of discrete distance data points to obtain three-dimensional point cloud data;

[0122] Based on the camera using the principle of optical imaging, focus the light in the detection area and convert it into a digital image signal, and identify the obstacle contour and features based on the digital image signal to obtain image data;

[0123] Based on the ultrasonic sensor using the piezoelectric effect, convert the electrical signal into ultrasonic waves, receive the reflected wave based on the ultrasonic sensor, and measure the time difference between transmission and reception;

[0124] Calculate the distance between the wheeled robot and the obstacle based on the time difference between transmission and reception to obtain ultrasonic data;

[0125] Obtain multi-source data based on the three-dimensional point cloud data, image data, and ultrasonic data.

[0126] It should be noted that the lidar has extremely high distance measurement accuracy, which can reach the millimeter level in an ideal environment, has a high data refresh rate, usually can be scanned multiple times per second, and can reflect the dynamic changes of the environment in real time. The camera can provide rich texture and color information, which helps to identify different types of obstacles, such as distinguishing pedestrians, vehicles, buildings, etc. The ultrasonic sensor has a low cost, a simple structure, is easy to install, and has a high accuracy in short-distance detection.

[0127] According to an embodiment of the present invention, multi-source data is fused based on a fusion strategy to obtain fused data, and the obstacle distribution information in the detection area is analyzed according to the fused data, specifically including:

[0128] Construct a data layer fusion strategy, a feature layer fusion strategy, and a probability statistics fusion strategy based on the fusion strategy;

[0129] Based on the data layer fusion strategy, perform coordinate transformation on the three-dimensional point cloud data and the image data, and convert the three-dimensional point cloud data and the image data to the same coordinate system for fusion to obtain data layer fusion information;

[0130] Extract data features from the three-dimensional point cloud data, the image data, and the ultrasonic data to obtain three-dimensional point cloud features, image features, and ultrasonic features, and fuse the three-dimensional point cloud features, image features, and ultrasonic features based on the feature layer fusion strategy to obtain feature fusion information;

[0131] Based on the probability statistics fusion strategy, analyze the probability distributions of the ultrasonic data and the three-dimensional point cloud data, and fuse the probability distributions of the ultrasonic data and the three-dimensional point cloud data to obtain probability statistics fusion information;

[0132] Fuse based on the data layer fusion information, the feature fusion information, and the probability statistics fusion information to obtain fused data, and analyze the obstacle positions and obstacle geometric shapes based on the fused data to obtain obstacle distribution information.

[0133] It should be noted that data layer fusion can retain all the information of the original data, make full use of the characteristics of each sensor, and provide rich materials for subsequent processing. Feature layer fusion can reduce the amount of data analysis and lower the computational complexity.

[0134] According to an embodiment of the present invention, the obstacle distribution information is dynamically analyzed based on an obstacle avoidance algorithm to generate an obstacle avoidance strategy, and the path planning information of the wheeled robot is generated based on the obstacle avoidance strategy, specifically including:

[0135] Obtain the obstacle position information at different time nodes, compare the obstacle position information of adjacent time nodes of the same obstacle, and obtain the position difference value;

[0136] Judge whether the position difference value is zero;

[0137] If it is zero, determine that the current obstacle is a stationary obstacle, and obtain the stationary obstacle distribution information based on all the stationary obstacles;

[0138] If it is not zero, determine that the current obstacle is a moving obstacle, obtain the moving direction and moving speed of the moving obstacle, and generate the moving trend information of the moving obstacle based on the moving parameters and moving speed of the moving obstacle;

[0139] Generate an obstacle avoidance strategy based on the distribution information of static obstacles and the moving trend information of moving obstacles, and generate path planning information for the wheeled robot based on the obstacle avoidance strategy.

[0140] It should be noted that the moving trend of obstacles is monitored through multi-sensor data. The camera uses a target tracking algorithm (such as the Kalman filter tracking algorithm) to track the movement trajectories of pedestrians and vehicles; the lidar analyzes the change rate of point cloud data to obtain the speed and direction of obstacles. If it is predicted that the obstacle will collide with the robot's movement path, the robot adjusts its movement direction in advance.

[0141] According to an embodiment of the present invention, obtain the movement trajectory information of the wheeled robot, compare the movement trajectory information of the wheeled robot with the path planning information of the wheeled robot, and obtain path difference information, specifically including:

[0142] Obtain the movement parameter information of the wheeled robot. The movement parameter information of the wheeled robot includes the movement speed, movement acceleration, and movement direction of the wheeled robot;

[0143] Analyze the movement trajectory information of the wheeled robot based on the movement speed, movement acceleration, and movement direction of the wheeled robot;

[0144] Set multiple acquisition time nodes, and obtain the current position information of multiple wheeled robots based on the multiple acquisition time nodes;

[0145] Obtain the standard position information at the same acquisition time node based on the path planning information of the wheeled robot;

[0146] Compare the current position information with the standard position information, calculate the Euclidean distance, and analyze the path difference information based on the Euclidean distance.

[0147] It should be noted that by analyzing the Euclidean distance between the robot's movement trajectory and the path planning information at the same time node, the path difference information of the wheeled robot can be accurately analyzed.

[0148] According to an embodiment of the present invention, analyze the performance index of the obstacle avoidance strategy based on the path difference information, and analyze the obstacle avoidance state information of the wheeled robot based on the performance index of the obstacle avoidance strategy, specifically including:

[0149] Obtain the path difference information, compare the path difference information with the set condition information, and obtain a path difference value;

[0150] Compare the path difference value with the set difference interval to obtain the performance index of the obstacle avoidance strategy. The performance index of the obstacle avoidance strategy includes the obstacle avoidance success rate, path planning efficiency, and obstacle avoidance time;

[0151] Analyze the obstacle avoidance state information of the wheeled robot based on the obstacle avoidance success rate, path planning efficiency, and obstacle avoidance time.

[0152] It should be noted that by analyzing the performance indicators of the obstacle avoidance strategy and the obstacle avoidance effect of the obstacle avoidance strategy, the obstacle avoidance state of the wheeled robot can be accurately analyzed, and then an effective basis can be provided for adjusting the movement trajectory and the obstacle avoidance strategy.

[0153] According to an embodiment of the present invention, the obstacle avoidance strategy and the path planning information of the wheeled robot are dynamically corrected based on the obstacle avoidance state information, which specifically includes:

[0154] Obtain the obstacle avoidance state information, compare the obstacle avoidance state information with the set standard state information, and obtain the state deviation rate;

[0155] Judge whether the state deviation rate is greater than or equal to the set state deviation rate threshold;

[0156] If it is greater than or equal to the set state deviation rate threshold, generate correction information, and adjust the obstacle avoidance strategy or the path planning information of the wheeled robot based on the correction information;

[0157] If it is less than the set state deviation rate threshold, the obstacle avoidance state information of the wheeled robot is detected in real time.

[0158] It should be noted that by analyzing the obstacle avoidance state of the wheeled robot in real time, the obstacle avoidance strategy is dynamically corrected according to the obstacle avoidance state, the autonomous obstacle avoidance effect is improved, and the obstacle avoidance accuracy of the wheeled robot is guaranteed.

[0159] A third aspect of the present invention provides a computer-readable storage medium, which includes a program for the autonomous obstacle avoidance method of a wheeled robot based on multi-sensor fusion. When the program for the autonomous obstacle avoidance method of a wheeled robot based on multi-sensor fusion is executed by a processor, the steps of the autonomous obstacle avoidance method of a wheeled robot based on multi-sensor fusion as described in any one of the above are implemented.

[0160] A wheeled robot autonomous obstacle avoidance method, system and medium based on the fusion of multiple sensors disclosed by the present invention. By setting a detection area, acquisition data within the detection area is obtained based on multiple sensors to obtain multi-source data; the multi-source data is fused and processed based on a fusion strategy to obtain fused data, and the obstacle distribution information within the detection area is analyzed according to the fused data; the obstacle distribution information is dynamically analyzed based on an obstacle avoidance algorithm to generate an obstacle avoidance strategy, and the wheeled robot path planning information is generated based on the obstacle avoidance strategy; the wheeled robot movement trajectory information is obtained, and the wheeled robot movement trajectory information is compared with the wheeled robot path planning information to obtain path difference information; the performance index of the obstacle avoidance strategy is analyzed based on the path difference information, the obstacle avoidance state information of the wheeled robot is analyzed based on the performance index of the obstacle avoidance strategy, and the obstacle avoidance strategy and the wheeled robot path planning information are dynamically corrected based on the obstacle avoidance state information; multi-source data is obtained through multiple sensors, the multi-source data is fused and processed, and an obstacle avoidance strategy is generated, so that the wheeled robot can plan an obstacle avoidance path more quickly and accurately, reduce unnecessary path adjustments, save obstacle avoidance time, and improve the overall movement efficiency.

[0161] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed with each other can be through some interfaces. The indirect coupling or communication connection of devices or units can be electrical, mechanical, or other forms.

[0162] The units described as separate components above may or may not be physically separated. The components shown as units may or may not be physical units; they can be located in one place or distributed to multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0163] In addition, each functional unit in the embodiments of the present invention can be all integrated in a processing unit, or each unit can be separately used as a unit, or two or more units can be integrated in one unit; the above integrated units can be implemented in the form of hardware, or in the form of hardware plus software functional units.

[0164] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a readable storage medium. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned storage medium includes: various media that can store program codes such as removable storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0165] Alternatively, if the above integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. And the aforementioned storage medium includes: various media that can store program codes such as removable storage devices, ROM, RAM, magnetic disks, or optical discs.

Claims

1. An autonomous obstacle avoidance method for a wheeled robot based on multi-sensor fusion, characterized in that, Including: Set a detection area, obtain acquisition data within the detection area based on multiple sensors, and obtain multi-source data; Perform fusion processing on the multi-source data based on a fusion strategy to obtain fusion data, and analyze the obstacle distribution information within the detection area according to the fusion data; Perform dynamic analysis on the obstacle distribution information based on an obstacle avoidance algorithm to generate an obstacle avoidance strategy, and generate wheeled robot path planning information based on the obstacle avoidance strategy; Obtain the moving trajectory information of the wheeled robot, compare the moving trajectory information of the wheeled robot with the wheeled robot path planning information, and obtain path difference information; Analyze the performance index of the obstacle avoidance strategy based on the path difference information, analyze the obstacle avoidance state information of the wheeled robot based on the performance index of the obstacle avoidance strategy, and dynamically correct the obstacle avoidance strategy and the wheeled robot path planning information based on the obstacle avoidance state information.

2. The method for autonomous obstacle avoidance of a wheeled robot based on multi-sensor fusion according to claim 1, characterized in that, Set a detection area, obtain acquisition data within the detection area based on multiple sensors, and obtain multi-source data, specifically including: Set a detection area, analyze the terrain and area of the detection area, and set at least one sensor of different types at different positions in the detection area based on the terrain and area. Different types of sensors include lidar, cameras, and ultrasonic sensors; Obtain the parameter information of the lidar, obtain a number of discrete distance data points based on the lidar parameter information, and form a three-dimensional point cloud map of the detection area based on the number of discrete distance data points to obtain three-dimensional point cloud data; Based on the camera using the principle of optical imaging, focus the light in the detection area and convert it into a digital image signal. Identify the obstacle contour and features based on the digital image signal to obtain image data; Based on the ultrasonic sensor using the piezoelectric effect, convert the electrical signal into ultrasonic waves, receive the reflected wave based on the ultrasonic sensor, and measure the time difference between transmission and reception; Calculate the distance between the wheeled robot and the obstacle based on the time difference between transmission and reception to obtain ultrasonic data; Obtain multi-source data based on the three-dimensional point cloud data, image data, and ultrasonic data.

3. The method for autonomous obstacle avoidance of a wheeled robot based on fused multi-sensors according to claim 2, wherein Perform fusion processing on the multi-source data based on a fusion strategy to obtain fusion data, and analyze the obstacle distribution information within the detection area according to the fusion data, specifically including: Construct a data layer fusion strategy, a feature layer fusion strategy, and a probability statistics fusion strategy based on the fusion strategy; Perform coordinate transformation on the three-dimensional point cloud data and the image data based on the data layer fusion strategy, and convert the three-dimensional point cloud data and the image data to the same coordinate system for fusion to obtain data layer fusion information; Extract data features from the three-dimensional point cloud data, image data, and ultrasonic data to obtain three-dimensional point cloud features, image features, and ultrasonic features. Perform fusion on the three-dimensional point cloud features, image features, and ultrasonic features based on the feature layer fusion strategy to obtain feature fusion information; Analyze the probability distributions of the ultrasonic data and the three-dimensional point cloud data based on the probability statistics fusion strategy, and fuse the probability distributions of the ultrasonic data and the three-dimensional point cloud data to obtain probability statistics fusion information; Perform fusion based on the data layer fusion information, feature fusion information, and probability statistics fusion information to obtain fusion data, and analyze the obstacle position and obstacle geometry based on the fusion data to obtain obstacle distribution information.

4. The method for autonomous obstacle avoidance of a wheeled robot based on fused multi-sensors according to claim 3, wherein, Dynamically analyze the obstacle distribution information based on the obstacle avoidance algorithm to generate an obstacle avoidance strategy, and generate the path planning information of the wheeled robot based on the obstacle avoidance strategy, specifically including: Obtain the obstacle position information at different time nodes, compare the obstacle position information of adjacent time nodes of the same obstacle, and obtain the position difference value; Judge whether the position difference value is zero; If it is zero, determine that the current obstacle is a stationary obstacle, and obtain the stationary obstacle distribution information based on all stationary obstacles; If it is not zero, determine that the current obstacle is a moving obstacle, obtain the moving direction and moving speed of the moving obstacle, and generate the moving trend information of the moving obstacle based on the moving parameters and moving speed of the moving obstacle; Generate an obstacle avoidance strategy based on the stationary obstacle distribution information and the moving trend information of the moving obstacle, and generate the path planning information of the wheeled robot based on the obstacle avoidance strategy.

5. The method for autonomous obstacle avoidance of a wheeled robot based on fused multi-sensors according to claim 4, wherein Obtain the moving trajectory information of the wheeled robot, compare the moving trajectory information of the wheeled robot with the path planning information of the wheeled robot, and obtain the path difference information, specifically including: Obtain the moving parameter information of the wheeled robot, and the moving parameter information of the wheeled robot includes the moving speed, moving acceleration and moving direction of the wheeled robot; Analyze the moving trajectory information of the wheeled robot based on the moving speed, moving acceleration and moving direction of the wheeled robot; Set multiple acquisition time nodes, and obtain the current position information of multiple wheeled robots based on the multiple acquisition time nodes; Obtain the standard position information at the same acquisition time node based on the path planning information of the wheeled robot; Compare the current position information with the standard position information, calculate the Euclidean distance, and analyze the path difference information based on the Euclidean distance.

6. The method for autonomous obstacle avoidance of a wheeled robot based on multi-sensor fusion according to claim 5, characterized in that Analyze the performance index of the obstacle avoidance strategy based on the path difference information, and analyze the obstacle avoidance state information of the wheeled robot based on the performance index of the obstacle avoidance strategy, specifically including: Obtain the path difference information, compare the path difference information with the set condition information, and obtain the path difference value; Compare the path difference value with the set difference interval to obtain the performance index of the obstacle avoidance strategy, and the performance index of the obstacle avoidance strategy includes the obstacle avoidance success rate, path planning efficiency and obstacle avoidance time; Analyze the obstacle avoidance state information of the wheeled robot based on the obstacle avoidance success rate, path planning efficiency and obstacle avoidance time.

7. A wheeled robot autonomous obstacle avoidance system based on the fusion of multiple sensors, characterized in that, The system includes: a memory and a processor, and the memory includes a program for the autonomous obstacle avoidance method of the wheeled robot based on the fusion of multiple sensors. When the program for the autonomous obstacle avoidance method of the wheeled robot based on the fusion of multiple sensors is executed by the processor, the following steps are implemented: Set a detection area, and obtain the acquisition data in the detection area based on multiple sensors to obtain multi-source data; Perform fusion processing on the multi-source data based on the fusion strategy to obtain the fusion data, and analyze the obstacle distribution information in the detection area according to the fusion data; Dynamically analyze the obstacle distribution information based on the obstacle avoidance algorithm to generate an obstacle avoidance strategy, and generate the path planning information of the wheeled robot based on the obstacle avoidance strategy; Obtain the moving trajectory information of the wheeled robot, compare the moving trajectory information of the wheeled robot with the path planning information of the wheeled robot, and obtain the path difference information; Analyze the performance metrics of the obstacle avoidance strategy based on path difference information, analyze the obstacle avoidance status information of the wheeled robot based on the performance metrics of the obstacle avoidance strategy, and dynamically correct the obstacle avoidance strategy and the path planning information of the wheeled robot based on the obstacle avoidance status information.

8. The wheeled robot autonomous obstacle avoidance system based on the fusion of multiple sensors according to claim 7, wherein Set the detection area, and obtain the acquisition data within the detection area based on multiple sensors to obtain multi-source data, specifically including: Set the detection area, analyze the terrain and area of the detection area, and set at least one sensor of different types at different positions in the detection area based on the terrain and area. Different types of sensors include lidar, cameras, and ultrasonic sensors; Obtain the parameter information of the lidar, obtain a number of discrete distance data points based on the lidar parameter information, and construct a three-dimensional point cloud map of the detection area based on the number of discrete distance data points to obtain three-dimensional point cloud data; Based on the camera using the principle of optical imaging, focus the light in the detection area and convert it into a digital image signal. Identify the obstacle contour and features based on the digital image signal to obtain image data; Based on the ultrasonic sensor using the piezoelectric effect, convert the electrical signal into ultrasonic waves, and receive the reflected wave based on the ultrasonic sensor to measure the time difference between transmission and reception; Calculate the distance between the wheeled robot and the obstacle based on the time difference between transmission and reception to obtain ultrasonic data; Obtain multi-source data based on the three-dimensional point cloud data, image data, and ultrasonic data.

9. The wheeled robot autonomous obstacle avoidance system based on the fusion of multiple sensors according to claim 8, characterized in that, Perform fusion processing on the multi-source data based on the fusion strategy to obtain fusion data, and analyze the obstacle distribution information in the detection area according to the fusion data, specifically including: Construct a data layer fusion strategy, a feature layer fusion strategy, and a probability statistical fusion strategy based on the fusion strategy; Perform coordinate transformation on the three-dimensional point cloud data and the image data based on the data layer fusion strategy, and convert the three-dimensional point cloud data and the image data to the same coordinate system for fusion to obtain data layer fusion information; Extract data features based on the three-dimensional point cloud data, image data, and ultrasonic data to obtain three-dimensional point cloud features, image features, and ultrasonic features, and fuse the three-dimensional point cloud features, image features, and ultrasonic features based on the feature layer fusion strategy to obtain feature fusion information; Analyze the probability distributions of the ultrasonic data and the three-dimensional point cloud data based on the probability statistical fusion strategy, and fuse the probability distributions of the ultrasonic data and the three-dimensional point cloud data to obtain probability statistical fusion information; Fuse the data layer fusion information, the feature fusion information, and the probability statistical fusion information to obtain fusion data, and analyze the obstacle position and the geometric shape of the obstacle based on the fusion data to obtain obstacle distribution information.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a program for the method of autonomous obstacle avoidance of a wheeled robot based on fused multi-sensors. When the program for the method of autonomous obstacle avoidance of a wheeled robot based on fused multi-sensors is executed by a processor, it implements the steps of the method of autonomous obstacle avoidance of a wheeled robot based on fused multi-sensors as described in any one of claims 1 to 6.

Images