Inspection robot obstacle avoidance method based on multi-sensor fusion

Abstract

The invention relates to the field of robot obstacle avoidance, in particular to an inspection robot obstacle avoidance method based on multi-sensor fusion, and is applied to an inspection robot. The inspection robot comprises a main body structure, a laser radar arranged at the top of the main body structure, a multi-view structure camera arranged on the front surface of the main body structure and a plurality of ultrasonic sensors which are arranged on the main body structure and are circumferentially distributed; the main body structure can steer in situ; the obstacle avoidance method comprises the following steps: scanning environment information on the peripheral side of the top surface of a main body structure at 360 degrees by adopting a laser radar to obtain first scanning data, and scanning environment information of at least part of areas below the top of the main body structure at 360 degrees by adopting a plurality of ultrasonic sensors to obtain second scanning data; filtering the second scanning data to remove noise in the second scanning data to obtain third scanning data; judging whether an obstacle exists in the advancing direction of the main body structure based on the first scanning data and the third scanning data, and outputting a judgment result; and selectively using the multi-view structure camera based on the judgment result.

Description

Technical Field

[0001] This invention relates to the field of robot obstacle avoidance technology, and specifically to an obstacle avoidance method for inspection robots based on multi-sensor fusion. Background Technology

[0002] With the continuous iteration of robotics technology and strong government support, robots have been widely integrated into production and daily life scenarios. Especially in the field of inspection, using robots to replace humans in performing related tasks can not only significantly improve work efficiency, but also effectively avoid exposing personnel to dangerous and harsh working environments, reduce safety risks, and has important practical application value.

[0003] Among various types of mobile robots, the two-wheeled differential robot is a commonly used structural form. By independently controlling the speed of the left and right wheels, it can flexibly realize forward, backward, and turning movements. It has outstanding advantages such as simple structure, high mobility, and low manufacturing cost, and therefore has been widely used in many fields such as warehousing and logistics, indoor services, and industrial automation.

[0004] However, in practical applications, two-wheeled differential robots often need to complete autonomous navigation and obstacle avoidance tasks in dynamic and uncertain environments. The nonholonomic constraints inherent in these robots significantly increase the complexity of motion control and obstacle avoidance decisions, which can easily lead to decreased navigation accuracy and delayed obstacle avoidance response. This, in turn, greatly reduces their work efficiency and operational reliability, becoming a key issue restricting the widespread application of two-wheeled differential robots in complex scenarios.

[0005] Current technologies for robot obstacle avoidance solutions largely rely on single-type sensors, such as ultrasonic sensors, vision sensors, or lidar. However, each type of sensor has inherent limitations: ultrasonic sensors are susceptible to environmental noise interference; lidar performance degrades significantly when facing transparent objects or highly reflective surfaces; and vision sensors are sensitive to lighting conditions, easily exhibiting detection errors in strong light, low light, or backlight environments. The performance deficiencies of single sensors result in robots' perception of environmental information being insufficiently comprehensive and accurate, making it difficult to meet the high-precision obstacle avoidance requirements in dynamic and uncertain environments, further exacerbating the obstacle avoidance challenges of two-wheeled differential robots.

[0006] Therefore, how to overcome the shortcomings of the existing technology is the subject of this invention. Summary of the Invention

[0007] The purpose of this invention is to provide an obstacle avoidance method for inspection robots based on multi-sensor fusion.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] An obstacle avoidance method for inspection robots based on multi-sensor fusion is applied to an inspection robot. The inspection robot includes a main structure, a lidar mounted on the top of the main structure, a multi-view camera mounted on the front of the main structure, and multiple ultrasonic sensors mounted on the main structure (preferably at the bottom, as will be described below) and arranged in a circular pattern. The main structure serves as a component capable of turning in place.

[0010] Obstacle avoidance methods include the following steps:

[0011] Step 1: Use the laser radar to scan the environmental information of the periphery of the top surface of the main structure in 360° to obtain the first scan data, and use multiple ultrasonic sensors to scan the environmental information of at least part of the area below the top of the main structure in 360° to obtain the second scan data;

[0012] Step 2: Filter the second scan data to remove noise and obtain the third scan data;

[0013] Step 3: Based on the first scan data and the third scan data, determine whether there are obstacles in the direction of travel of the main structure, and output the determination result;

[0014] Step 4: Selectively use the multi-view structure camera based on the judgment result, including:

[0015] When the judgment result indicates that there is an obstacle in the direction of travel, the multi-view structure camera is used to identify the category of the obstacle, and an obstacle avoidance algorithm is used to plan an obstacle avoidance path based on the category.

[0016] When the determination result is that there are no obstacles in the direction of travel, the multi-view structure camera remains off, or the information collected by the multi-view structure camera is ignored.

[0017] Each detection device can transmit data in real time via Wi-Fi module, which will not be elaborated here.

[0018] LiDAR (which can use the UART communication protocol for data transmission) can achieve 360° scanning with a single structure. Using it to scan the environmental information of the area around the top surface of the main structure can reduce the implementation cost of obstacle avoidance methods, including structural and detection costs. If multiple ultrasonic sensors are also used here, the data from multiple ultrasonic sensors needs to be fused for judgment, which will increase both structural and detection costs.

[0019] LiDAR is not very effective at identifying transparent or mirror-reflective objects, and it cannot scan environmental information below the top of the main structure. It should be noted that although the total number of transparent, mirror-reflective, and low-lying objects in the inspection environment of the inspection robot may be very small, and the inspection robot itself may be low in height to further reduce the total number of possible low-lying objects, these objects still need to be taken into account to reduce the risk of collision.

[0020] The main structure, as a component capable of turning in place, may include a chassis capable of turning in place, and this can be achieved using existing methods, which will not be elaborated upon here. This configuration allows for the mounting of a multi-view camera only on the front of the main structure.

[0021] This application describes a main structure with four circumferential surfaces. Addressing the limitations of lidar, at least one ultrasonic sensor is installed on each of the four surfaces of the main structure for auxiliary scanning. The installation positions of the ultrasonic sensors allow them to scan blind spots that lidar cannot cover, and to accurately scan transparent and specularly reflective objects where lidar scanning is ineffective. In other words, using ultrasonic sensors for obstacle avoidance reduces the risk of missed scans. The obstacle avoidance path is planned using existing algorithms, such as Dijkstra's algorithm. When planning the obstacle avoidance path, it is necessary to consider obstacles outside the path; therefore, the scan is a 360° scan, and attention must be paid to obstacles outside the path, including low-lying obstacles.

[0022] Ultrasonic sensors (which can transmit data using the RS485 communication protocol) have a lower accuracy compared to lidar. To address this, the second scan data is filtered in step two to remove noise (using a moving average filtering algorithm), thus appropriately mitigating the defect and further reducing the risk of missed scans. Furthermore, the first scan data is not filtered in step two, avoiding unnecessary steps and reducing the detection burden.

[0023] In step three, the presence of obstacles in the direction of travel of the main structure is determined based on the first and third scan data. The determination method can be any method well-known to those skilled in the art, and will not be elaborated upon here. The main structure has a predetermined inspection path; the main purpose is to determine whether there are obstacles along this path.

[0024] In step four, obstacle avoidance algorithms can be used to plan obstacle avoidance paths. Here, the same methods known to those skilled in the art can be used, which will not be elaborated here.

[0025] In step four, multi-view structure cameras are used selectively; that is, multi-view structure cameras may not be used in step one. It should be noted that visual sensors are susceptible to changes in lighting conditions, exhibiting a significant performance degradation at night. While multi-view structure cameras mitigate this effect compared to ordinary visual sensors, the limitation still exists. Multi-view structure cameras can use USB for data transfer and can accurately identify obstacles using existing target detection algorithms.

[0026] Due to the limitations of multi-view cameras, neither the ultrasonic sensor nor the lidar is replaced in step one, thus ensuring the applicability of the obstacle avoidance method in this application. Multi-view cameras, however, excel at identifying obstacle categories. By determining the obstacle category and generating an obstacle avoidance path accordingly, the collision risk of the inspection robot can be further reduced. It should be noted that if the multi-view camera fails to identify the obstacle category, it can be classified as an unidentified obstacle. This mitigates the impact of lighting changes on the multi-view camera and prevents the inability to generate an obstacle avoidance path due to the camera's failure to identify the obstacle. In this case, an obstacle avoidance path can be directly generated based on the unidentified obstacle category according to a predetermined procedure, ensuring the applicability of the obstacle avoidance method in this application.

[0027] The obstacle avoidance method in this application is designed for a specific inspection robot with a main structure capable of turning in place. Considering that the multi-view structure camera is always aimed at the direction of travel of the inspection robot, only one multi-view structure camera needs to be installed on the front of the inspection robot (i.e. the front of the main structure), thereby further reducing the implementation cost of the obstacle avoidance method in this application, including structural cost and detection cost.

[0028] In summary, the obstacle avoidance method in this application is based on a specific type of inspection robot and employs a single LiDAR, multiple ultrasonic sensors, and a single multi-view camera to collaboratively achieve obstacle avoidance for this type of inspection robot. This ensures that the obstacle avoidance method can be applied to various environments, such as environments with sufficient light and shadow, environments lacking light sources, and environments with obstacles such as low-lying objects. Simultaneously, it effectively controls implementation costs and the collision risk of the inspection robot. In other words, the obstacle avoidance method in this application improves the accuracy and reliability of environmental perception by fusing complementary information from multiple sensors, thereby providing the inspection robot with a more powerful autonomous obstacle avoidance capability.

[0029] Furthermore, the obstacle avoidance method in this application can still avoid obstacles even when a single sensor fails, which improves reliability and fault tolerance, while also enhancing adaptability in environments such as day and night.

[0030] It should be noted that for ultrasonic sensors, lidar, and multi-view structure cameras, the timestamp information in the data headers of the three can be unified with the current system time to achieve time synchronization. Simultaneously, the transformation relationship between the coordinate systems of the three can be determined using the robot's center of mass as the origin, thus completing spatial calibration. This is existing technology and well-known to those skilled in the art, and will not be elaborated upon here.

[0031] It should also be noted that feature extraction and multimodal fusion can be performed on the first and third scan data. Feature extraction is as follows: extracting geometric features such as edges and corners, as well as depth features such as distance, from the LiDAR point cloud; extracting depth information from a specific area fed back by the ultrasonic sensor. Multimodal fusion is as follows: using an extended Kalman filter algorithm to fuse the extracted depth information from the LiDAR and ultrasonic sensors; associating and calibrating the point cloud information fed back by the LiDAR, such as the size and specific location of obstacles, with the one-dimensional depth information fed back by the ultrasonic sensor, and then performing feature fusion. This is not an innovation of this application; existing feature extraction and data fusion methods can be referenced.

[0032] In a further technical solution, in step three, the first scan data and the third scan data are used based on the first decision fusion mechanism or the second decision fusion mechanism;

[0033] The first decision fusion mechanism includes:

[0034] First, based on the first scan data, determine whether there are obstacles in the direction of travel of the main structure, and obtain a first determination result;

[0035] If the first determination result indicates the presence of an obstacle, then the third scan data is discarded;

[0036] If the first determination result is that there is no obstacle, then the third scan data is used to determine whether there is an obstacle in the direction of travel of the main structure;

[0037] The second decision fusion mechanism includes:

[0038] Simultaneously, the first scan data and the third scan data are used to determine whether there are obstacles in the direction of travel of the main structure.

[0039] Different inspection robots are used in different environments, for example as follows:

[0040] Some inspection robots are used on complex roads (representing the first environment), where there are low obstacles such as flower beds and bricks, as well as transparent objects (such as plastic films covering equipment) or reflective objects (such as reflective signs).

[0041] Some inspection robots are used in factories with strict requirements (representing a second environment), where there are basically no low obstacles on the ground, and there are basically no transparent or reflective objects.

[0042] When the inspection robot is applied in the first environment, for risk control, especially to reduce the risk of collision, the first and third scan data need to be used based on the second decision fusion mechanism. That is, regardless of whether the obstacle is detected based on the first scan data, the third scan data needs to be used to avoid the risk that the lidar has difficulty in identifying obstacles at low altitudes, transparent obstacles, and obstacles with reflective surfaces. If the third scan data is not used synchronously, there is a greater risk of collision. Therefore, in the first environment, the focus is on reducing the risk of collision.

[0043] When the inspection robot is applied in the second environment, since there are basically no low-lying obstacles, transparent obstacles, or obstacles with reflective surfaces, the first scan data is prioritized for detection. If information about obstacles in the forward direction is obtained, the third scan data is discarded to reduce the detection burden. If no obstacle is detected, the third scan data is then detected for secondary verification, reducing the risk of collision for the inspection robot and balancing the need to reduce the detection burden (such as reducing data processing volume) with the need to reduce the risk of collision.

[0044] It should be noted that in the second environment, it is generally easier to guarantee the absence of low-lying obstacles, while the presence of transparent or reflective obstacles is relatively difficult to guarantee. For example, in factories, due to strict management, there is no scattered goods, and in such cases, low-lying obstacles are generally absent. However, transparent or reflective obstacles may exist, though they are relatively few. In some factories, the inspection robot's path is controlled to prevent obstacles from appearing, such as moving workers and forklifts. In this case, obstacles are mainly those that are difficult for LiDAR to detect. The first decision fusion mechanism is particularly well-suited to this environment, making the obstacle avoidance method in this application widely applicable. It is understandable that the second environment emphasizes balancing the need to reduce detection burden with the need to reduce collision risk.

[0045] In a further technical solution, step three involves using the first scan data and the third scan data based on the first decision fusion mechanism.

[0046] In step four, after planning the obstacle avoidance path, the collision situation is detected during the obstacle avoidance process of the inspection robot. When the collision frequency exceeds the predetermined safe frequency, the second decision fusion mechanism is temporarily switched to be used within the first predetermined time period.

[0047] The collision detection method uses existing techniques and is not innovative in this application. To briefly explain: it involves deploying a collision switch, force sensor, or elastic collision ring with a Hall effect sensor on the device body to collect contact signals or sudden force / torque signals in real time, which are then used to determine the collision frequency based on threshold values. The time period used to reference the collision frequency will be determined according to actual requirements.

[0048] The first scheduled time will be confirmed based on actual needs, and there is no specific time limit. The same applies to similar settings.

[0049] To facilitate understanding, based on the above explanation, we would like to supplement this by taking a factory as an example. In many factories, due to strict management, there is no situation where goods are scattered or disorderly. In such cases, there are basically no low-lying obstacles. However, these factories may have transparent obstacles or obstacles with reflective surfaces, but such obstacles account for a relatively small percentage. Based on this, we will explain in detail as follows:

[0050] There is a risk of missing obstacles. For example, if there is an obstacle at 10 meters that is detected by the lidar, but there is also a reflective obstacle at 5 meters, the obstacle will be missed because the third scanning data is not used, resulting in an unexpected collision. When the frequency of such unexpected collisions reaches a predetermined threshold (i.e., a predetermined safe frequency), such as two unexpected collisions occurring during a one-hour inspection, the system will temporarily switch to the second decision fusion mechanism to meet the current need to reduce the risk of collisions.

[0051] In some implementations, in step four, collision information is collected, and when the collision frequency exceeds a predetermined safe frequency, an alarm is issued to remind staff to conduct an inspection, such as checking for reflective obstacles to be dealt with, thereby reducing the risk of collision.

[0052] In some implementations, if the first judgment result is that there is no obstacle, but the third scan data determines that there is an obstacle in the direction of travel of the main structure, and the frequency of this situation reaches the set frequency, it means that the total number of low-level obstacles, transparent obstacles and surface reflective obstacles in the inspection environment exceeds the expected number. At this time, the second decision fusion mechanism is temporarily switched to be used, and an alarm can be set to remind staff to handle the situation and reduce the risk of collision.

[0053] It should be noted that the inspection robot can be configured to avoid obstacles based on the estimated environmental conditions or experimental conditions, using either the first or second decision fusion mechanism. However, to avoid errors in the estimated conditions, a switching mechanism should be set for the decision fusion mechanism so that the inspection robot can adapt to the actual situation of the inspection environment and reduce the risk of collision.

[0054] A further technical solution involves temporarily switching to the second decision fusion mechanism. If an obstacle is detected in the direction of travel of the inspection robot based solely on the third scanning data, the remaining switching time is extended to match the first predetermined time.

[0055] The robot temporarily switches to the second decision fusion mechanism for a first predetermined time period, during which a timer is run. If, during this period, an obstacle is detected in the robot's direction of travel based solely on the third scan data, it is determined that the robot's current environment still carries a higher-than-expected collision risk. Therefore, the timer is reset, and the second decision fusion mechanism is used again for the subsequent first predetermined time period, with the process repeating thereafter. This timer reset mechanism further enables the inspection robot to adapt to the actual conditions of the inspection environment, thereby further reducing the collision risk.

[0056] In a further technical solution, in step three, the first scan data and the third scan data are used based on the second decision fusion mechanism. If the recognition result obtained based on the first scan data and the recognition result obtained based on the third scan data are consistent within a second predetermined time, then the first decision fusion mechanism will be temporarily switched to be used within a subsequent third predetermined time.

[0057] The recognition result here refers to the recognition result of obstacles, such as both identifying obstacle one and obstacle two.

[0058] For example, during the timing process, if the recognition results based on the first scan data and the recognition results based on the third scan data are consistent within ten minutes, then the first decision fusion mechanism will be temporarily switched to be used for the next five minutes. If the two recognition results are inconsistent at the sixth minute, the timer will be reset and restarted.

[0059] If the recognition results obtained based on the first scan data and the recognition results obtained based on the third scan data are consistent within the second predetermined time, it indicates that the inspection environment is relatively safe, and it is not necessary to simultaneously use the third scan data to reduce the risk of collision, thereby reducing the detection burden. By setting a switching mechanism for the decision fusion mechanism, the inspection robot can adapt to the actual situation of the inspection environment, balancing the need to reduce the detection burden and the need to reduce the risk of collision.

[0060] A further technical solution involves temporarily switching to the first decision fusion mechanism while detecting collisions during the obstacle avoidance process of the inspection robot. If the collision frequency remains below a predetermined safe frequency, the time for using the first decision fusion mechanism is extended (i.e., the remaining switching time is extended) to make it consistent with the third predetermined time.

[0061] It can be done by directly determining whether the collision frequency during the process of switching to the first decision fusion mechanism is lower than the predetermined safe frequency. Alternatively, the time period can be divided into multiple sub-time periods for judgment.

[0062] Temporarily switching to the first decision fusion mechanism indicates that the current inspection environment is considered relatively safe. To adapt to potential changes in the inspection environment, collision detection is performed during the obstacle avoidance process of the inspection robot. Obstacle detection may have a risk of missed detection. For example, an obstacle may be detected by the lidar at 10 meters, but another obstacle with a reflective surface may be detected at 5 meters. Because the third scan data was not used, this leads to a missed detection and an unexpected collision. If the frequency of unexpected collisions within the third predetermined time period remains below the predetermined safe frequency (e.g., the number of collisions is less than 2 in any hour), the current inspection environment can be considered relatively safe. In this case, the time for using the first decision fusion mechanism is extended to align with the third predetermined time, further balancing the need to reduce the detection burden and the need to reduce collision risk based on the actual inspection environment.

[0063] It can be set to immediately switch back to the third decision fusion mechanism if a collision occurs during the switching process, and there are no specific restrictions.

[0064] A further technical solution involves setting a safe distance in step one;

[0065] In step three, when the judgment result indicates that there is an obstacle in the direction of travel, the distance between the main structure and the obstacle is measured to obtain the obstacle avoidance distance;

[0066] In step four, if the obstacle avoidance distance is less than the safe distance, the main structure is immediately stopped and reversed in place, and then obstacle avoidance is performed based on the obstacle avoidance path; if the obstacle avoidance distance is greater than or equal to the safe distance, the movement state of the main structure is maintained and obstacle avoidance is performed based on the obstacle avoidance path.

[0067] The ranging method is existing and is not an innovation of this application. For example, it can be implemented based on a multi-view camera structure, with binocular vision simulating human eye parallax, and calculating the three-dimensional distance by matching the pixels of two images.

[0068] It should be noted that obstacles refer to those in the direction of travel. Obstacles not initially in the direction of travel are only considered obstacles requiring obstacle avoidance if they subsequently move into that direction. Obstacles not initially in the direction of travel are considered obstacles that must be avoided along the planned obstacle avoidance path. If multiple obstacles exist in the direction of travel, the nearest obstacle is used as the reference obstacle for the obstacle avoidance distance.

[0069] For example, the safety distance is set to one meter. If the obstacle avoidance distance is less than one meter, it is considered that the obstacle avoidance space is insufficient. To avoid colliding with obstacles during obstacle avoidance, the inspection robot first stops, then turns in place to avoid moving towards the obstacle, and then performs obstacle avoidance to cross the obstacle and return to the predetermined inspection path. If the obstacle avoidance distance is greater than or equal to one meter, it is considered that the obstacle avoidance space is sufficient. The inspection robot can continue moving and directly perform obstacle avoidance, thereby shortening the obstacle avoidance time and ensuring the inspection efficiency of the inspection robot.

[0070] A further technical solution involves pre-classifying obstacles into different categories in step one. Obstacles of the same category correspond to the same safe distance, while obstacles of different categories correspond to different safe distances.

[0071] For example, in a factory with workers and forklifts, all workers are grouped into one category. Because workers are relatively small, the inspection robot generally won't collide with them during turning and obstacle avoidance; in this case, a safety distance of half a meter can be set. Forklifts of different models are grouped into another category. Because forklifts are larger, the inspection robot is more likely to collide with them during turning and obstacle avoidance; in this case, a safety distance of 2 meters can be set to reduce the risk of such collisions. Based on this obstacle classification mechanism, a balance can be struck between ensuring the inspection efficiency of the inspection robot and reducing the risk of collisions.

[0072] The terms "first," "second," etc., used in this article do not specifically refer to order or sequence, nor are they intended to limit this case; they are merely used to distinguish components or operations described using the same technical terms.

[0073] The terms "connection" or "positioning" as used in this article can refer to two or more components or devices making direct physical contact with each other, or making indirect physical contact with each other, or to two or more components or devices operating or moving with each other.

[0074] The terms “include,” “including,” and “have” used in this article are all open-ended, meaning they include but are not limited to.

[0075] Unless otherwise specified, the terms used herein generally have their ordinary meaning in the context of the art, the subject matter, and the specific context. Certain terms used to describe this case will be discussed below or elsewhere in this specification to provide additional guidance to those skilled in the art in describing this case.

[0076] The terms “front,” “back,” “up,” “down,” “left,” and “right” used in this article are directional terms. In this case, they are only used to describe the positional relationship between the structures and are not intended to limit the specific direction of the protection scheme or its actual implementation.

[0077] The working principle and advantages of this invention are as follows:

[0078] LiDAR can achieve 360° scanning with a single structure. By using it to scan the environmental information around the top surface of the main structure, the implementation cost of obstacle avoidance methods can be reduced, including structural costs and detection costs.

[0079] LiDAR is not very effective at identifying transparent or mirror-reflective objects, and it cannot scan environmental information below the top of the main structure. It should be noted that although the total number of transparent, mirror-reflective, and low-lying objects in the inspection environment of the inspection robot may be very small, and the inspection robot itself may be low in height to further reduce the total number of possible low-lying objects, these objects still need to be taken into account to reduce the risk of collision.

[0080] This application describes a main structure with four circumferential surfaces. Addressing the limitations of lidar, at least one ultrasonic sensor is installed on each of the four surfaces of the main structure for auxiliary scanning. The installation position of the ultrasonic sensors allows them to scan blind spots that lidar cannot cover, and to accurately scan transparent and specularly reflective objects where lidar scanning is ineffective. In other words, using ultrasonic sensors for obstacle avoidance reduces the risk of missed scans.

[0081] Ultrasonic sensors have a lower accuracy compared to lidar. To address this, the second scan data is filtered in step two to remove noise, thus mitigating the risk of missed scans. Furthermore, the first scan data is not filtered in step two, avoiding unnecessary steps and reducing the detection burden.

[0082] In step four, a multi-view camera is selectively used; that is, the multi-view camera may not be used in step one. It should be noted that visual sensors are susceptible to changes in lighting conditions, exhibiting a significant performance degradation at night. While the multi-view camera mitigates this effect compared to ordinary visual sensors, it still retains this limitation. Due to this limitation, the multi-view camera is not used to replace either the ultrasonic sensor or the lidar in step one, thus ensuring the applicability of the obstacle avoidance method in this application. Furthermore, the multi-view camera excels at identifying obstacle categories; by determining the obstacle category and generating an obstacle avoidance path accordingly, the collision risk of the inspection robot can be further reduced.

[0083] The obstacle avoidance method in this application is designed for a specific inspection robot with a main structure capable of turning in place. Considering that the multi-view structure camera is always aimed at the direction of travel of the inspection robot, only one multi-view structure camera needs to be installed on the front of the inspection robot (i.e. the front of the main structure), thereby further reducing the implementation cost of the obstacle avoidance method in this application, including structural cost and detection cost.

[0084] In summary, the obstacle avoidance method in this application is based on a specific type of inspection robot and employs a single LiDAR, multiple ultrasonic sensors, and a single multi-view camera to collaboratively achieve obstacle avoidance for this type of inspection robot. This ensures that the obstacle avoidance method can be applied to various environments, such as environments with sufficient light and shadow, environments lacking light sources, and environments with obstacles such as low-lying objects. Simultaneously, it effectively controls implementation costs and the collision risk of the inspection robot. In other words, the obstacle avoidance method in this application improves the accuracy and reliability of environmental perception by fusing complementary information from multiple sensors, thereby providing the inspection robot with a more powerful autonomous obstacle avoidance capability. Attached Figure Description

[0085] Appendix Figure 1 This is a flowchart of the obstacle avoidance method for the inspection robot according to an embodiment of the present invention. Detailed Implementation

[0086] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0087] Example: The present invention will be clearly described below with illustrations and detailed description. Any person skilled in the art who understands the examples of the present invention can make changes and modifications based on the technology taught in the present invention without departing from the spirit and scope of the present invention.

[0088] The terminology used herein is for the purpose of describing specific embodiments only and is not intended to limit the scope of this work. Singular forms such as “a,” “this,” “this,” “the,” and “the” as used herein also include plural forms.

[0089] See appendix Figure 1 An obstacle avoidance method for inspection robots based on multi-sensor fusion is applied to an inspection robot. The inspection robot includes a main structure, a lidar mounted on the top of the main structure, a multi-view camera mounted on the front of the main structure, and multiple ultrasonic sensors mounted on the main structure (preferably at the bottom, as will be described below) and arranged in a circular pattern. The main structure serves as a component capable of turning in place.

[0090] Obstacle avoidance methods include the following steps:

[0091] Step 1: Use the laser radar to scan the environmental information of the periphery of the top surface of the main structure in 360° to obtain the first scan data, and use multiple ultrasonic sensors to scan the environmental information of at least part of the area below the top of the main structure in 360° to obtain the second scan data;

[0092] Step 2: Filter the second scan data to remove noise and obtain the third scan data;

[0093] Step 3: Based on the first scan data and the third scan data, determine whether there are obstacles in the direction of travel of the main structure, and output the determination result;

[0094] Step 4: Selectively use the multi-view structure camera based on the judgment result, including:

[0095] When the judgment result indicates that there is an obstacle in the direction of travel, the multi-view structure camera is used to identify the category of the obstacle, and an obstacle avoidance algorithm is used to plan an obstacle avoidance path based on the category.

[0096] When the determination result indicates that there are no obstacles in the direction of travel, the multi-view structure camera remains off, or the information collected by the multi-view structure camera is ignored. That is, the multi-view structure camera can initially be off and then turned on when it is necessary to detect the type of obstacle; or it can initially remain on but only collect environmental information without identifying the type of obstacle.

[0097] Each detection device can transmit data in real time via Wi-Fi module, which will not be elaborated here.

[0098] LiDAR (which can use the UART communication protocol for data transmission) can achieve 360° scanning with a single structure. By using environmental information from the top surface and periphery of the scanning main structure (the scanning range of LiDAR is understood based on existing methods, which perform 360° scanning on a horizontal plane), the implementation cost of obstacle avoidance methods can be reduced, including structural and detection costs. If multiple ultrasonic sensors are also used, data from multiple ultrasonic sensors needs to be fused for judgment, increasing both structural and detection costs. 2D LiDAR is preferred to reduce costs.

[0099] LiDAR is not very effective at identifying transparent or mirror-reflective objects, and it cannot scan environmental information below the top of the main structure. It should be noted that although the total number of transparent, mirror-reflective, and low-lying objects in the inspection environment of the inspection robot may be very small, and the inspection robot itself may be low in height to further reduce the total number of possible low-lying objects, these objects still need to be taken into account to reduce the risk of collision.

[0100] The main structure, as a component capable of turning in place, may include a chassis capable of turning in place, and this can be achieved using existing methods, which will not be elaborated upon here. This configuration allows for the mounting of a multi-view camera only on the front of the main structure.

[0101] This embodiment is illustrated using a main structure with four circumferential surfaces. Due to the limitations of lidar, at least one (usually two) ultrasonic sensors are installed on each of the four surfaces of the main structure for auxiliary scanning. The installation positions of the ultrasonic sensors allow them to scan blind spots that lidar cannot cover, and to accurately scan transparent and specularly reflective objects where lidar scanning is ineffective. In other words, using ultrasonic sensors for obstacle avoidance reduces the risk of missed scans. The obstacle avoidance path is planned using existing algorithms, such as Dijkstra's algorithm. When planning the obstacle avoidance path, it is necessary to consider obstacles outside the path; therefore, the scan is a 360° scan, and attention must be paid to obstacles outside the path, including low-lying obstacles.

[0102] Ultrasonic sensors (which can transmit data using the RS485 communication protocol) have a lower accuracy compared to lidar. To address this, the second scan data is filtered in step two to remove noise (using a moving average filtering algorithm), thus appropriately mitigating the defect and further reducing the risk of missed scans. Furthermore, the first scan data is not filtered in step two, avoiding unnecessary steps and reducing the detection burden.

[0103] In step three, the presence of obstacles in the direction of travel of the main structure is determined based on the first and third scan data. The determination method can be any method well-known to those skilled in the art, and will not be elaborated upon here. The main structure has a predetermined inspection path; the main purpose is to determine whether there are obstacles along this path. It is understood that obtaining obstacle information based on either the first or third scan data indicates the presence of an obstacle in the direction of travel of the main structure.

[0104] In step four, obstacle avoidance algorithms can be used to plan obstacle avoidance paths. Here, the same methods known to those skilled in the art can be used, which will not be elaborated here.

[0105] In step four, a multi-view camera is selectively used; that is, a multi-view camera may not be used in step one. It should be noted that visual sensors are susceptible to changes in lighting conditions, exhibiting a significant performance degradation at night. While multi-view cameras mitigate this effect compared to ordinary visual sensors, the limitation still exists. Multi-view cameras can use USB for data transfer and can accurately identify obstacles using existing target detection algorithms. A binocular camera is preferred to reduce costs.

[0106] Due to the limitations of multi-view cameras, neither the ultrasonic sensor nor the lidar is replaced in step one, thus ensuring the applicability of the obstacle avoidance method in this embodiment. Multi-view cameras, however, excel at identifying obstacle categories. By determining the obstacle category and generating an obstacle avoidance path accordingly, the collision risk of the inspection robot can be further reduced. It should be noted that if the multi-view camera fails to identify the obstacle category, it can be classified as an unidentified obstacle. This mitigates the impact of lighting changes on the multi-view camera and prevents the inability to generate an obstacle avoidance path due to the camera's failure to identify the obstacle. In this case, an obstacle avoidance path can be directly generated based on the unidentified obstacle category according to a predetermined procedure, ensuring the applicability of the obstacle avoidance method in this embodiment. For some obstacles, such as small stones, they can be classified as obstacles that do not need to be avoided.

[0107] The obstacle avoidance method in this embodiment is designed for a specific inspection robot that has a main structure capable of turning in place. Considering that the multi-view camera is always aimed at the direction of travel of the inspection robot, only one multi-view camera needs to be installed on the front of the inspection robot (i.e., the front of the main structure), thereby further reducing the implementation cost of the obstacle avoidance method in this embodiment, including structural cost and detection cost.

[0108] In summary, the obstacle avoidance method in this embodiment is based on a specific type of inspection robot and employs a single LiDAR, multiple ultrasonic sensors, and a single multi-view camera to collaboratively achieve obstacle avoidance for this type of inspection robot. This ensures that the obstacle avoidance method in this embodiment can be applied to various environments, such as environments with sufficient light and shadow, environments lacking light sources, and environments with obstacles such as low-lying objects. Simultaneously, it effectively controls implementation costs and the collision risk of the inspection robot. That is, the obstacle avoidance method in this embodiment improves the accuracy and reliability of environmental perception by fusing complementary information from multiple sensors, thereby providing the inspection robot with a more powerful autonomous obstacle avoidance capability.

[0109] Furthermore, the obstacle avoidance method in this embodiment can still avoid obstacles even when a single sensor fails, improving reliability and fault tolerance, while also enhancing adaptability in environments such as day and night.

[0110] It should be noted that for ultrasonic sensors, lidar, and multi-view structure cameras, the timestamp information in the data headers of the three can be unified with the current system time to achieve time synchronization. Simultaneously, the transformation relationship between the coordinate systems of the three can be determined using the robot's center of mass as the origin, thus completing spatial calibration. This is existing technology and well-known to those skilled in the art, and will not be elaborated upon here.

[0111] It should also be noted that feature extraction and multimodal fusion can be performed on the first and third scan data. Feature extraction is as follows: extracting geometric features such as edges and corners, as well as depth features such as distance, from the LiDAR point cloud; extracting depth information from a specific area fed back by the ultrasonic sensor. Multimodal fusion is as follows: using an extended Kalman filter algorithm to fuse the extracted depth information from the LiDAR and ultrasonic sensors; associating and calibrating the point cloud information fed back by the LiDAR, such as the size and specific location of obstacles, with the one-dimensional depth information fed back by the ultrasonic sensor, and then performing feature fusion. This is not an innovation of this application; existing feature extraction and data fusion methods can be referenced.

[0112] In this embodiment, in step three, the first scan data and the third scan data are used based on the first decision fusion mechanism or the second decision fusion mechanism;

[0113] The first decision fusion mechanism includes:

[0114] First, based on the first scan data, determine whether there are obstacles in the direction of travel of the main structure, and obtain a first determination result;

[0115] If the first determination result indicates the presence of an obstacle, then the third scan data is discarded;

[0116] If the first determination result is that there is no obstacle, then the third scan data is used to determine whether there is an obstacle in the direction of travel of the main structure;

[0117] The second decision fusion mechanism includes:

[0118] Simultaneously, the first scan data and the third scan data are used to determine whether there are obstacles in the direction of travel of the main structure.

[0119] Different inspection robots are used in different environments, for example as follows:

[0120] Some inspection robots are used on complex roads (representing the first environment), where there are low obstacles such as flower beds and bricks, as well as transparent objects (such as plastic films covering equipment) or reflective objects (such as reflective signs).

[0121] Some inspection robots are used in factories with strict requirements (representing a second environment), where there are basically no low obstacles on the ground, and there are basically no transparent or reflective objects.

[0122] When the inspection robot is applied in the first environment, for risk control, especially to reduce the risk of collision, the first and third scan data need to be used based on the second decision fusion mechanism. That is, regardless of whether the obstacle is detected based on the first scan data, the third scan data needs to be used to avoid the risk that the lidar has difficulty in identifying obstacles at low altitudes, transparent obstacles, and obstacles with reflective surfaces. If the third scan data is not used synchronously, there is a greater risk of collision. Therefore, in the first environment, the focus is on reducing the risk of collision.

[0123] When the inspection robot is applied in the second environment, since there are basically no low-lying obstacles, transparent obstacles, or obstacles with reflective surfaces, the first scan data is prioritized for detection. If information about obstacles in the forward direction is obtained, the third scan data is discarded to reduce the detection burden. If no obstacle is detected, the third scan data is then detected for secondary verification, reducing the risk of collision for the inspection robot and balancing the need to reduce the detection burden (such as reducing data processing volume) with the need to reduce the risk of collision.

[0124] It should be noted that in the second environment, it is generally easier to guarantee the absence of low-lying obstacles, while the presence of transparent or reflective obstacles is relatively difficult to guarantee. For example, in factories, due to strict management, there is no scattered goods, and in such cases, low-lying obstacles are generally absent. However, transparent or reflective obstacles may exist, though they are relatively few. In some factories, the inspection robot's path is controlled to prevent obstacles from appearing, such as moving workers and forklifts. In this case, the obstacles are mainly moving workers and forklifts, and obstacles that are difficult for LiDAR to detect are generally absent. The first decision fusion mechanism is particularly well-suited to this environment, making the obstacle avoidance method in this embodiment widely applicable. It is understandable that the second environment emphasizes balancing the need to reduce detection burden with the need to reduce collision risk.

[0125] In this embodiment, in step three, the first scan data and the third scan data are used based on the first decision fusion mechanism;

[0126] In step four, after planning the obstacle avoidance path, the collision situation is detected during the obstacle avoidance process of the inspection robot. When the collision frequency exceeds the predetermined safe frequency, the second decision fusion mechanism is temporarily switched to be used within the first predetermined time period.

[0127] The collision detection method uses existing techniques and is not innovative in this application. To briefly explain: it involves deploying a collision switch, force sensor, or elastic collision ring with a Hall effect sensor on the device body to collect contact signals or sudden force / torque signals in real time, which are then used to determine the collision frequency based on threshold values. The time period used to reference the collision frequency will be determined according to actual requirements.

[0128] The first scheduled time will be confirmed based on actual needs, and there is no specific time limit. The same applies to similar settings.

[0129] To facilitate understanding, based on the above explanation, we would like to supplement this by taking a factory as an example. In many factories, due to strict management, there is no situation where goods are scattered or disorderly. In such cases, there are basically no low-lying obstacles. However, these factories may have transparent obstacles or obstacles with reflective surfaces, but such obstacles account for a relatively small percentage. Based on this, we will explain in detail as follows:

[0130] There is a risk of missing obstacles. For example, if there is an obstacle at 10 meters that is detected by the lidar, but there is also a reflective obstacle at 5 meters, the obstacle will be missed because the third scanning data is not used, resulting in an unexpected collision. When the frequency of such unexpected collisions reaches a predetermined threshold (i.e., a predetermined safe frequency), such as two unexpected collisions occurring during a one-hour inspection, the system will temporarily switch to the second decision fusion mechanism to meet the current need to reduce the risk of collisions.

[0131] In some embodiments, in step four, collision information is collected, and when the collision frequency exceeds a predetermined safe frequency, an alarm is issued to remind staff to conduct an inspection, such as checking for reflective obstacles to be dealt with, thereby reducing the risk of collision.

[0132] In some embodiments, if the first judgment result is that there is no obstacle, but the third scan data determines that there is an obstacle in the direction of travel of the main structure, and the frequency of this situation reaches the set frequency, it indicates that the total number of low-level obstacles, transparent obstacles and surface reflective obstacles in the inspection environment exceeds the expected number. At this time, the second decision fusion mechanism is temporarily switched to be used, and an alarm can be set to remind staff to handle the situation and reduce the risk of collision.

[0133] It should be noted that the inspection robot can be configured to avoid obstacles based on the estimated environmental conditions or experimental conditions, using either the first or second decision fusion mechanism. However, to avoid errors in the estimated conditions, a switching mechanism should be set for the decision fusion mechanism so that the inspection robot can adapt to the actual situation of the inspection environment and reduce the risk of collision.

[0134] In this embodiment, during the temporary switch to the second decision fusion mechanism, if an obstacle is identified in the direction of travel of the inspection robot based solely on the third scanning data, the remaining switching time is extended to match the first predetermined time.

[0135] The robot temporarily switches to the second decision fusion mechanism for a first predetermined time period, during which a timer is run. If, during this period, an obstacle is detected in the robot's direction of travel based solely on the third scan data, it is determined that the robot's current environment still carries a higher-than-expected collision risk. Therefore, the timer is reset, and the second decision fusion mechanism is used again for the subsequent first predetermined time period, with the process repeating thereafter. This timer reset mechanism further enables the inspection robot to adapt to the actual conditions of the inspection environment, thereby further reducing the collision risk.

[0136] In this embodiment, in step three, the first scan data and the third scan data are used based on the second decision fusion mechanism. If the recognition result obtained based on the first scan data and the recognition result obtained based on the third scan data are consistent within a second predetermined time, then the first decision fusion mechanism is temporarily switched to be used in the subsequent third predetermined time.

[0137] The recognition result here refers to the recognition result of obstacles, such as both identifying obstacle one and obstacle two.

[0138] For example, during the timing process, if the recognition results based on the first scan data and the recognition results based on the third scan data are consistent within ten minutes, then the first decision fusion mechanism will be temporarily switched to be used for the next five minutes. If the two recognition results are inconsistent at the sixth minute, the timer will be reset and restarted.

[0139] If the recognition results obtained based on the first scan data and the recognition results obtained based on the third scan data are consistent within the second predetermined time, it indicates that the inspection environment is relatively safe, and it is not necessary to simultaneously use the third scan data to reduce the risk of collision, thereby reducing the detection burden. By setting a switching mechanism for the decision fusion mechanism, the inspection robot can adapt to the actual situation of the inspection environment, balancing the need to reduce the detection burden and the need to reduce the risk of collision.

[0140] In this embodiment, during the temporary switch to the first decision fusion mechanism, the collision situation of the inspection robot is detected during obstacle avoidance. If the collision frequency is consistently lower than the predetermined safe frequency, the time for subsequent use of the first decision fusion mechanism is extended (i.e., the remaining switching time is extended) to make it consistent with the third predetermined time.

[0141] It can be done by directly determining whether the collision frequency during the process of switching to the first decision fusion mechanism is lower than the predetermined safe frequency. Alternatively, the time period can be divided into multiple sub-time periods for judgment.

[0142] Temporarily switching to the first decision fusion mechanism indicates that the current inspection environment is considered relatively safe. To adapt to potential changes in the inspection environment, collision detection is performed during the obstacle avoidance process of the inspection robot. Obstacle detection may have a risk of missed detection. For example, an obstacle may be detected by the lidar at 10 meters, but another obstacle with a reflective surface may be detected at 5 meters. Because the third scan data was not used, this leads to a missed detection and an unexpected collision. If the frequency of unexpected collisions within the third predetermined time period remains below the predetermined safe frequency (e.g., the number of collisions is less than 2 in any hour), the current inspection environment can be considered relatively safe. In this case, the time for using the first decision fusion mechanism is extended to align with the third predetermined time, further balancing the need to reduce the detection burden and the need to reduce collision risk based on the actual inspection environment.

[0143] It can be set to immediately switch back to the third decision fusion mechanism if a collision occurs during the switching process, and there are no specific restrictions.

[0144] In this embodiment, in step one, a preset safe distance is established;

[0145] In step three, when the judgment result indicates that there is an obstacle in the direction of travel, the distance between the main structure and the obstacle is measured to obtain the obstacle avoidance distance;

[0146] In step four, if the obstacle avoidance distance is less than the safe distance, the main structure is immediately stopped and reversed in place, and then obstacle avoidance is performed based on the obstacle avoidance path; if the obstacle avoidance distance is greater than or equal to the safe distance, the movement state of the main structure is maintained and obstacle avoidance is performed based on the obstacle avoidance path.

[0147] The ranging method is existing and is not an innovation of this application. For example, it can be implemented based on a multi-view camera structure, with binocular vision simulating human eye parallax, and calculating the three-dimensional distance by matching the pixels of two images.

[0148] It should be noted that obstacles refer to those in the direction of travel. Obstacles not initially in the direction of travel are only considered obstacles requiring obstacle avoidance if they subsequently move into that direction. Obstacles not initially in the direction of travel are considered obstacles that must be avoided along the planned obstacle avoidance path. If multiple obstacles exist in the direction of travel, the nearest obstacle is used as the reference obstacle for the obstacle avoidance distance.

[0149] For example, the safety distance is set to one meter. If the obstacle avoidance distance is less than one meter, it is considered that the obstacle avoidance space is insufficient. To avoid colliding with obstacles during obstacle avoidance, the inspection robot first stops, then turns in place to avoid moving towards the obstacle, and then performs obstacle avoidance to cross the obstacle and return to the predetermined inspection path. If the obstacle avoidance distance is greater than or equal to one meter, it is considered that the obstacle avoidance space is sufficient. The inspection robot can continue moving and directly perform obstacle avoidance, thereby shortening the obstacle avoidance time and ensuring the inspection efficiency of the inspection robot.

[0150] In this embodiment, in step one, several obstacles are pre-divided into different categories. Obstacles of the same category correspond to the same safe distance, while obstacles of different categories correspond to different safe distances.

[0151] For example, in a factory with workers and forklifts, all workers are grouped into one category. Because workers are relatively small, the inspection robot generally won't collide with them during turning and obstacle avoidance; in this case, a safety distance of half a meter can be set. Forklifts of different models are grouped into another category. Because forklifts are larger, the inspection robot is more likely to collide with them during turning and obstacle avoidance; in this case, a safety distance of 2 meters can be set to reduce the risk of such collisions. Based on this obstacle classification mechanism, a balance can be struck between ensuring the inspection efficiency of the inspection robot and reducing the risk of collisions.

[0152] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. An obstacle avoidance method for inspection robots based on multi-sensor fusion, characterized in that: This invention relates to an inspection robot, which includes a main structure, a lidar mounted on the top of the main structure, a multi-view camera mounted on the front of the main structure, and multiple ultrasonic sensors mounted on the main structure in a circular arrangement; the main structure serves as a component capable of turning in place. Obstacle avoidance methods include the following steps: Step 1: Use the laser radar to scan the environmental information of the periphery of the top surface of the main structure in 360° to obtain the first scan data, and use multiple ultrasonic sensors to scan the environmental information of at least part of the area below the top of the main structure in 360° to obtain the second scan data; Step 2: Filter the second scan data to remove noise and obtain the third scan data; Step 3: Based on the first scan data and the third scan data, determine whether there are obstacles in the direction of travel of the main structure, and output the determination result; Step 4: Selectively use the multi-view structure camera based on the judgment result, including: When the judgment result indicates that there is an obstacle in the direction of travel, the multi-view structure camera is used to identify the category of the obstacle, and an obstacle avoidance algorithm is used to plan an obstacle avoidance path based on the category. When the determination result indicates that there are no obstacles in the direction of travel, the multi-view structure camera remains off, or the information collected by the multi-view structure camera is ignored.

2. The obstacle avoidance method for an inspection robot based on multi-sensor fusion according to claim 1, characterized in that: In step three, the first scan data and the third scan data are used based on the first decision fusion mechanism or the second decision fusion mechanism; The first decision fusion mechanism includes: First, based on the first scan data, determine whether there are obstacles in the direction of travel of the main structure, and obtain a first determination result; If the first determination result indicates the presence of an obstacle, then the third scan data is discarded; If the first determination result is that there is no obstacle, then the third scan data is used to determine whether there is an obstacle in the direction of travel of the main structure; The second decision fusion mechanism includes: Simultaneously, the first scan data and the third scan data are used to determine whether there are obstacles in the direction of travel of the main structure.

3. The obstacle avoidance method for an inspection robot based on multi-sensor fusion according to claim 2, characterized in that: In step three, the first scan data and the third scan data are used based on the first decision fusion mechanism; In step four, after planning the obstacle avoidance path, the collision situation is detected during the obstacle avoidance process of the inspection robot. When the collision frequency exceeds the predetermined safe frequency, the second decision fusion mechanism is temporarily switched to be used within the first predetermined time period.

4. The obstacle avoidance method for an inspection robot based on multi-sensor fusion according to claim 3, characterized in that: During the temporary switch to the second decision fusion mechanism, if an obstacle is detected in the direction of travel of the inspection robot based solely on the third scan data, the remaining switching time is extended to match the first predetermined time.

5. The obstacle avoidance method for an inspection robot based on multi-sensor fusion according to claim 2, characterized in that: In step three, the first scan data and the third scan data are used based on the second decision fusion mechanism. If the recognition result obtained based on the first scan data and the recognition result obtained based on the third scan data are consistent within a second predetermined time, the first decision fusion mechanism will be temporarily switched to be used within a subsequent third predetermined time.

6. The obstacle avoidance method for an inspection robot based on multi-sensor fusion according to claim 5, characterized in that: During the temporary switch to the first decision fusion mechanism, the collision situation of the inspection robot is detected during obstacle avoidance. If the collision frequency is consistently lower than the predetermined safe frequency, the remaining switching time is extended to make it consistent with the third predetermined time.

7. A method for obstacle avoidance of an inspection robot based on multi-sensor fusion according to any one of claims 1-6, characterized in that: In step one, a safe distance is preset; In step three, when the judgment result indicates that there is an obstacle in the direction of travel, the distance between the main structure and the obstacle is measured to obtain the obstacle avoidance distance; In step four, if the obstacle avoidance distance is less than the safe distance, the main structure is immediately stopped and reversed in place, and then obstacle avoidance is performed based on the obstacle avoidance path; if the obstacle avoidance distance is greater than or equal to the safe distance, the movement state of the main structure is maintained and obstacle avoidance is performed based on the obstacle avoidance path.

8. The obstacle avoidance method for an inspection robot based on multi-sensor fusion according to claim 7, characterized in that: In step one, several obstacles are pre-classified into different categories. Obstacles of the same category correspond to the same safe distance, while obstacles of different categories correspond to different safe distances.

Images