Substation inspection robot multi-sensor fusion obstacle avoidance method and system

Abstract

The invention belongs to the technical field of substation inspection, and provides a substation inspection robot multi-sensor fusion obstacle avoidance method and system, and the technical scheme is as follows: obtaining multi-modal sensor data, including laser point cloud data, millimeter wave radar point cloud data, ultrasonic radar observation data and visual image data; preprocessing the obtained multi-modal sensor data to obtain effective observation data, and determining initial obstacle data based on the effective observation data; time synchronization and fusion are carried out on the obtained multiple kinds of initial obstacle data, the fused data serve as obstacle data, scene segmentation is carried out in combination with multiple kinds of sensor data in the fusion process, confidence coefficient decision rules are dynamically set according to different scenes, and obstacle information is determined in combination with the set confidence coefficient decision rules; and generating an optimal obstacle avoidance route based on the determined obstacle information. The problem that a traditional single sensor cannot cope with a complex scene of substation inspection business is solved, and the overall stable operation capacity of the robot is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of substation inspection, and in particular relates to a multi-sensor fusion obstacle avoidance method and system for a substation inspection robot. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] As substation inspection services mature, the requirements for robot attendance and effective inspection rates have increased significantly. Existing technical solutions are unable to cope with the complex operational scenarios of substation inspections, requiring robots to be operational 24 / 7. Visual cameras cannot effectively perceive objects at night, and lidar cannot effectively perceive objects in inclement weather such as rain, snow, fog, and dust. Multi-sensor fusion is urgently needed for obstacle perception to meet inspection needs.

[0004] Existing methods for obstacle perception using sensor fusion in inspection fields do not consider whether the data acquired in complex scenarios and inclement weather is valid perception. Instead, all acquired data is directly fused and processed. For example, preliminary fusion results of visual data, preliminary fusion results of lidar data, and preliminary fusion results of ultrasonic data are directly used to perform global fusion of multi-sensor data to obtain the final obstacle information around the robot. Another example is extracting obstacle information based on depth camera images, single-line lidar point clouds, and ultrasonic radar data, and then fusing the obstacle information and presenting it in a bird's-eye view. This also directly obtains obstacle information by fusing various acquired data.

[0005] However, in actual inspections, faced with complex scenarios and inclement weather, existing sensors have limitations based on their perception principles, resulting in distortion in some scenarios. For example, visual cameras and lidar are prone to false detections in rain, snow, fog, haze, and dust. Visual sensors also have specific lighting requirements; strong sunlight at night or midday can affect visual data. Ultrasonic sensors are only effective at detecting nearby obstacles. Neither of these two approaches can meet the year-round, all-weather patrol needs of outdoor substation inspection robots. Summary of the Invention

[0006] In order to solve at least one technical problem existing in the above-mentioned background technology, the present invention provides a multi-sensor fusion obstacle avoidance method and system for a substation inspection robot, which can cope with various complex scenarios and severe weather and ensure the safety and stability of the robot during operation.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] A first aspect of the present invention provides a multi-sensor fusion obstacle avoidance method for a substation inspection robot, comprising the following steps:

[0009] Acquire multimodal sensor data, including laser point cloud data, millimeter wave radar point cloud data, ultrasonic radar observation data, and visual image data;

[0010] Preprocessing the acquired multimodal sensor data to obtain valid observation data, and determining initial obstacle data based on the valid observation data;

[0011] The acquired initial obstacle data is synchronized and fused, and the fused data is used as obstacle data. During the fusion process, scene segmentation is performed by combining multiple sensor data. Confidence decision rules are dynamically set according to different scenarios, and obstacle information is determined based on the set confidence decision rules.

[0012] Generate the optimal obstacle avoidance route based on the determined obstacle information.

[0013] Furthermore, the acquired millimeter-wave radar point cloud data is preprocessed to obtain valid observation data, including: comparing the actual observation value of the millimeter-wave radar with the predicted state, calculating the uncertainty matrix of the state estimation and the uncertainty matrix of the measurement value, calculating the Kalman gain by combining the uncertainty matrix of the state estimation and the uncertainty matrix of the measurement value, and using the Kalman gain to adjust the predicted state to obtain the updated state estimate as the valid observation data.

[0014] Furthermore, the fusion process combines multiple sensor data to perform scene segmentation, including:

[0015] The scene is determined based on the laser point cloud density information. If the laser point cloud density change rate is less than the set first density threshold, the scene is a normal scene.

[0016] If the laser point cloud density is greater than the set first density change rate threshold and less than the set second density change rate threshold, the scene is a general severe weather scene;

[0017] If the laser point cloud density is greater than the set second density change rate threshold, the scene is a severely harsh scene.

[0018] Furthermore, the calculation formula of laser point cloud density is:

[0019]

[0020] Among them, N(d) represents the point cloud density at distance d, N0 is the starting point cloud density, σ extis the extinction coefficient, N(d) and N0 are known quantities, and the absolute value of the first-order derivative |N′(d)| represents the rate of change of the point cloud density, that is, the degree of influence of weather on lidar observations.

[0021] Furthermore, in conventional scenarios, the LiDAR point cloud data is used as the main sensor for obstacle perception, combined with the ultrasonic radar and depth camera observation data for blind spot observation of nearby small obstacles;

[0022] In severe weather, the absolute value of the first-order derivative of the laser point cloud density image is used to characterize the degree of change in the point cloud density. The confidence level of the lidar point cloud data is set based on the absolute value of the first-order derivative. The weight of each sensor data is dynamically adjusted based on the confidence level result. The lidar point cloud data, ultrasonic radar observation data, depth camera perception, and millimeter-wave radar point cloud data are integrated to determine obstacle information.

[0023] Furthermore, if the robot collides with an obstacle, a certain edge sensor located in a set direction is triggered, and the movement direction of the robot that can safely escape is obtained based on the triggering direction of the edge sensor.

[0024] A second aspect of the present invention provides a multi-sensor fusion obstacle avoidance control system for a substation inspection robot, comprising:

[0025] A data acquisition module is used to acquire multimodal sensor data, including laser point cloud data, millimeter wave radar point cloud data, ultrasonic radar observation data, and visual image data;

[0026] An initial obstacle determination module is used to preprocess the acquired multimodal sensor data to obtain valid observation data, and determine initial obstacle data based on the valid observation data;

[0027] The obstacle information determination module is used to synchronize and fuse the acquired initial obstacle data, using the fused data as obstacle data. During the fusion process, the module combines the data from various sensors to segment the scene, dynamically sets confidence decision rules based on different scenarios, and determines obstacle information based on the set confidence decision rules.

[0028] The obstacle avoidance strategy generation module is used to generate the optimal obstacle avoidance route based on the determined obstacle information.

[0029] A third aspect of the present invention provides a computer-readable storage medium.

[0030] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the multi-sensor fusion obstacle avoidance control method for a substation inspection robot as described above.

[0031] A fourth aspect of the present invention provides a computer device.

[0032] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the multi-sensor fusion obstacle avoidance control method for a substation inspection robot as described above are implemented.

[0033] A fifth aspect of the present invention is a program product.

[0034] A program product, which is a computer program product, includes a computer program. When the computer program is executed by a processor, it implements the steps in the multi-sensor fusion obstacle avoidance control method for a substation inspection robot as described above.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] This invention proposes a multi-sensor fusion obstacle avoidance method for substation inspection robots. This method uses valid multimodal sensor data and predefined confidence judgment rules to derive optimal combined data for determining obstacle information. This method then determines obstacle information based on this optimal combined data. This method addresses the inability of traditional single sensors to handle the complex scenarios of substation inspections. The method can complete inspections in adverse weather conditions such as rain, snow, fog, and dust, allowing the robot to return to the charging room smoothly, preventing it from being trapped outside the charging room until its battery runs out. This method improves the robot's overall operational stability.

[0037] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0039] Figure 1 This is a flow chart of a multi-sensor fusion obstacle avoidance control method for a substation inspection robot provided by an embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram of a grid occupied by a single lidar point cloud under a costmap in severe weather provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0042] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0043] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0044] Example 1

[0045] like Figure 1 As shown, this embodiment provides a multi-sensor fusion obstacle avoidance control method for a substation inspection robot, including the following steps:

[0046] S101: Acquire historical laser point cloud data, millimeter wave radar point cloud data, ultrasonic radar observation data, and visual image data;

[0047] In this embodiment, multiple sensors are set in the robot body, including laser radar, millimeter wave radar, ultrasonic radar, depth camera and edge sensor, and corresponding historical laser point cloud data, millimeter wave radar point cloud data, ultrasonic radar observation data and visual image data are obtained according to the set sensors respectively;

[0048] S102: Preprocessing the acquired laser point cloud data, millimeter wave radar point cloud data, ultrasonic radar observation data, and visual image data to obtain valid observation data of each sensor, and determining initial obstacle data based on the valid observation data;

[0049] The specific steps include:

[0050] S201, preprocessing the acquired laser point cloud data;

[0051] Specifically include:

[0052] The first step is to remove isolated noise points or "flying points" by statistical filtering;

[0053] In the second step, the point cloud data is downsampled by voxel filtering to reduce the data volume and smooth the data;

[0054] The third step is to obtain point cloud data with Euclidean distance in the range of 0.5 to 6 meters in the xy direction through conditional filtering;

[0055] The fourth step is to convert the point cloud data from the liadr coordinate system to the robot body coordinate system;

[0056] The fifth step is to filter out the point cloud on the ground through the ground filtering algorithm;

[0057] The sixth step is to convert the point cloud data into the map coordinate system to obtain the real-time obstacle data initially acquired by the lidar.

[0058] S202, preprocessing the acquired millimeter-wave radar point cloud data;

[0059] Since the amount of millimeter-wave radar point cloud data is small, an extended Kalman filter (EKF) is used to filter the point cloud data to help smooth the target trajectory. The specific process includes:

[0060] S2021. Initialize the state quantity X0 according to the initial observation data of the sensor;

[0061] The state vector X0 is the core vector that describes the current state of the system. In millimeter-wave radar point cloud data processing, the state vector includes information such as the position and speed of the tracked target, which can be expressed as:

[0062] X0=[x,y,v x ,v y ]T,

[0063] Among them, x, y represent the initial position of the target, v x ,v y Indicates the initial velocity of the target;

[0064] S2022,

[0065] The actual observation value of the millimeter-wave radar is compared with the predicted state, and the uncertainty matrix of the state estimation and the uncertainty matrix of the measurement value are calculated. The Kalman gain is calculated by combining the uncertainty matrix of the state estimation and the uncertainty matrix of the measurement value. The predicted state is adjusted using the Kalman gain to obtain the updated state estimate as the valid observation data.

[0066] Specifically, the covariance matrix is ​​initialized, and the covariance matrix P0 represents the uncertainty of the state estimation.

[0067] For the state vector X0=[x,y,v x ,v y ]T, the corresponding T covariance matrix P0 can be expressed as:

[0068]

[0069] in, and represents the uncertainty of the position estimate, and represents the uncertainty of the velocity estimate.

[0070] It should be noted that the smaller the P0 value is, the closer the predicted state is to the actual observed value.

[0071] The measurement noise covariance matrix is ​​initialized. The measurement noise covariance matrix R describes the uncertainty of the sensor measurement value and is expressed as:

[0072]

[0073] in, represents the variance of the ranging error, Indicates the variance of the angle error. The smaller the R value, the more accurate the sensor measurement value.

[0074] Matrices P0 and R are important parameters of the extended Kalman filter, which interact indirectly by determining the Kalman gain to achieve the optimal state estimation.

[0075] S2023. Convert the error-adjusted millimeter-wave radar point cloud data to obtain real-time obstacle data.

[0076] Conditional filtering is used to obtain point cloud data with a Euclidean distance between 0.5 and 6 meters in the xy direction. This point cloud data is then converted from the millimeter-wave coordinate system to the robot's body coordinate system. Because millimeter-wave radar data is relatively sparse, these points are clustered using the DBSCAN algorithm to identify obstacles. Finally, the point cloud data is converted to the map coordinate system to obtain preliminary real-time obstacle data acquired by the millimeter-wave radar.

[0077] S203, preprocessing ultrasonic radar observation data;

[0078] Ultrasonic data can be filtered to remove high and low frequency noise, and the perception data in the distance range of 0.2 to 0.5 m in front of the robot can be obtained.

[0079] S204, pre-processing the depth camera data;

[0080] In the first step, the depth camera data is subjected to median filtering to remove noise. In the second step, the depth map and RGB image are aligned. In the third step, only valid data intervals are retained by setting a distance detection threshold between 0.5m and 3m. In the fourth step, the depth information is converted from the camera coordinate system to the robot body coordinate system. In the fifth step, the ground is fitted using the RANSAC algorithm and the ground point cloud is filtered out. In the sixth step, the depth information data is converted to the map coordinate system to obtain the real-time obstacle data initially acquired by the depth camera.

[0081] S3: Time-synchronize and fuse the acquired obstacle data, using the fused data as obstacle data. During the fusion process, combine the data from multiple sensors to segment the scene. Dynamically set confidence decision rules based on different scenarios, and determine obstacle information based on the set confidence decision rules.

[0082] In this embodiment, the fusion process combines multiple sensor data to perform scene segmentation, specifically including:

[0083] The scene is determined based on the laser point cloud density change rate information. If the laser point cloud density change rate is less than the set first density change rate threshold, the scene is a normal scene, such as clear weather.

[0084] If the laser point cloud density change rate is greater than the set first density change rate threshold, the scene is severe weather, such as heavy rain, heavy snow, fog and haze, and dust weather;

[0085] If the laser point cloud density change rate is greater than the set first density change rate threshold and less than the set second density change rate threshold, the scene is a normal severe weather scene; if the laser point cloud density is greater than the set second density change rate threshold, the scene is a severe severe weather scene;

[0086] Specifically, the calculation formula for the laser point cloud density is:

[0087]

[0088] Among them, N(d) represents the point cloud density at distance d, N0 is the starting point cloud density, σ ext is the extinction coefficient, N(d) and N0 are known quantities, and the absolute value of the first-order derivative |N′(d)| can characterize the rate of change of point cloud density, that is, the degree of influence of weather on lidar observations;

[0089] Specifically, when the extinction coefficient is lower than the first extinction coefficient threshold, it is a normal scenario; when the extinction coefficient is between the first extinction coefficient threshold and the second extinction coefficient threshold, it is a general impact scenario, and the laser radar's detection of obstacles begins to be distorted, which is a slightly impacted scenario; when the extinction coefficient reaches above the second extinction coefficient threshold, the laser radar's detection of obstacles is seriously distorted, which is a severely impacted scenario.

[0090] In this embodiment, the first extinction coefficient threshold is 0.01498m -1 , the corresponding point cloud density change rate at 6m |N′(6)|=0.01369N0, and the second extinction coefficient threshold is 0.1498m -1 , the corresponding point cloud density change rate at 6m |N′(6)|=0.06097N0;

[0091] In this embodiment, confidence decision rules are dynamically set according to different scenarios, and obstacle information is determined in combination with the set confidence decision rules, specifically including:

[0092] The absolute value of the first-order derivative of the laser point cloud density function is used to characterize the degree of change in the point cloud density. The confidence of the lidar point cloud data is set according to the absolute value of the first-order derivative, and the weight of each sensor data is dynamically adjusted based on the confidence result.

[0093] In conventional scenarios, the confidence level of the lidar point cloud data is high. The lidar point cloud data is used as the main sensor for obstacle perception and integrated with the millimeter-wave radar data. The ultrasonic radar and depth camera observation data are combined to provide blind spot observation for nearby small obstacles. For example, the ultrasonic radar uses observation data within 0.5m as blind spot observation for the lidar in the near distance to prevent obstacles from suddenly entering the robot's near perception blind spot, thus making up for the defect that the lidar and millimeter-wave radar cannot observe some small obstacles.

[0094] For example, in a conventional scenario, the decision module uses lidar point cloud data as the primary sensor for obstacle perception with a confidence level of 90%, and millimeter-wave radar obstacle perception data with a confidence level of 10%. Ultrasonic radar and depth camera observation data are then combined to provide blind spot detection for nearby small obstacles.

[0095] In bad weather, the system dynamically integrates lidar point cloud data, ultrasonic radar observation data, depth camera perception, and millimeter-wave radar point cloud data to determine obstacle information.

[0096] For example, in a generally harsh scenario, the decision module sets the confidence level of the lidar obstacle perception data to 50%, and the confidence level of the millimeter-wave radar obstacle perception data to 50%, and combines the ultrasonic radar and depth camera observation data as blind spot observations for nearby small obstacles;

[0097] If it is a severe scene, the data of lidar, ultrasonic wave and visible light camera are almost invalid. The decision module sets the confidence of lidar obstacle perception data to 10% and the confidence of millimeter wave radar obstacle perception data to 90%. In this application, scene segmentation is performed based on a large amount of collected sensor data. When the lidar data indicates that there is a large amount of obstacle information within the range of 0.5 to 6m, and it is inconsistent with the obstacle information represented by the millimeter wave radar data, the sensor confidence value is obtained according to the laser point cloud density change rate information. The absolute value of the first-order derivative of the function graph represents the degree of change in point cloud density. A larger absolute value indicates a rapid decrease in point cloud density, indicating a greater impact of weather on lidar observations. Therefore, the confidence level of the lidar observation data can be set based on the absolute value of the first-order derivative. Due to physical properties, the confidence level of the depth camera's perception data also decreases. In this case, the confidence level of the millimeter-wave radar is increased, leveraging the high penetration of millimeter-wave radar for obstacle detection in inclement weather. The relatively small amount of millimeter-wave radar data serves only as dynamic compensation for distortion in lidar and depth camera perception data during inclement weather. Finally, obstacle data is generated by dynamically fusing data from the four sensors based on different scenarios and updated in the costmap.

[0098] S4: Generate an optimal obstacle avoidance route based on the determined obstacle information;

[0099] In this embodiment, after obtaining the obstacle information in step 3, the optimal obstacle avoidance route can be generated by combining it with the existing obstacle avoidance route planner;

[0100] Existing obstacle avoidance path planners, such as those using TEB (Time Elastic Band), modify the global trajectory to optimize the robot's local motion trajectory, a form of local path planning. During trajectory optimization, the algorithm has multiple optimization objectives, including but not limited to: overall path length, trajectory runtime, distance to obstacles, passing intermediate path points, and compliance with the robot's dynamic, kinematic, and geometric constraints.

[0101] Furthermore, if the robot collides with an obstacle, one of the four edge sensors is triggered, and the direction in which the robot can safely escape is obtained based on the direction in which the edge sensor is triggered.

[0102] Physical edge sensors act as a last line of defense to prevent the robot from collisions.

[0103] When the ultrasonic signal is triggered, the robot slows down and the planner makes a sharp turn to bypass the obstacle. When the edge touch signal is triggered, the robot retreats 0.5m in the opposite direction within a set time, such as 3 seconds, to escape.

[0104] Specifically, edge sensors are located in the robot's four directions: front left, front right, rear left, and rear right. When a sensor is continuously triggered, the velocity component in that direction is locked to zero to prevent the robot's motor from exerting force in the event of an accidental collision, potentially damaging the robot and the equipment. The linear and angular velocities transmitted by the robot's motion control module are integrated to generate a trajectory within 0.5 meters. When an edge signal is triggered, the navigation planner plans a reverse escape route based on the trajectory within 0.5 meters, allowing the robot to escape safely.

[0105] Through the above series of obstacle avoidance measures, the robot can successfully complete the inspection work and return to the charging room in severe weather conditions, preventing the robot from being trapped in the maintenance area and running out of power.

[0106] Example 2

[0107] This embodiment provides a multi-sensor fusion obstacle avoidance system for a substation inspection robot, including:

[0108] A data acquisition module is used to acquire multimodal sensor data, including laser point cloud data, millimeter wave radar point cloud data, ultrasonic radar observation data, and visual image data;

[0109] An initial obstacle determination module is used to preprocess the acquired multimodal sensor data to obtain valid observation data, and determine initial obstacle data based on the valid observation data;

[0110] The obstacle information determination module is used to synchronize and fuse the acquired initial obstacle data, using the fused data as obstacle data. During the fusion process, the module combines the data from various sensors to segment the scene, dynamically sets confidence decision rules based on different scenarios, and determines obstacle information based on the set confidence decision rules.

[0111] The obstacle avoidance strategy generation module is used to generate the optimal obstacle avoidance route based on the determined obstacle information.

[0112] It should be noted that the specific implementation method of the multi-sensor fusion obstacle avoidance system of the substation inspection robot in an embodiment of the present invention is similar to the specific implementation method of the multi-sensor fusion obstacle avoidance method of the substation inspection robot in an embodiment of the present invention. Please refer to the description of the method part for details. In order to reduce redundancy, it will not be repeated here.

[0113] Example 3

[0114] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in the multi-sensor fusion obstacle avoidance of the substation inspection robot as described above are implemented.

[0115] Example 4

[0116] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the multi-sensor fusion obstacle avoidance of the substation inspection robot as described above are implemented.

[0117] Example 5

[0118] This embodiment provides a program product, which is a computer program product and includes a computer program. When the computer program is executed by a processor, the steps in the multi-sensor fusion obstacle avoidance of the substation inspection robot as described above are implemented.

[0119] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0120] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0121] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0122] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0123] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0124] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A multi-sensor fusion obstacle avoidance method for a substation inspection robot, characterized in that: The steps include: Acquire multimodal sensor data, including laser point cloud data, millimeter wave radar point cloud data, ultrasonic radar observation data, and visual image data; Preprocessing the acquired multimodal sensor data to obtain valid observation data, and determining initial obstacle data based on the valid observation data; The acquired initial obstacle data is synchronized and fused, and the fused data is used as obstacle data. During the fusion process, scene segmentation is performed by combining multiple sensor data. Confidence decision rules are dynamically set according to different scenarios, and obstacle information is determined based on the set confidence decision rules. Generate the optimal obstacle avoidance route based on the determined obstacle information.

2. The multi-sensor fusion obstacle avoidance method for a substation inspection robot according to claim 1, characterized in that: The acquired millimeter-wave radar point cloud data is preprocessed to obtain valid observation data, including: comparing the actual observation value of the millimeter-wave radar with the predicted state, calculating the uncertainty matrix of the state estimation and the uncertainty matrix of the measurement value, calculating the Kalman gain by combining the uncertainty matrix of the state estimation and the uncertainty matrix of the measurement value, and using the Kalman gain to adjust the predicted state to obtain the updated state estimate as the valid observation data.

3. The multi-sensor fusion obstacle avoidance method for a substation inspection robot according to claim 1, characterized in that: The fusion process combines multiple sensor data to perform scene segmentation, including: The scene is determined based on the laser point cloud density information. If the laser point cloud density change rate is less than the set first density threshold, the scene is a normal scene. If the laser point cloud density is greater than the set first density change rate threshold and less than the set second density change rate threshold, the scene is a general severe weather scene; If the laser point cloud density is greater than the set second density change rate threshold, the scene is a severely harsh scene.

4. The multi-sensor fusion obstacle avoidance method for a substation inspection robot according to claim 3, characterized in that: The calculation formula for laser point cloud density is: N(d)=N0·e -σextd , Among them, N(d) represents the point cloud density at distance d, N0 is the starting point cloud density, σ ext is the extinction coefficient, N(d) and N0 are known quantities, and the absolute value of the first-order derivative |N′(d)| represents the rate of change of the point cloud density, that is, the degree of influence of weather on lidar observations.

5. The multi-sensor fusion obstacle avoidance method for a substation inspection robot according to claim 1, characterized in that: In conventional scenarios, the LiDAR point cloud data is used as the main sensor for obstacle perception, combined with the ultrasonic radar and depth camera observation data for blind spot observation of nearby small obstacles; In severe weather, the absolute value of the first-order derivative of the laser point cloud density image is used to characterize the degree of change in the point cloud density. The confidence level of the lidar point cloud data is set based on the absolute value of the first-order derivative. The weight of each sensor data is dynamically adjusted based on the confidence level result. The lidar point cloud data, ultrasonic radar observation data, depth camera perception, and millimeter-wave radar point cloud data are integrated to determine obstacle information.

6. The multi-sensor fusion obstacle avoidance method for a substation inspection robot according to claim 4, characterized in that: If the robot collides with an obstacle, a certain edge sensor located in a set direction is triggered, and the movement direction in which the robot can safely escape is obtained based on the trigger direction of the edge sensor.

7. The multi-sensor fusion obstacle avoidance system of the substation inspection robot is characterized by: include: A data acquisition module is used to acquire multimodal sensor data, including laser point cloud data, millimeter wave radar point cloud data, ultrasonic radar observation data, and visual image data; An initial obstacle determination module is used to preprocess the acquired multimodal sensor data to obtain valid observation data, and determine initial obstacle data based on the valid observation data; The obstacle information determination module is used to synchronize and fuse the acquired initial obstacle data, using the fused data as obstacle data. During the fusion process, the module combines the data from various sensors to segment the scene, dynamically sets confidence decision rules based on different scenarios, and determines obstacle information based on the set confidence decision rules. The obstacle avoidance strategy generation module is used to generate the optimal obstacle avoidance route based on the determined obstacle information.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the multi-sensor fusion obstacle avoidance method for a substation inspection robot are implemented as described in any one of claims 1 to 7.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the multi-sensor fusion obstacle avoidance method for a substation inspection robot are implemented as described in any one of claims 1 to 7.

10. A program product, wherein the program product is a computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the multi-sensor fusion obstacle avoidance method for a substation inspection robot are implemented as described in any one of claims 1 to 7.

Images