A kind of image processing and environmental perception system of inspection robot adapting to extreme weather

Abstract

The application relates to the technical field of robot environment perception and intelligent control, in particular to a kind of image processing and environment perception system of inspection robot adapting to extreme weather;Contain sensor acquisition module, interference identification module, motion estimation module, space-time registration module, state fusion module, traffic area generation module and control output module;System combines environmental features and wheel speed difference solution local disturbance and slip index, construct the data confidence of three-dimensional space unit;The core is to dynamically fuse multi-source perception data with confidence as weight, calculate the obstacle occupancy probability and unknown area ratio, combine the prediction braking area to evaluate the control risk and passable degree, and output the deceleration, bypass or parking instruction;The application overcomes the problem of perception distortion and risk misjudgment caused by rain curtain echo, lens block and wheel slip under extreme weather, uses dynamic confidence correction mechanism to exclude multi-source heterogeneous interference, improves the accuracy of environment state analysis and system robustness.

Description

Technical Field

[0001] This invention relates to the field of robot environmental perception and intelligent control technology, specifically to an image processing and environmental perception system for an inspection robot adapted to extreme weather conditions. Background Technology

[0002] In the current environment of unmanned outdoor inspections, wheeled robots and other devices periodically collect a large amount of environmental perception data. The data modalities vary with the sensors, and the information structure often uses asynchronous frame sequences and is written to the control unit with timestamps. Existing solutions generally adopt a static fusion architecture when performing environmental perception and control on this data. That is, multi-source data is spatiotemporally registered and fused with fixed weights through a preset period, and then obstacle determination is performed in the control system. Existing solutions have basic navigation functions in normal road conditions, but because they are highly dependent on ideal environmental observations and multi-source heterogeneous interference often requires manual intervention, they are easily affected by rain echoes, lens obstruction, and slippage when wading in heavy rain and water accumulation conditions. This results in confidence distortion, high risk of misjudgment, and easy conflict of control commands, making it difficult to support the rapid generation of passage areas and safe movement control of the chassis in complex road conditions.

[0003] Therefore, improving the timeliness and accuracy of environmental condition analysis and chassis motion control under extreme weather conditions has become an urgent technical problem to be solved. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an image processing and environmental perception system for an inspection robot adapted to extreme weather conditions. The technical solution of this invention includes: The sensor acquisition module simultaneously acquires multi-source sensing data, including vision, radar, inertial navigation, and wheel speed encoder data, and records timestamps and delays. The spatial partitioning module divides the environment into three-dimensional spatial units based on the robot coordinate system and a preset three-dimensional grid resolution. The interference identification module extracts the environmental interference characteristics of the multi-source sensing data within a three-dimensional spatial unit and calculates the local interference index of each sensor. The motion estimation module uses weighted fusion of inertial navigation and wheel speed encoder data to acquire speed and pose increments, calculates the wheel slip index based on the speed difference between the two, and calculates the three-dimensional predictive braking area extending along the direction of travel based on speed, direction of travel, and safety redundancy distance. The spatiotemporal registration module performs time compensation on sensor data based on timestamps, delays, and pose increments, and then transforms the sensor data to a unified coordinate system based on sensor external parameters. The confidence calculation module calculates the data confidence of each sensor within the spatial unit based on the local interference index and the wheel slippage index. The state fusion module fuses sensor data with confidence scores as weights to calculate the obstacle occupancy probability of three-dimensional spatial units, and calculates the effective observation coverage based on the sum of confidence scores to determine the proportion of unknown areas; The passage zone generation module calculates the control risk value and passability based on the occupancy probability, the proportion of unknown areas, and the spatial overlap of the three-dimensional predicted braking area. The control output module outputs deceleration, detour, or stop commands to the chassis actuators based on the control risk value and passability.

[0005] Furthermore, the multi-source sensing data is acquired by a multi-modal sensor group, which includes a visible light camera, an infrared thermal imager, a lidar, a millimeter-wave radar, an ultrasonic sensor, an inertial measurement unit, and a wheel speed encoder. The multi-source sensing data each has a corresponding acquisition timestamp and processing delay, and includes visible light images, infrared thermal images, lidar point clouds, millimeter-wave radar data, ultrasonic ranging data, inertial measurement data, and wheel speed encoder data.

[0006] Furthermore, the interference identification module extracts the projection area of ​​the visible light image in the corresponding three-dimensional spatial unit based on regional statistics and frame difference features, and calculates the occlusion ratio, fogging ratio, contrast reduction amount relative to normal weather reference value, and edge continuity reduction amount for the projection area. Based on the occlusion ratio, fogging ratio, contrast reduction amount, and edge continuity reduction amount, the local interference index of the visible light camera in the corresponding three-dimensional spatial unit is determined.

[0007] Furthermore: the interference identification module determines rain curtain echo when the laser point cloud has a density increase exceeding a preset threshold, is suspended within the detectable height range, and has a continuous frame position change exceeding a preset threshold; it determines water surface reflection when the laser point cloud has no spatial matching correspondence with the millimeter-wave radar target and the error with the ground height is within a preset height error threshold; the interference identification module also determines the thermal resolution based on the target-background temperature difference, effective temperature dynamic range, and heat saturation ratio of the infrared thermal image. When the thermal resolution is lower than a preset thermal resolution threshold or the heat saturation ratio exceeds a preset heat saturation threshold, the data confidence of the infrared thermal imager in the corresponding three-dimensional spatial unit is reduced according to a preset confidence correction coefficient.

[0008] Furthermore: the motion estimation module determines the instantaneous wheel speed estimate and the instantaneous inertial speed estimate based on the inertial measurement data and the wheel speed encoder data, respectively, and calculates the wheel slippage index based on the degree of difference between the instantaneous wheel speed estimate and the instantaneous inertial speed estimate; When the wheel slippage index exceeds the preset slippage threshold, the pose calculation is performed by dynamically adjusting the weights of the wheel speed encoder, inertial measurement data, and laser point cloud in the pose calculation. At the same time, the position covariance component corresponding to the robot pose prediction covariance is increased according to the wheel slippage index.

[0009] Furthermore: the spatiotemporal registration module determines the effective observation time by adding or subtracting the processing delay based on the acquisition timestamp, and performs time compensation and coordinate transformation on the sensor data based on the robot's pose increment between the effective observation time and the end of the control cycle; when the wheel slippage index exceeds the preset slippage threshold, the spatiotemporal registration module increases the corresponding position covariance component in the spatial covariance matrix of the registered spatial observation data according to the wheel slippage index.

[0010] Furthermore, the confidence calculation module calculates the initial confidence level based on the sensor's own data quality based on detectable parameters, the consistency of multi-sensor observations based on spatial overlap, and the degree of matching between the observed value and the entity target features. The initial confidence level is then corrected by combining the local interference index and the wheel slippage index to obtain the data confidence level. For rain curtain echo point clouds that are concentrated in a single distance layer and lack height structure, water surface reflection point clouds with a height lower than the preset ground height and a connected area greater than the preset area threshold, and heat saturated areas with a temperature value higher than the preset ambient background temperature, the matching degree of their corresponding target instances is reduced.

[0011] Furthermore: the state fusion module adopts a Bayesian update rule, and fuses the observation data with obstacles according to the data confidence of each sensor in the three-dimensional spatial unit to calculate the obstacle occupancy probability; it determines the effective observation coverage based on the data confidence of each sensor, and calculates the proportion of unknown areas accordingly; When the probability of an obstacle being occupied is greater than or equal to a preset occupancy probability threshold, the three-dimensional spatial unit is marked as an obstacle region; when the probability of an obstacle being occupied is less than a preset occupancy probability threshold and the proportion of the unknown region is less than or equal to a preset unknown proportion threshold, the three-dimensional spatial unit is marked as a passable region; when the probability of an obstacle being occupied is less than a preset occupancy probability threshold and the proportion of the unknown region is greater than a preset unknown proportion threshold, the three-dimensional spatial unit is marked as an unknown region.

[0012] Further: the passage area generation module obtains the three-dimensional predicted braking area that expands along the direction of travel output by the motion estimation module, and calculates the passability by weighting the historical obstacle status, the proportion of unknown areas and the control risk value within a preset time window; When the distance to an obstacle within the three-dimensional predicted braking area is less than the sum of the predicted braking distance and the safety redundancy distance, and the obstacle occupancy probability exceeds or equals a preset occupancy probability threshold or the control risk value exceeds or equals a preset risk value threshold, the control output module outputs a parking command. When there is an unknown area within the three-dimensional predictive braking area where the control risk value exceeds a preset risk value threshold, if the robot's current speed is greater than a preset safe speed, a deceleration command is output; otherwise, a stop command is output. When there are no conditions to output the stop command or the deceleration command, and the passability is lower than a preset passability threshold and there are consecutive adjacent passable areas that meet the size requirements, the control output module outputs a detour command.

[0013] The present invention has the following beneficial effects: 1. To address the issues of distorted environmental perception confidence and frequent risk misjudgments caused by rain curtain echoes, camera obstruction, and wheel slippage during extreme weather conditions such as heavy rain and flooding, this system extracts environmental interference features from multi-source perception data and calculates the local interference index through an interference identification module. It then calculates the wheel slippage index using a motion estimation module and dynamically corrects the data confidence level and calculates the probability of obstacle occupancy and the proportion of unknown areas using a confidence calculation module and a state fusion module. This approach effectively overcomes multi-source heterogeneous interference and improves the accuracy and robustness of environmental state analysis under extreme weather conditions.

[0014] 2. To address the issues of slow traffic zone generation, insufficient chassis motion control, and frequent control conflicts under complex road conditions, this system acquires the robot's motion state and calculates the wheel slippage index through a motion estimation module. It then uses a spatiotemporal registration module for time compensation and coordinate transformation. Combined with the traffic zone generation module, it calculates the control risk value and passability based on the obstacle occupancy probability, the proportion of unknown areas, and the three-dimensional predicted braking area. Finally, the control output module selectively outputs deceleration, detour, or stopping commands to the chassis actuators. This design achieves rapid traffic zone generation and safe chassis motion control under extreme weather conditions. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the system structure provided in an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0017] like Figure 1As shown, an image processing and environmental perception system for an inspection robot adapted to extreme weather conditions includes: The sensor acquisition module simultaneously acquires multi-source sensing data, including vision, radar, inertial navigation, and wheel speed encoder data, and records timestamps and delays. The spatial partitioning module divides the environment into three-dimensional spatial units based on the robot coordinate system and a preset three-dimensional grid resolution. The interference identification module extracts the environmental interference characteristics of multi-source sensing data within a three-dimensional spatial unit and calculates the local interference index of each sensor. The motion estimation module uses weighted fusion of inertial navigation and wheel speed encoder data to acquire speed and pose increments, calculates the wheel slip index based on the speed difference between the two, and calculates the three-dimensional predictive braking area extending along the direction of travel based on speed, direction of travel, and safety redundancy distance. The spatiotemporal registration module performs time compensation on sensor data based on timestamps, delays, and pose increments, and then transforms the sensor data to a unified coordinate system based on sensor external parameters. The confidence calculation module calculates the data confidence of each sensor within the spatial unit based on the local interference index and the wheel slippage index. The state fusion module fuses sensor data with confidence scores as weights to calculate the obstacle occupancy probability of three-dimensional spatial units, and calculates the effective observation coverage based on the sum of confidence scores to determine the proportion of unknown areas; The passage zone generation module calculates the control risk value and passability based on the occupancy probability, the proportion of unknown areas, and the spatial overlap of the three-dimensional predicted braking area. The control output module outputs deceleration, detour, or stop commands to the chassis actuators based on the control risk value and passability.

[0018] Multi-source sensing data is acquired by a multi-modal sensor group, which includes a visible light camera, an infrared thermal imager, a lidar, a millimeter-wave radar, an ultrasonic sensor, an inertial measurement unit, and a wheel speed encoder. The multi-source sensing data has corresponding acquisition timestamps and processing delays, and includes visible light images, infrared thermal images, lidar point clouds, millimeter-wave radar data, ultrasonic ranging data, inertial measurement data, and wheel speed encoder data.

[0019] The interference identification module extracts the projection area of ​​the visible light image in the corresponding three-dimensional spatial unit based on regional statistics and frame difference features. It calculates the occlusion ratio, fogging ratio, contrast reduction relative to the normal weather baseline, and edge continuity reduction of the projection area. Based on the occlusion ratio, fogging ratio, contrast reduction, and edge continuity reduction, it determines the local interference index of the visible light camera in the corresponding three-dimensional spatial unit.

[0020] The system performs a multi-source sensing data processing process from acquisition, spatial division, local interference identification, motion estimation and spatiotemporal registration, to confidence fusion, passage judgment and chassis control. The system uses a substation outdoor inspection wheeled robot performing a preset inspection task under heavy rain and road water accumulation conditions as a unified application environment. When the robot runs along the inspection path, the environmental perception system updates the three-dimensional spatial state in front of the robot according to the control cycle and provides the update results to the chassis actuator. The robot's multimodal sensor suite includes a visible light camera, an infrared thermal imager, a lidar, a millimeter-wave radar, an ultrasonic sensor, an inertial measurement unit, and a wheel speed encoder. Heavy rain causes local water droplet obstruction of the visible light image and fogging in areas exceeding a preset distance threshold. The rain curtain causes density changes in the lidar point cloud within certain distance layers, and water accumulation on the road causes reflections in the point cloud close to the ground. Simultaneously, the robot undergoes posture changes and wheel slippage when wading through water. The system processes the above local disturbances and motion changes within the same control cycle and generates environmental states for deceleration, detour, or stopping. The sensor acquisition module receives visible light images, infrared thermal images, laser point clouds, millimeter-wave radar data, ultrasonic ranging data, inertial measurement data, and wheel speed encoder data, respectively. Each frame of data is associated with an acquisition timestamp and processing delay, enabling subsequent processing to distinguish between the sensor acquisition time and the time when the data enters the fusion process. The sensor number, data frame sequence number, sensor coordinate system identifier, and current working status can be recorded along with the corresponding data to identify the data source and determine the coordinate transformation relationship. Each sensor can use a unified clock to correct the acquisition timestamp; for timestamps generated by the sensors themselves, the acquisition module records their time relationship according to the unified clock; for data that cannot provide a valid timestamp, the data will not participate in the current period fusion that requires precise time compensation, and the data confidence of its corresponding three-dimensional spatial unit will be reduced. If the processing delay cannot be determined temporarily, the processing delay recorded in the previous valid period of the sensor is used, and the relevant observations are marked as observations with reduced timeliness. By using the processing delay of the previous valid period, the loss of time information from a single sensor can prevent all sensing data in the current period from stopping updating. The spatial division module uses the robot coordinate system as a reference and divides the robot's current perception range into multiple three-dimensional spatial units according to a preset three-dimensional grid resolution. The three-dimensional grid resolution is predetermined based on the robot size, chassis ground clearance, minimum turning radius, sensor spatial resolution, and braking safety requirements. Each three-dimensional spatial unit is used to associate the point cloud, radar target, ultrasonic ranging results, and area information obtained by projecting visible light images or infrared thermal images into that unit. Visible light images and infrared thermal images are mapped to corresponding three-dimensional spatial units based on camera imaging parameters and sensor external parameters; laser point clouds and millimeter-wave radar targets are directly placed into corresponding spatial units after coordinate transformation; ultrasonic ranging data are associated with corresponding spatial units according to sensor detection direction and effective ranging range; for areas in the image where spatial correspondence cannot be established due to occlusion or imaging range limitations, they are not treated as unobstructed observations, but the original state of the corresponding spatial unit is preserved or the proportion of unknown areas is increased. The interference identification module extracts environmental interference features according to the sensor and the three-dimensional spatial unit respectively; the same sensor has different local interference indices in different spatial units; for example, the projection area in the visible light image that is blocked by lens water droplets has a high local interference index, while the area that is not blocked and still maintains the edge structure can continue to be used for environmental state fusion, without uniformly failing the entire visible light image. For visible light cameras, the interference identification module first determines the projection area of ​​each three-dimensional spatial unit in the visible light image, and then calculates the occlusion ratio, fogging ratio, contrast reduction amount relative to the normal weather baseline value, and edge continuity reduction amount based on regional statistics and differences between adjacent frames. The normal weather baseline value can be determined by normal weather data collected by the robot under the same sensor configuration, or by using the undisturbed neighboring area in the current image as a comparison basis. Occlusion ratio indicates the extent to which the projected area is covered by water droplets, stains, or frost; fogging ratio indicates the extent to which local brightness converges, details are reduced, and consecutive frames maintain a similar blurry appearance; contrast reduction is determined by the difference in local contrast between the current projected area and the normal weather reference area; edge continuity reduction is determined based on the degree of breakage of road edges, equipment outlines, or other structural edges in the current area; frame difference features are used to distinguish between physical structures that change with environmental motion and relatively fixed occlusions attached to the lens; In one implementation, the projection area forms an effective pixel set by pixels; occluded pixels are determined by pixels that remain relatively fixed with environmental movement in consecutive frames, have abrupt changes in brightness or color, and have low edge response, and the occlusion ratio is the ratio of the number of occluded pixels to the number of effective pixels; fogged pixels are determined by pixels with decreased saturation, local contrast below the reference lower limit, and high-frequency texture energy below the reference lower limit, and the fogging ratio is the ratio of the number of fogged pixels to the number of effective pixels; The contrast reduction is the ratio of the difference between the baseline local contrast and the current local contrast to the baseline local contrast, and is limited to the range of zero to one; the edge continuity reduction is the difference between the baseline continuous edge length ratio and the current continuous edge length ratio, and is also limited to the range of zero to one; the effective pixel count is the number of pixels in the projection area after excluding invalid pixels after camera distortion correction, pixels that cannot be judged due to exposure saturation, and pixels outside the calibration mask; when the effective pixel count is zero, the projection area will not output a clear observation. The interference identification module normalizes the above features and combines them according to preset weights to obtain the local interference index of the visible light camera in the corresponding three-dimensional spatial unit. The sum of the weights of each feature is one, and the weights are determined based on calibration data under normal weather, rainstorm, fog, and lens water droplet conditions. When the local interference index increases, the confidence calculation module reduces the confidence of the visible light data in that spatial unit. When the local interference index does not exceed the corresponding judgment condition, the visible light data maintains the confidence determined according to its own data quality. In heavy rain inspection environments, the local interference index of visible light cameras is obtained by weighting and summing the occlusion ratio, fogging ratio, contrast reduction, and edge continuity reduction at 30%, 25%, 25%, and 20% respectively. The above weighting ratios are based on the statistical analysis of the contribution of each type of interference to visual perception failure in historical heavy rain inspection environments, to ensure that higher sensitivity penalties are applied to frequently occurring lens occlusion and water fog blurring. If a feature cannot be calculated due to a missing frame, an unavailable reference region, or an invalid projection region, the feature is not substituted with zero. Instead, its weight is redistributed according to the original weight ratio of the remaining valid features. When there are fewer than two valid features, the data confidence of the projection region is processed as observations with reduced timeliness, and the proportion of unknown regions in the corresponding spatial unit is increased. The visible light clarity threshold and severe interference threshold are determined by the distribution of local interference index in the same camera in clear samples under normal weather and raindrop samples, taking the upper limit of the normal clarity state and the lower limit of the severe occlusion state, respectively. In the rainstorm inspection environment of substations, the visible light clarity threshold can be taken as 0.35 and the severe interference threshold can be taken as 0.7. When the local interference index is not greater than the clarity threshold, the normal confidence correction path of visible light observation is retained. When the local interference index reaches or exceeds the severe interference threshold, the visible light observation is regarded as a low confidence observation, and the difference between the two is continuously reduced according to the magnitude of the local interference index. When the normal weather baseline is unavailable in the current area, the interference identification module compares it with the stable records of the adjacent valid areas or the previous valid period. If neither the adjacent areas nor the historical records can provide a valid basis for comparison, the current area is not identified as a clear area. Instead, its data confidence is reduced and the proportion of unknown areas in the corresponding spatial unit is increased. When consecutive frames are missing, the occlusion ratio and fog ratio can still be calculated based on the statistical results of the current frame area. Features that depend on frame differences are not used as a valid basis for judgment in the current period. The motion estimation module performs weighted fusion of inertial measurement data and wheel speed encoder data to obtain the robot's velocity and attitude changes. The robot velocity includes at least the velocity component along the current direction of travel, and the attitude changes reflect at least the robot's yaw, pitch, and roll changes. The velocity difference between the inertial measurement data and the wheel speed encoder data is used to calculate the wheel slip index. An increase in the wheel slip index indicates a decrease in the consistency between the wheel speed reflected by the wheel speed encoder and the robot's motion trend. The motion estimation module calculates the three-dimensional predictive braking region based on the robot's speed, current direction of travel, and safety redundancy distance. This region extends along the robot's direction of travel, and its length varies with the predicted braking distance determined by the robot's speed and the degree of slippage, while also taking into account the safety redundancy distance. Its width and height are determined based on the robot's body dimensions and the safe passage range. The three-dimensional predictive braking region is updated with the robot's motion state in the current control cycle, providing information for subsequent calculations of the overlap between the spatial unit and the braking region. If inertial measurement data is missing in the current cycle, the motion estimation module can use the attitude change trend of the previous effective cycle and the current wheel speed encoder data to maintain short-term motion estimation, while improving the uncertainty of pose prediction; if wheel speed encoder data is missing, motion changes are determined based on inertial measurement data and available continuous frame spatial observations; when neither type of data can provide effective motion information, the nominal motion amount is not used for precise projection, but the uncertainty range of the relevant spatial state is expanded, and the unidentifiable spatial units in the braking area remain unknown. The spatiotemporal registration module determines the effective observation time based on the acquisition timestamps and processing delays of each sensor data, and reads the robot's pose increment between the effective observation time and the end of the current control cycle. For effective observation times located between two known pose records, the corresponding robot pose is obtained by applying linear interpolation to the translation vector and spherical linear interpolation to the rotation quaternion. The spherical linear interpolation formula is as follows: ; in, The quaternion for the corresponding robot pose obtained by interpolation. , For adjacent valid poses, quaternion records, Let θ be the time interpolation ratio, and θ be the angle between the two quaternions. The calculation formula is: ; in, The effective observation time after time delay compensation. They are respectively The corresponding physical system timestamp; then motion compensation is performed on the sensor observations; after time compensation is completed, the spatiotemporal registration module uses the external parameters of each sensor relative to the robot body to uniformly transform the image projection results, point cloud, radar target and ranging results to the robot coordinate system; Registered data is re-collected according to three-dimensional spatial units; each observation retains the sensor source, effective observation time, and corresponding local interference index so that the confidence calculation module can use it directly without having to read the unregistered original data again; when there is a lack of effective pose records in the pose interpolation interval, the system uses the most recent effective pose record and increases the spatial uncertainty of the relevant observation; when the uncertainty exceeds the preset allowable range, the corresponding observation is not used to confirm the passable area, but can only be used as an auxiliary basis for risk assessment; The confidence calculation module combines the local interference index and the wheel slippage index to calculate the data confidence of each sensor in each three-dimensional spatial cell. The local interference index is used to reflect the degree to which the sensor is affected by rain, fog, water droplets or reflection within a specific spatial range. The wheel slippage index is used to reflect the uncertainty of motion compensation and spatial registration results. For sensors that do not cover a certain spatial cell, whose data has exceeded the validity period, or whose coordinate correspondence cannot be completed, the corresponding data confidence in that spatial cell is treated as no valid observation. The state fusion module performs weighted fusion of obstacle observations from each sensor according to data confidence level to obtain the obstacle occupancy probability of each three-dimensional spatial unit. When multiple sensors provide effective obstacle observations for the same spatial unit, the occupancy probability increases accordingly. When only the interfered sensor provides observations, the effect of the observation on the occupancy probability is limited by its data confidence level. The state fusion module also calculates the effective observation coverage rate based on the sum of the data confidence rates of each sensor within the same spatial unit, and subtracts the effective observation coverage rate to obtain the proportion of unknown areas. Therefore, spatial units that do not detect obstacles but have an effective observation coverage rate lower than the preset coverage rate threshold remain unknown areas and are not directly used as passable areas. If all sensors fail to make effective observations of a certain spatial unit in the current cycle, the effective observation coverage rate of that unit is taken as the lower limit, the proportion of unknown areas is taken as the upper limit, and its historical obstacle information is retained. The passage zone generation module reads the obstacle occupancy probability, the proportion of unknown areas, and the three-dimensional predicted braking area, and calculates the spatial overlap between each spatial unit and the three-dimensional predicted braking area. The obstacle occupancy probability reflects the probability that there are physical obstacles in the spatial unit, the proportion of unknown areas reflects the degree to which the spatial unit has not been fully observed, and the spatial overlap reflects the degree of influence of the spatial unit on the current braking process. The three factors are used to form a control risk value according to a preset risk weight, and the passability is further calculated. Unknown spatial units located within the three-dimensional predictive braking area maintain corresponding control risks even if the obstacle occupancy probability does not meet the obstacle confirmation criteria, due to the proportion of unknown areas and spatial overlap. Spatial units located outside the predictive braking area and with sufficient observation can form higher drivability based on lower obstacle occupancy probabilities. For spatial units with abnormal ground boundaries, water accumulation boundaries, or low obstacles, their risk impact can be retained in the calculation of control risk values ​​to avoid determining them as drivable solely based on ordinary target identification results. The control output module generates deceleration, detour, or stop commands based on the control risk value and accessibility. When the accessibility of a spatial unit on the robot's current path decreases, but there are still adjacent spatial units that meet the passage conditions, a detour command is output. When the risk has not yet reached the stopping condition and reducing the speed can meet the braking safety requirements, a deceleration command is output. When it is predicted that there are obstacles or high-risk unknown areas within the braking area that meet the stopping conditions, a stop command is output. Each control cycle outputs only one type of command to the chassis actuator to avoid conflicting control requirements within the same cycle. If the control risk value or passability of the current cycle cannot be updated due to missing data, the control output module will not interpret the missing result as no risk, but will use the control result from the previous effective cycle with a restriction level no lower than the current state; when the predicted braking area continues to lack effective observations, the deceleration or stop command will be output according to the control conditions corresponding to the unknown area.

[0021] The interference identification module determines rain curtain echoes when the density increment of the laser point cloud exceeds a preset threshold, is suspended within the detectable height range, and the position change of consecutive frames exceeds a preset threshold; it determines water surface reflections when the laser point cloud has no spatial matching correspondence with the millimeter-wave radar target and the error with the ground height is within a preset height error threshold. The interference identification module also determines the thermal resolution based on the target-background temperature difference, effective temperature dynamic range, and heat saturation ratio of the infrared thermal image. When the thermal resolution is lower than the preset thermal resolution threshold or the heat saturation ratio exceeds the preset heat saturation threshold, the data confidence of the infrared thermal imager in the corresponding three-dimensional spatial unit is reduced according to a preset confidence correction coefficient.

[0022] The motion estimation module determines the instantaneous wheel speed estimate and the instantaneous inertial speed estimate based on the inertial measurement data and the wheel speed encoder data, respectively, and calculates the wheel slippage index based on the degree of difference between the instantaneous wheel speed estimate and the instantaneous inertial speed estimate. When the wheel slippage index exceeds the preset slippage threshold, the pose calculation is performed by dynamically adjusting the weights of the wheel speed encoder, inertial measurement data, and laser point cloud in the pose calculation. At the same time, the position covariance component corresponding to the robot pose prediction covariance is increased according to the wheel slippage index.

[0023] The spatiotemporal registration module determines the effective observation time by adding or subtracting the processing delay based on the acquisition timestamp, and performs time compensation and coordinate transformation on the sensor data based on the robot's pose increment between the effective observation time and the end of the control cycle. When the wheel slippage index exceeds the preset slippage threshold, the spatiotemporal registration module increases the corresponding position covariance component in the spatial covariance matrix of the registered spatial observation data according to the wheel slippage index.

[0024] When heavy rain, road flooding, and water skidding coexist, the system distinguishes between rain curtain echoes, water surface reflections, and infrared thermal background interference, and corrects the pose calculation and spatiotemporal registration results according to the degree of wheel slippage. Its basic processing method is to first determine the type of local interference from laser point cloud, millimeter-wave radar data, and infrared thermal images, and then use the interference judgment results and motion estimation results together to adjust the data confidence and spatial uncertainty. The above processing can reduce the impact of interference from observations by spatial unit, but it is difficult to distinguish between weather echoes with certain statistical stability and physical obstacles based solely on basic local mass changes. For example, rain cloud points may form high point cloud density within a single period, water surface reflection point clouds may remain close to the ground, and infrared high temperature background may form continuous high temperature regions. If the above observations are all treated as ordinary obstacles and fused together, the probability of obstacle occupancy may be inconsistent with the road conditions. The system further determines the rain echo, water surface reflection, and thermal resolution. The robot runs along the outdoor inspection path of the substation; heavy rain forms suspended and continuously changing echoes within the detection range of the lidar, and road water forms reflection point clouds near the ground; the rising ambient temperature reduces the temperature difference between some equipment and the background; after the robot enters the waterlogged area, the speed reflected by the wheel speed encoder differs from the speed reflected by the inertial measurement data; the system processes the above data within the same spatial division and control cycle system without changing the application environment and sensor configuration; The interference identification module counts the density of laser point clouds according to distance layer, height range and three-dimensional spatial unit, and compares the point cloud density of the current period with the point cloud density of normal observation or the previous stable period. When the point cloud density of a certain spatial range suddenly increases beyond the preset threshold, the point cloud is suspended in the height range that the lidar can detect, and the position change of the point cloud in consecutive frames exceeds the preset threshold, the corresponding point cloud is identified as rain curtain echo. Among them, density abrupt increment is used to exclude entity surfaces where the number of point clouds is within the normal range of variation; suspended distribution is used to exclude point clouds that are consistent with continuous ground or stable three-dimensional contours; continuous frame position change is used to exclude fixed obstacles whose position and height structure remain stable; rain curtain echo judgment is made only when all three conditions are met; point clouds judged as rain curtain echoes still retain their acquisition time and spatial position, but their entity target matching degree and corresponding data confidence are reduced, so that their effect on the probability of obstacle occupation is reduced; In one implementation, the density surge threshold is determined based on the upper limit of the number of point clouds at the same distance layer and height range under normal rainfall or no rainfall conditions, preferably taking the average value under normal stable conditions plus two to three times the fluctuation amount; the continuous frame position change threshold is determined based on the upper limit of the average position change of point clouds in a fixed environment within adjacent control cycles; the detectable height range is the common obstacle detection height range above the ground and below the top of the device for lidar; each preset threshold in the rain curtain echo judgment is configured with corresponding calibration basis. If consecutive frame point clouds are missing, rain curtain echo determination cannot be directly completed based solely on single frame density surges and suspended distributions. In this case, the corresponding point cloud remains in an unconfirmed state, and its presence or absence is determined by combining millimeter-wave radar, visible light, or infrared observations. When other sensors cannot provide effective observations, the spatial unit remains unknown, and the point cloud is neither directly deleted nor directly identified as an obstacle. For laser point clouds near the ground, the interference identification module matches their spatial location with the millimeter-wave radar target; when there is no spatial matching correspondence between the laser point cloud and the millimeter-wave radar target, and the error between the point cloud height and the ground height is within the allowable range, the point cloud is determined to be a water surface reflection; the ground height can be determined by the current effective ground observation or the ground record of the previous effective period, and the allowable error is set according to the laser radar measurement error, robot posture change and ground undulation range; If a stable target exists within the corresponding spatial unit of the millimeter-wave radar, or if the laser point cloud has a continuous height structure, it will not be directly determined as water surface reflection simply because the point cloud is close to the ground. For point clouds where there are changes in ground elevation at the water accumulation boundary, even if some of the point clouds meet the reflection characteristics, the boundary anomaly is still retained as a passage risk. The water surface reflection judgment only reduces the contribution of the reflected point cloud as a physical obstacle, and does not cancel the risk information corresponding to the water accumulation boundary or the edge of the ditch. When millimeter-wave radar data is missing or its operating status is invalid, the system does not replace the lack of spatial matching correspondence with the absence of millimeter-wave radar target; instead, it combines the stability of continuous frame point clouds, point cloud height structure, and spatial observations from other sensors to make a judgment; if it still cannot be determined, the corresponding spatial unit is marked as unconfirmable, and the proportion of unknown areas is increased. For infrared thermal images, the interference identification module determines the candidate target area and the surrounding background area within the projection area corresponding to each three-dimensional spatial unit, calculates the temperature difference between the target and the background, and determines the thermal resolution by combining the effective temperature dynamic range and heat saturation ratio of the current infrared image. When the temperature difference between the target and the background decreases or the effective temperature dynamic range shrinks, the thermal resolution decreases. When the proportion of pixels that reach the heat saturation condition within the projection area increases, the ability of infrared data to distinguish target outlines and road obstacles decreases. In one implementation, the thermal resolution threshold is determined jointly based on the lower limit of the temperature difference between the target and the background under normal inspection temperature conditions and the minimum resolvable temperature difference of the infrared imager; the thermal saturation threshold is determined based on the upper limit of the proportion of pixels in the infrared image that reach the sensor's saturation temperature range; and the confidence correction coefficient... Determined according to the following piecewise formula: when hour, ;when hour: ; in, For thermal resolution, The thermal resolution threshold, The preset thermal resolution penalty coefficient, It has dimensionless properties, so that the terms on the right side of the equation remain dimensionless; the system is configured with confidence decay criteria corresponding to low thermal resolution and high thermal saturation ratio; When the thermal resolution is lower than the preset thermal resolution threshold, or the thermal saturation ratio exceeds the preset thermal saturation threshold, the confidence calculation module reduces the data confidence of the infrared thermal imager in the corresponding three-dimensional spatial unit according to the preset confidence correction coefficient; this correction only applies to the affected spatial unit and does not change the data confidence of other regions with effective temperature difference and stable contour in the infrared thermal image. The thermal anomaly status of equipment and the status of road obstacles in infrared images are processed separately; for high-temperature areas with insufficient thermal resolution, their role in confirming road obstacles is reduced; for areas that still have significance for equipment thermal anomalies, the corresponding thermal status information is retained; if an effective temperature dynamic range cannot be obtained, road obstacles are not confirmed based on thermal resolution in the current cycle, but the confidence level of infrared observation is reduced and spatial observation is continued by visible light, lidar and millimeter-wave radar. In motion estimation, when only inertial measurement data and wheel speed encoder data are fused with fixed weights, wheel spin on a flooded surface may cause the estimated wheel speed to deviate from the robot's motion speed, resulting in pose calculation and spatial registration errors. Therefore, the motion estimation module determines the instantaneous wheel speed estimate based on both wheel speed encoder data and inertial measurement data. and instantaneous inertial estimation velocity The wheel slip index is calculated based on the speed difference between the two. In one implementation, the wheel slippage index is calculated as follows: ; in, The wheel slip index. To estimate speed for instantaneous wheel speed, For instantaneous inertial velocity estimation, The preset bias constants are velocity-dimensional, in units of: The calculation uses this minimal positive number to limit the denominator under low-speed or stationary conditions, so as to avoid the slippage index losing its effective range due to the divisor being too small when the speed is close to zero; both sides of the formula are dimensionless; when the wheel speed estimate and the inertial speed estimate are both close to zero, if the denominator is still less than the preset minimum speed scale, the denominator will be taken as the minimum speed scale, and the slippage index of this cycle will be marked as a conservative estimate under low-speed conditions, without triggering the slippage weight switch separately based on this. Take a positive number no greater than one-tenth of the robot's minimum stable driving speed, and can be set according to the value corresponding to the lowest recognizable speed of the chassis; the slip threshold is based on the normal adhesion surface. and The statistical upper limit of the difference is determined, preferably a value greater than or equal to twice the fluctuation amount plus the average difference during normal operation; the pre-defined decay function with monotonically decreasing characteristics adopts a linear decreasing relationship, and the specific formula is as follows: ; in, The relative weights are corrected for the wheel speed encoder. The wheel slip index. The slippage threshold The preset attenuation coefficient is greater than zero; the relevant constants and thresholds in the slippage index calculation are configured according to preset parameters; the instantaneous wheel speed estimation speed is determined based on the wheel speed of each drive wheel and the motion relationship of the robot chassis; the instantaneous inertial speed estimation speed is determined based on inertial measurement data and historical pose changes. The slippage threshold is determined based on the statistical range of the difference between the two types of speed estimates under normal road conditions. When the wheel slippage index does not exceed the slippage threshold, the wheel speed encoder, inertial measurement data, and the available laser point cloud continuous frame registration results participate in the pose calculation with normal weights. When the wheel slippage index exceeds the preset slippage threshold, the motion estimation module reduces the weight of the wheel speed encoder in the pose calculation according to the preset decay function that monotonically decreases as the slippage index increases; at the same time, it increases the weight of the inertial measurement data and the robot's relative pose change obtained by continuous frame registration of laser point cloud; the adjusted weights are normalized so that their sum is kept to one, and then the robot's current pose increment is calculated accordingly. In one implementation, under normal conditions, the initial weights corresponding to the wheel speed encoder, inertial measurement data, and laser point cloud continuous frame registration results are predetermined according to the chassis flat road surface calibration results. When the wheel slippage index exceeds the slippage threshold, the wheel speed encoder correction weight is first obtained according to the preset attenuation function, and then the deducted weights are allocated to the latter two according to the validity of the inertial measurement data and laser point cloud continuous frame registration results in the current cycle, and normalized. If the current period's laser point cloud continuous frame registration result is invalid, all remaining weights are allocated to the inertial measurement data. The system performs weight adjustments under different branches according to the above conditions. The point cloud used for laser point cloud continuous frame registration excludes low-confidence point clouds that have been identified as rain curtain echoes or water surface reflections, and prioritizes point clouds with continuous height structure and position changes that conform to fixed environmental characteristics. If heavy rain leads to insufficient effective point clouds that can be used for continuous frame registration, the role of this data in pose calculation is not increased, the remaining effective weights are borne by the inertial measurement data, and the pose prediction uncertainty is increased accordingly. The motion estimation module also increases the preset components of the robot pose prediction covariance based on the wheel slippage index. The pose prediction covariance is represented by a covariance that includes translation and attitude states, where the unit of the position component is the square of the length, the unit of the attitude component is the square of the angle, and the cross component between position and attitude is represented according to the corresponding length and angle units. When the slippage index increases, the position component of the travel direction, the lateral position component, or the yaw attitude component affected by slippage in the pose prediction covariance is increased first, and then the pose covariance is propagated to the spatial observation position through the motion model. The spatiotemporal registration module uses pose prediction covariance propagation to obtain the spatial covariance matrix of the registered spatial observation data. The spatial covariance matrix contains only three-dimensional position components, and its rows and columns correspond to the longitudinal, lateral, and height positions in the robot coordinate system. Attitude uncertainties such as yaw are not directly used as independent rows and columns of the spatial covariance matrix, but are amplified by propagating the spatial position error through pose uncertainty to the corresponding position components and position intersection components. The spatial covariance with the dimension of length square and the attitude covariance with the dimension of angle square are calculated separately. When either type of data from the wheel speed encoder or inertial measurement data is missing, the wheel slip index is not updated based on the difference between the two; the system uses the most recent effective slip index and increases the pose prediction uncertainty based on the number of cycles that the data has been used for; if continuous frame registration of the laser point cloud cannot form an effective change in the robot's relative pose, the unreliable wheel speed pose is stopped from being used as a deterministic value, and the relevant spatial units in the predicted braking area are kept in a conservative state. The spatiotemporal registration module determines the valid observation time based on the sensor's timestamp recording method. When the acquisition timestamp represents the time when data acquisition is completed, and the processing delay represents the time difference from the observation to the recording time, the processing delay is subtracted from the acquisition timestamp. When the timestamp represents the start time of data entering the processing process, and the processing delay is used to determine the available observation time, the processing delay is added to the acquisition timestamp. The meaning of the timestamp is recorded according to the sensor configuration to avoid mixing addition and subtraction on the same sensor. After determining the effective observation time, the spatiotemporal registration module reads the robot's pose increment from the effective observation time to the end of the control cycle. The pose increment is provided by the motion estimation module and determined by pose interpolation between the effective pose records. The observation point in the sensor coordinate system is first transformed to the robot body coordinate system according to the sensor external parameters, and then the position change caused by the robot's motion is compensated according to the pose increment in the time interval to obtain the spatial observation position at the end of the control cycle. When the wheel slippage index exceeds the preset slippage threshold, the spatiotemporal registration module reads the pose prediction covariance generated by the motion estimation module, and increases the position component and necessary position cross component in the spatial covariance matrix of the registered spatial observation data according to the wheel slippage index. Attitude uncertainty is first propagated as position uncertainty through the positional relationship between the sensor observation point and the robot reference point, and then participates in the calculation of spatial observation covariance; the corresponding spatial observation can be associated with the expanded spatial range during subsequent grid collection, and the registration position affected by slippage is not used as the error-free position; The preset components in the spatial covariance matrix preferably include the direction of travel position component and the lateral position component; when slippage is mainly manifested as the drive wheels spinning freely, the direction of travel position component is increased first; when slippage is accompanied by obvious yaw, the affected position component and position cross component are increased simultaneously based on the propagation result of the yaw attitude uncertainty to the sensor observation position; the corresponding increase amount is selected according to the segmented interval where the slippage index is located to ensure that the spatial uncertainty after registration is consistent with the degree of slippage; If the effective observation time exceeds the coverage of the available pose record, the nearest effective pose record is used to complete the limited compensation and increase the spatial covariance matrix; if it exceeds the allowable compensation range, the observation is not used to reduce the proportion of unknown areas; if the sensor external parameters are invalid, the corresponding data is retained in the original sensor record and is not entered into the state fusion under the unified robot coordinate system; sensors with other valid external parameters continue to complete the current cycle processing. Through the above processing, rain curtain echoes and water surface reflections are not directly used as ordinary physical obstacles in high-weight fusion. Changes in infrared thermal background only reduce the confidence of infrared data in the affected area. Wheel slippage affects both the pose calculation weight and the spatial uncertainty after registration. The results of each processing are finally transmitted to the confidence calculation module and the state fusion module according to the original three-dimensional spatial units.

[0025] The confidence calculation module calculates the initial confidence based on the sensor's own data quality based on detectable parameters, the consistency of multi-sensor observations based on spatial overlap, and the degree of matching between the observed values ​​and the features of the entity target. It then corrects the data confidence by combining the local interference index and the wheel slippage index. For rain curtain echo point clouds that are concentrated in a single distance layer and lack height structure, water surface reflection point clouds with heights lower than the preset ground height and connected areas larger than the preset area threshold, and heat saturated areas with temperatures higher than the preset ambient background temperature, the matching degree of their corresponding target instances is reduced.

[0026] The state fusion module adopts a Bayesian update rule to fuse observation data with obstacles according to the data confidence of each sensor in the three-dimensional spatial unit and calculate the obstacle occupancy probability; it determines the effective observation coverage based on the data confidence of each sensor and calculates the proportion of unknown areas accordingly; when the obstacle occupancy probability is greater than or equal to the preset occupancy probability threshold, the three-dimensional spatial unit is marked as an obstacle area. When the probability of an obstacle being occupied is less than a preset occupancy probability threshold and the proportion of unknown areas is less than or equal to a preset unknown proportion threshold, the three-dimensional spatial unit is marked as a passable area; when the probability of an obstacle being occupied is less than a preset occupancy probability threshold and the proportion of unknown areas is greater than a preset unknown proportion threshold, the three-dimensional spatial unit is marked as an unknown area.

[0027] The passage area generation module obtains the three-dimensional predicted braking area that expands along the direction of travel output by the motion estimation module, and calculates the passability based on the historical obstacle status, the proportion of unknown areas and the control risk value within a preset time window. When the distance to an obstacle within the three-dimensional predicted braking area is less than the sum of the predicted braking distance and the safety redundancy distance, and the obstacle occupancy probability exceeds or equals the preset occupancy probability threshold or the control risk value exceeds or equals the preset risk value threshold, the control output module outputs a parking command. When there is an unknown area within the 3D predicted braking area where the control risk value exceeds the preset risk value threshold, if the robot's current speed is greater than the preset safe speed, a deceleration command is output; otherwise, a stop command is output. When there are no conditions to output a stop command or a deceleration command, and the passability is lower than the preset passability threshold and there are consecutive adjacent passable areas that meet the size requirements, the control output module outputs a detour command.

[0028] The results of local interference judgment are converted into data confidence scores. The state fusion module distinguishes between obstacle areas, passable areas, and unknown areas, and generates control commands based on the predicted braking areas. When the robot passes through rain-fed areas along the outdoor inspection path of the substation, it uses three-dimensional spatial units as a unified processing object. Each control cycle performs observation quality evaluation, confidence score correction, obstacle probability update, coverage calculation, risk calculation, and control command determination. The confidence calculation module establishes three sub-evaluation scores for the observations formed by each sensor in each three-dimensional spatial unit: sensor data quality value, cross-sensor observation consistency value, and entity target matching value. All three sub-evaluation scores are normalized to the range of zero to one. The sensor's own data quality value is determined by parameters that can be directly detected. Among them, visible light cameras use local contrast, edge continuity, occlusion ratio and fog ratio; lidar uses effective point count, distance layer distribution and height structure; millimeter wave radar uses target stability and spatial position continuity; ultrasonic sensors use effective ranging status; and infrared thermal imagers use target-background temperature difference, effective temperature dynamic range and heat saturation ratio. Each detectable parameter is segmented and normalized according to the normal state limit and failure state limit of the corresponding sensor. For parameters whose larger values ​​indicate higher quality, the value is taken as 1 when the parameter reaches the normal state limit, and as 0 when it reaches the failure state limit. When it is in between, the value is taken linearly according to its position between the two boundaries. For parameters such as the occlusion ratio, atomization ratio, and heat saturation ratio, whose larger values ​​indicate lower quality, the values ​​are taken in the opposite direction. Results exceeding the boundaries are limited to 0 or 1 respectively; the lower limit of the normal state and the upper limit of the failure state are determined by the parameter distribution recorded by the same sensor in a stable environment without rainfall and under corresponding extreme weather conditions, and are saved with the sensor number. The boundaries of other sensor models are not used to replace the current cycle. When a sensor has multiple quality parameters, first determine whether each parameter is valid, then sum the valid parameters according to preset parameter weights; the sum of parameter weights is one; if a parameter is missing, its weight is redistributed according to the original weight ratio of the remaining valid parameters; if all parameters are invalid, the sensor's own data quality value is zero; for example, the local contrast, edge continuity, occlusion ratio, and fog ratio of a visible light camera use weights of 30%, 30%, 20%, and 20% respectively; this set of weights is used in rainstorm inspection environments and is saved in the sensor configuration record along with the applicable weather type; The cross-sensor observation consistency value is determined based on the degree of spatial overlap of observations from different sensors in a unified robot coordinate system. After projecting each observation onto a three-dimensional spatial cell according to the spatial covariance, the number of overlapping observation pairs is counted. When the distance between the observation centers is not greater than the allowable distance determined by the corresponding covariance and the spatial cells they occupy have an intersection, they are counted as a consistent observation pair; otherwise, they are not counted as a consistent observation pair. The target matching value is determined based on the height structure, position stability of consecutive frames, target outline continuity, and spatial correspondence with millimeter-wave radar targets, visible light edge targets, or infrared targets. Observations lacking height structure, concentrated in a single range layer, or with random and dispersed position changes in consecutive frames will have lower target matching values. The cross-sensor observation consistency value is determined according to the proportion of the number of consistent observation pairs to the total number of comparable observation pairs; comparable observation pairs refer to two different sensor observations whose effective observation ranges both cover the current space cell, whose effective observation times differ by no more than one control cycle, and whose coordinate transformations have both been completed. When no comparable observation pair exists, the cross-sensor observation consistency value is not set to one, but is marked as missing, and the weight of this evaluation dimension is reassigned during the initial confidence calculation; when there are contradictory occupied observations and unobstructed observations, the observation pair is not counted as a consistent observation pair, and the two observations are retained and entered into the subsequent Bayesian update respectively. The entity target matching value is determined according to four judgment results: when there is a continuous height structure that crosses at least two height grids, an additional valid match is added; when the target center displacement after deducting the robot pose increment does not exceed the allowable distance of spatial covariance within three consecutive valid periods, an additional valid match is added. When the continuous ratio of adjacent edge units of the target contour reaches the preset contour ratio, an effective match is added; when it forms a spatial correspondence with any effective observation of millimeter-wave radar target, visible light edge, or infrared target, an effective match is added. The entity target matching value is the ratio of the number of valid matching items to four; if an item cannot be determined due to the limitation of the sensor detection range, the item is deducted from the denominator; if all items cannot be determined, the entity target matching value is marked as missing. Within a control cycle, the sensor's own data quality value, observation consistency value, and entity target matching value are multiplied by preset weights and then summed to obtain the initial confidence level. The initial confidence level is then corrected according to the local interference index and the wheel slippage index, and the corrected data confidence level is limited to the range of zero to one. Specifically, the larger the local disturbance index, the smaller the disturbance attenuation coefficient; the larger the wheel slippage index, the smaller the spatial registration attenuation coefficient; when any data is missing, exceeds the validity period, cannot complete the external parameter transformation, or the spatial covariance exceeds the allowable range, the corresponding sub-item evaluation score will be treated as an invalid observation, and the effective observation contribution of the spatial unit will be set to zero. In one implementation, the initial confidence level is obtained by weighting the sensor's own data quality value, cross-sensor observation consistency value, and entity target matching value by 40%, 35%, and 25%, respectively. During correction, the interference attenuation coefficient is obtained by subtracting the local interference index from one, and the spatial registration attenuation coefficient is obtained by subtracting the product of the slippage correction coefficient and the wheel slippage index from one. When the spatial registration attenuation coefficient is less than zero, it is treated as zero, and when it is greater than one, it is treated as one. The initial confidence level, interference attenuation coefficient, and spatial registration attenuation coefficient are multiplied to obtain the data confidence level. For rain curtain echo point clouds, water surface reflection point clouds, and thermal saturation regions, the entity target matching value is reduced first, and then the above confidence level calculation is performed without reducing the entity target observations that have been confirmed by other sensors within the same spatial cell. In heavy rain and waterlogging inspection environments, the slippage correction coefficient is set to 0.5. When cross-sensor observation consistency values ​​or entity target matching values ​​are marked as missing due to the lack of comparable observations, the missing values ​​are not substituted as zero. Instead, the corresponding weights are redistributed according to the original weight ratio of the sensor's own data quality value and the remaining valid evaluation scores. When the sensor's own data quality value is invalid, the entire sensor observation is treated as an invalid observation. When the wheel slippage index is greater than two, it is used as two in the confidence correction to give the spatial registration attenuation coefficient a definite lower limit. When the correction result reaches zero, this observation does not participate in the obstacle occupancy probability update. For rain curtain echo point clouds that are concentrated in a single distance layer and lack height structure, reduce their entity target matching value to below the preset rain curtain matching upper limit; for water surface reflection point clouds that are lower than the preset ground height and have a connected area greater than the preset area threshold, reduce their entity target matching value to below the preset reflection matching upper limit. For heat saturated areas where the temperature value is higher than the preset ambient background temperature, the matching value of the entity target is reduced as a road obstacle; if there are abrupt changes in ground elevation, ditch boundaries or other continuous height structures at the edge of the water surface reflection area, only the matching value of the reflection center area is reduced, and edge anomalies are still retained as risk observations; The upper limit for rain curtain matching is set to 0.2, and the upper limit for reflection matching is set to 0.25. When the thermal saturation area reaches the current range limit of the infrared thermal imager and its area occupies more than 30% of the corresponding projection area, its entity target matching value as a road obstacle is no higher than 0.2. When the continuous entity structure can still be confirmed by lidar or millimeter-wave radar, only the infrared target instance is corrected, and the entity target matching value of other sensors is not corrected. The area threshold is determined according to not less than one-quarter of the robot chassis projection area and continuously covering at least four adjacent ground grids to exclude isolated reflection points. The state fusion module uses the obstacle occupancy probability of each three-dimensional spatial unit in the previous effective period as prior information to update the effective obstacle observation and effective unobstructed observation in the current period respectively; for each sensor, the observation is only regarded as evidence of the current period when its data confidence is not less than the minimum effective confidence and the observation has not exceeded the validity period. Repeated point clouds or targets from the same sensor within the same spatial cell are first merged before participating in an update; observations below the minimum effective confidence level are not used to reduce the probability of historical obstacles, but can be used to increase the risk or unknown level of that spatial cell; in one implementation, Bayesian updates are performed using log-odds; let the obstacle occupancy probability of the previous effective period be... Convert it to logarithmic odds The formula is: ; After completing the effective observation update, the log odds will be... Convert back to occupancy probability The formula is: ; First, the obstacle occupancy probability of the previous effective period is converted into a log-odds ratio. When the current observation is an obstacle observation, a base likelihood ratio greater than 1 is used to form a positive update. When the current observation is an obstacle-free observation, a base likelihood ratio less than 1 is used to form a negative update. The magnitude of the update is determined by the product of the natural logarithm of the base likelihood ratio and the data confidence level of the observation. The specific log-odds update formula is as follows: ; in, The logarithmic odds for the current period. The logarithmic odds of the previous period. The confidence level of the data for this observation. Based on the basic likelihood ratio, the higher the data confidence, the more significant the change in the state of the previous period. After all valid observations are updated, the log probability is converted back to the obstacle occupancy probability. When there are no valid environmental observations in the current period, the historical state is not updated with obstacle-free observations, but the obstacle probability of the previous valid period is retained, and the valid observation coverage of the spatial unit is set to zero or recalculated based on the existing valid observations. The baseline likelihood ratio for obstacle observations is set to 4, and the baseline likelihood ratio for unobstructed observations is set to 1 / 4. The minimum effective confidence level is set to 0.2. When multiple effective sensors update the same spatial cell, they are updated sequentially from earliest to latest according to the effective observation time. When the effective observation times are the same, they are updated according to the pre-recorded sensor number order. The updated obstacle occupancy probability is limited to between 0.01 and 0.99 to prevent a single observation from completely negating the updating effect of subsequent contradictory evidence. If no effective observations are obtained for five consecutive control cycles in the historical state, the occupancy probability is not further reduced according to unobstructed evidence, but is retained and the control risk is increased through the proportion of unknown areas. The effective observation coverage is determined by the sum of the confidence levels of sensor data capable of forming environmental observations of the three-dimensional spatial unit, divided by the number of sensors that can participate in the environmental observation of the spatial unit, and is limited to the range of zero to one; the inertial measurement unit and wheel speed encoder are only used for motion estimation and registration correction and are not included in the number of environmental observation sensors. When the denominator is zero, the effective observation coverage rate is zero and the proportion of unknown areas is one; otherwise, the proportion of unknown areas is equal to one minus the effective observation coverage rate; for sensors that cannot form effective spatial observations due to occlusion, exceeding the detection range, time compensation failure, invalid external parameters, or excessive spatial covariance, their data confidence is counted as zero. The number of sensors that can participate in environmental observation of a certain space unit is determined in the current period based on the effective field of view, effective ranging range, and space unit location of each sensor after external parameter conversion. When the space unit is originally located outside the effective field of view of a sensor, that sensor is not included in the denominator. When the space unit is located within the effective field of view but cannot form an observation due to rainstorm obstruction, data expiration, or registration failure, that sensor is included in the denominator and its confidence level is counted as zero. The system distinguishes between two situations: the inherent detection range is not covered and the current observation is invalid even though it should have been covered. When the probability of an obstacle being occupied is greater than or equal to a preset occupancy probability threshold, the spatial unit is marked as an obstacle area; when the probability of an obstacle being occupied is less than the occupancy probability threshold and the proportion of unknown areas is less than or equal to a preset unknown proportion threshold, the spatial unit is marked as a passable area; when the proportion of unknown areas is greater than the unknown proportion threshold, the spatial unit is marked as an unknown area. Obstacle area marking takes precedence over unknown area marking and passable area marking; therefore, when the obstacle occupancy probability reaches the threshold and the unknown area proportion is also high, the obstacle area marking is still retained; when the obstacle occupancy probability does not reach the threshold but the unknown area proportion exceeds the threshold, it is marked as an unknown area; in one implementation, the occupancy probability threshold is 0.65 and the unknown proportion threshold is 0.4. When the probability of obstacle occupancy is less than 0.65 but the proportion of unknown area is greater than 0.4, the spatial unit is only marked as an unknown area and not as a passable area at the same time; when the probability of obstacle occupancy reaches 0.65, it is marked as an obstacle area regardless of the proportion of unknown area; the above thresholds are associated with and saved with the corresponding robot chassis size, braking performance and sensor configuration, and are recalibrated after replacing the chassis or the main environmental observation sensors. The passage zone generation module determines the predicted braking distance based on the robot's current speed and wheel slippage index; the higher the current speed or the higher the wheel slippage index, the less the predicted braking distance decreases; the sum of the predicted braking distance and the safety redundancy distance is used as the length of the three-dimensional predicted braking area along the travel direction, the robot body width plus the lateral safety margin is used as the width of the area, and the robot body height plus the vertical safety margin is used as the height of the area; the spatial overlap of each spatial unit is determined according to the ratio of the volume of the spatial unit falling into the predicted braking area to the volume of the spatial unit, and is limited to the range of zero to one. The predicted braking distance is determined based on the effective deceleration corresponding to different wheel slip ranges of the chassis. In one implementation, when the wheel slip index is no greater than 0.1, the effective deceleration is taken as the dry road surface calibration deceleration; when the wheel slip index is greater than 0.1 but no greater than 0.3, 75% of the dry road surface calibration deceleration is taken; when the wheel slip index is greater than 0.3 but no greater than 0.6, 50% is taken; and when the wheel slip index is greater than 0.6, 30% is taken. When calculating the predicted braking distance, the braking distance is first obtained by dividing the square of the current speed by twice the effective deceleration, and then the product of the current speed and the total response time is added to obtain the response distance; the sum of the two is used as the predicted braking distance; the total response time is determined by the sum of one control cycle, the maximum effective processing delay of the current sensor, and the chassis command response delay; when the current speed is less than zero, the absolute value of its speed component along the current direction of travel is used; when there is no effective calibration value for the effective deceleration, the lowest effective deceleration is used and the corresponding spatial unit is processed in a conservative state; The safety redundancy distance is determined by the sum of half the longitudinal length of the robot body and a longitudinal grid size, and is not less than the travel distance of the chassis in one control cycle at the lowest effective deceleration; the lateral safety margin is the larger of 20% of the robot body width and a lateral grid size, and the vertical safety margin is the larger of the chassis ground clearance and a height grid size; the above dimensions are recorded during the robot configuration phase and are directly read in the current control cycle without being temporarily reduced based on observations; The control risk value is jointly determined by the obstacle occupancy probability, the proportion of unknown areas, and the spatial overlap. First, the historical obstacle status, the proportion of unknown areas, and the obstacle occupancy probability are uniformly converted into zero to one components of the risk direction, and then summed according to preset weights. The spatial overlap is used as the braking influence coefficient to increase the risk weight of spatial units located within the predicted braking area. Spatial units whose obstacle occupancy probability is lower than the preset probability threshold but whose unknown area proportion is higher than the preset proportion threshold and are located within the predicted braking area form a corresponding set control risk value; spatial units outside the predicted braking area that have unobstructed observation that meet the preset observation condition indicators have a spatial overlap of 0 and will not be directly triggered to stop due to historical risks outside the unknown area. The historical obstacle status component is determined according to the five most recent valid periods within a preset time window: 1 is taken when any period is marked as an obstacle area; 0.5 is taken when none of them are marked as obstacle areas but at least one period is marked as an unknown area; 0 is taken when all are passable areas; the base risk value is obtained by multiplying the current obstacle occupancy probability, the current unknown area proportion, and the historical obstacle status component by 45%, 35%, and 20% respectively. When the spatial overlap is 0, the control risk value is 20% of the base risk value; when the spatial overlap is greater than zero, the base risk value is multiplied by 0.5 and the sum of the spatial overlap, and any result exceeding one is limited to one; for the same observation state, the higher the degree of overlap between the spatial unit and the predicted braking area, the higher its control risk value, and the calculation order from input to output is fixed. The passability is calculated by weighting the historical obstacle status, the proportion of unknown areas, and the control risk value within a preset time window; the higher the historical obstacle status, the proportion of unknown areas, and the control risk value, the lower the passability; for adjacent spatial units, only when adjacent units continuously form an area that meets the requirements of robot width, chassis ground clearance, and minimum turning radius will it be used as a detour candidate area; if a continuous area cannot be formed, no detour command will be output. The mobility is calculated using the five most recent effective periods that are the same as the control risk value; the current unknown area proportion, the current control risk value, and the historical obstacle status component are weighted and summed at 30%, 50%, and 20% respectively to obtain the impassable component, and then the mobility is obtained by subtracting the impassable component from one, and the result is limited to the range of zero to one. When all historical states are missing, the historical obstacle state component is set to 0.5; when the current unknown area proportion is missing, it is set to 1; when the current control risk value cannot be updated, the larger value between the previous effective control risk value and 0.5 is used; the passage threshold is set to 0.6, and spatial units with a passability greater than or equal to this threshold are used to form detour candidate areas; the detour candidate areas are searched starting from the spatial units adjacent to the robot on the left and right sides of the current path, and only adjacent units marked as passable areas and whose passability reaches the passage threshold are connected; The candidate area has a width that is not less than the sum of the robot body width and twice the lateral safety margin in the horizontal direction, and a clearance in the vertical direction that is not less than the sum of the robot body height and the vertical safety margin. Furthermore, the turning radius formed from the current robot position to the center of the candidate area is not less than the minimum turning radius of the chassis. When both the left and right sides are satisfied, the side with the shorter path length is selected; when the path lengths are the same, the side with the smaller minimum control risk value is selected; when they are still the same, the unique detour direction is output according to the preset right-side priority order; the control output module selects one to output instruction in each control cycle according to the judgment order of parking conditions, unknown area conditions and detour conditions. When the distance to an obstacle within the predicted braking area is less than the sum of the predicted braking distance and the safety redundancy distance, and the obstacle occupancy probability reaches the occupancy probability threshold or the control risk value reaches the risk value threshold, a stop command is output; when there is an unknown area within the predicted braking area with a risk value exceeding the risk value threshold, and the robot's current speed is greater than the preset safe speed, a deceleration command is output; when the robot's current speed is less than or equal to the preset safe speed, a stop command is output. When there are no parking conditions or high-risk unknown conditions, but the current path's traversability is below the traversability threshold and there are consecutive adjacent traversable areas that meet the size requirements, a detour instruction is output; when none of the above conditions are met, the current inspection direction is maintained and normal passage control quantities are output according to the traversability of the current cycle. In one implementation, the risk threshold is set to 0.7, the preset safe speed is set to 20% of the robot's rated inspection speed, the obstacle distance is the projected distance from the front edge of the robot body to the near edge boundary of the nearest obstacle space unit in the predicted braking area along the travel direction, and the minimum value is taken when there are multiple obstacle space units. The target speed corresponding to the deceleration command is the smaller value between the current speed and the preset safe speed; the target speed corresponding to the stop command is zero; the detour command includes a unique detour direction determined according to the aforementioned candidate area selection rules and a target speed not exceeding the preset safe speed; when the stop condition, unknown area condition and detour condition are all met simultaneously, only the stop command is output. When distance, pose, or environmental observation data is missing in the current period, the missing result is not interpreted as risk-free; if the obstacle or the current spatial unit has been confirmed to be within the predicted braking area in the previous effective period, the corresponding historical risk is retained and the proportion of unknown areas is increased; when spatial uncertainty makes it impossible to reliably determine the distance to the obstacle, it is treated as a high-risk unknown area within the predicted braking area. Through the above processing, the sensor's own data quality, cross-sensor spatial consistency, entity target matching degree, local interference index and wheel slippage index all have clear data input, calculation order and output purpose, ultimately forming spatial unit level obstacle occupancy probability, unknown area proportion, control risk value, passability and chassis control commands.

[0029] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing and environmental perception system for an inspection robot adapted to extreme weather, characterized in that, include: The sensor acquisition module simultaneously acquires multi-source sensing data, including vision, radar, inertial navigation, and wheel speed encoder data, and records timestamps and delays. The spatial partitioning module divides the environment into three-dimensional spatial units based on the robot coordinate system and a preset three-dimensional grid resolution. The interference identification module extracts the environmental interference characteristics of multi-source sensing data within a three-dimensional spatial unit and calculates the local interference index of each sensor. The motion estimation module uses weighted fusion of inertial navigation and wheel speed encoder data to acquire speed and pose increments, calculates the wheel slip index based on the speed difference between the two, and calculates the three-dimensional predictive braking area extending along the direction of travel based on speed, direction of travel, and safety redundancy distance. The spatiotemporal registration module performs time compensation on sensor data based on timestamps, delays, and pose increments, and then transforms the sensor data to a unified coordinate system based on sensor external parameters. The confidence calculation module calculates the data confidence of each sensor within the spatial unit based on the local interference index and the wheel slippage index. The state fusion module fuses sensor data with confidence scores as weights to calculate the obstacle occupancy probability of three-dimensional spatial units, and calculates the effective observation coverage based on the sum of confidence scores to determine the proportion of unknown areas; The passage zone generation module calculates the control risk value and passability based on the occupancy probability, the proportion of unknown areas, and the spatial overlap of the three-dimensional predicted braking area. The control output module outputs deceleration, detour, or stop commands to the chassis actuators based on the control risk value and passability.

2. The image processing and environmental perception system for inspection robots adapted to extreme weather as described in claim 1, characterized in that: The multi-source sensing data is acquired by a multi-modal sensor group, which includes a visible light camera, an infrared thermal imager, a lidar, a millimeter-wave radar, an ultrasonic sensor, an inertial measurement unit, and a wheel speed encoder. The multi-source sensing data each has a corresponding acquisition timestamp and processing delay, and includes visible light images, infrared thermal images, lidar point clouds, millimeter-wave radar data, ultrasonic ranging data, inertial measurement data, and wheel speed encoder data.

3. The image processing and environmental perception system for inspection robots adapted to extreme weather as described in claim 2, characterized in that: The interference identification module extracts the projection area of ​​the visible light image in the corresponding three-dimensional spatial unit based on regional statistics and frame difference features. It then calculates the occlusion ratio, fogging ratio, contrast reduction relative to normal weather baseline, and edge continuity reduction of the projection area. Based on the occlusion ratio, fogging ratio, contrast reduction, and edge continuity reduction, it determines the local interference index of the visible light camera in the corresponding three-dimensional spatial unit.

4. The image processing and environmental perception system for inspection robots adapted to extreme weather as described in claim 2, characterized in that: The interference identification module determines that the laser point cloud is a rain curtain echo when the density increment exceeds a preset threshold, the distribution is suspended in the air within the detectable height range, and the position change of consecutive frames exceeds a preset threshold; and determines that the laser point cloud is a water surface reflection when there is no spatial matching correspondence with the millimeter-wave radar target and the error with the ground height is within a preset height error threshold. The interference identification module also determines the thermal resolution based on the target-background temperature difference, effective temperature dynamic range, and heat saturation ratio of the infrared thermal image. When the thermal resolution is lower than the preset thermal resolution threshold or the heat saturation ratio exceeds the preset heat saturation threshold, the data confidence of the infrared thermal imager in the corresponding three-dimensional spatial unit is reduced according to a preset confidence correction coefficient.

5. The image processing and environmental perception system for inspection robots adapted to extreme weather as described in claim 2, characterized in that: The motion estimation module determines the instantaneous wheel speed estimate and the instantaneous inertial speed estimate based on the inertial measurement data and the wheel speed encoder data, respectively, and calculates the wheel slippage index based on the degree of difference between the instantaneous wheel speed estimate and the instantaneous inertial speed estimate. When the wheel slippage index exceeds the preset slippage threshold, the pose calculation is performed by dynamically adjusting the weights of the wheel speed encoder, inertial measurement data, and laser point cloud in the pose calculation. At the same time, the position covariance component corresponding to the robot pose prediction covariance is increased according to the wheel slippage index.

6. The image processing and environmental perception system for an inspection robot adapted to extreme weather as described in claim 5, characterized in that: The spatiotemporal registration module determines the effective observation time by adding or subtracting the processing delay based on the acquisition timestamp, and performs time compensation and coordinate transformation on the sensor data based on the robot's pose increment between the effective observation time and the end of the control cycle. When the wheel slippage index exceeds the preset slippage threshold, the spatiotemporal registration module increases the corresponding position covariance component in the spatial covariance matrix of the registered spatial observation data according to the wheel slippage index.

7. The image processing and environmental perception system for an inspection robot adapted to extreme weather as described in claim 5, characterized in that: The confidence calculation module calculates the initial confidence based on the sensor's own data quality based on detectable parameters, the consistency of multi-sensor observations based on spatial overlap, and the degree of matching between the observed values ​​and the features of the entity target as evaluation dimensions, and then corrects it by combining the local interference index and the wheel slippage index to obtain the data confidence. For rain-reflection point clouds concentrated in a single distance layer and lacking height structure, water surface reflection point clouds with heights lower than the preset ground height and connected areas larger than the preset area threshold, and heat-saturated areas with temperatures higher than the preset ambient background temperature, the matching degree of their corresponding target instances is reduced.

8. The image processing and environmental perception system for inspection robots adapted to extreme weather as described in claim 2, characterized in that: The state fusion module adopts a Bayesian update rule to fuse observation data with obstacles according to the data confidence of each sensor in the three-dimensional spatial unit, and calculates the probability of obstacle occupancy. The effective observation coverage is determined based on the confidence level of each sensor data, and the proportion of unknown areas is calculated accordingly. When the probability of an obstacle being occupied is greater than or equal to a preset occupancy probability threshold, the three-dimensional spatial unit is marked as an obstacle region; when the probability of an obstacle being occupied is less than a preset occupancy probability threshold and the proportion of the unknown region is less than or equal to a preset unknown proportion threshold, the three-dimensional spatial unit is marked as a passable region; when the probability of an obstacle being occupied is less than a preset occupancy probability threshold and the proportion of the unknown region is greater than a preset unknown proportion threshold, the three-dimensional spatial unit is marked as an unknown region.

9. The image processing and environmental perception system for an inspection robot adapted to extreme weather as described in claim 8, characterized in that: The passage area generation module obtains the three-dimensional predicted braking area that expands along the direction of travel output by the motion estimation module, and calculates the passability based on the historical obstacle status, the proportion of unknown areas and the control risk value within a preset time window. When the distance to an obstacle within the three-dimensional predicted braking area is less than the sum of the predicted braking distance and the safety redundancy distance, and the obstacle occupancy probability exceeds or equals a preset occupancy probability threshold or the control risk value exceeds or equals a preset risk value threshold, the control output module outputs a parking command. When there is an unknown area within the three-dimensional predictive braking area where the control risk value exceeds the preset risk value threshold, if the robot's current speed is greater than the preset safe speed, a deceleration command is output; otherwise, a stop command is output. When there are no conditions to output the parking command or the deceleration command, and the passability is lower than the preset passability threshold and there are consecutive adjacent passable areas that meet the size requirements, the control output module outputs a detour command.

Images