Target positioning and path optimization method for gas inspection robot based on multi-modal sensing

Abstract

The invention belongs to the technical field of gas inspection robot positioning and path optimization, and discloses a gas inspection robot target positioning and path optimization method based on multi-modal sensing, and the method comprises the steps: constructing a four-dimensional weight model of environment, gas characteristics, sensor health degree and characteristic reliability, and designing a weight adaptive rule; for example, in a high-humidity or high-wind-speed environment, the weight ratio of the gas concentration data can be dynamically reduced, and the weight of the visual feature data can be increased at the same time; when the drift distance of the sensor reaches a set threshold value, the weight of the sensor data is reduced, a standby sensor is called, and an auxiliary correction value is generated by combining an LSTM model trained by historical data; by improving the D-S evidence theory and introducing a conflict coefficient threshold value, when different evidence bodies have conflicts, sensor health data are called for arbitration, and leakage sources are screened according to fusion confidence, so that the limitation of fixed weights is effectively avoided, and the misjudgment rate in a complex environment and a sensor abnormal scene is reduced.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of gas inspection robot positioning and path optimization, and particularly relates to a gas inspection robot target positioning and path optimization method based on multi-modal perception. BACKGROUND

[0002] Chemical industry parks and underground pipe corridors often store flammable, explosive or toxic gases, and leakage can easily cause serious accidents such as explosion and poisoning. Artificial inspection has high safety risks, limited coverage, poor real-time data, and other problems. Therefore, gas inspection robots have become the mainstream solution. The current industry mostly adopts a multi-modal technical route integrating gas, vision, and positioning modules. However, there are still the following technical problems in actual application: Existing solutions mostly use preset fixed weights, such as a gas concentration of 0.6 and a visual feature of 0.4, without considering the influence of dynamic environment on modal reliability. In a high-humidity environment with a humidity of more than 85% RH, the response of an electrochemical sensor is delayed, and the concentration detection error is increased. The fixed weight still forces reliance on gas data, leading to an increase in the misjudgment rate of suspected leakage sources. When the sensor has a zero drift of more than 5% FS, there is a lack of a dynamic weight reduction mechanism, further amplifying the error.

[0003] Although some solutions introduce a Gaussian plume diffusion model, key parameters such as leakage rate and diffusion coefficient are mostly preset statically, without combining real-time motion data such as dynamic acceleration and turning angle collected by an IMU. When the robot speeds up, the sampling point and the model-predicted concentration distribution are easily misaligned, the positioning error is enlarged, and it is still not possible to distinguish between the fluctuation of the moving sampling concentration and the real concentration change of the leakage source, affecting the positioning accuracy.

[0004] Existing dynamic path planning is only triggered by new leakage detection or obstacle avoidance, without considering the concentration change of the leakage source after inspection as feedback. When the leakage is alleviated, the inspection priority is not reduced, and the same area is still covered repeatedly. When the leakage is intensified, it is not possible to schedule a second inspection preferentially, forming a planning-execution-feedback breakpoint, and ultimately, the proportion of invalid inspection paths is high, and the response to the core leakage point is delayed. SUMMARY

[0005] The purpose of the present application is to provide a gas inspection robot target positioning and path optimization method based on multi-modal perception to solve the problems raised in the background.

[0006] In order to achieve the above purpose, the present application provides the following technical scheme: a gas inspection robot target positioning and path optimization method based on multi-modal perception, the specific steps of which are as follows: Build a hardware system: including a gas perception module, an environment perception module, a visual perception module, a positioning reference module, a data processing and communication module, and a collaborative control module, for data collection; Data preprocessing: Kalman filter is used to eliminate noise and double-factor correction for gas data, dark channel prior algorithm is used to defog for visual data, and untraceable Kalman filter is used to fuse three sources of data and adjust the weight for positioning data. Finally, the gas and visual data are time-aligned; Multi-modal feature fusion: a four-dimensional weight model of environment, gas characteristics, sensor health, and feature reliability is established, a weight self-adaptive rule is designed, an improved D-S evidence theory is used, three core evidence bodies are defined, and suspected leakage sources are screened; Leakage source positioning: based on the suspected leakage sources, a dynamic Gaussian plume model is used to calculate the diffusion coefficient, the parameters are corrected through multi-source calibration, and the improved particle filter is used to locate the coordinates of the leakage source; Initial path planning: a four-dimensional priority matrix is established by dividing the leakage priority, an improved A path planning algorithm is used to design a comprehensive cost function, and an optimal path is planned; Feedback real-time path optimization: three types of triggering scenarios are designed, NSGA-II non-dominated sorting genetic algorithm is used, four optimization objectives are set, the optimal path is selected, and multi-robot collaborative task allocation is performed; Positioning and path optimization effect verification: three types of typical scenarios are designed to test the core indicators including dynamic positioning accuracy and response time, and a mapping database of environmental parameters, fusion weights, and model parameters is established.

[0007] Preferably, the hardware system is specifically as follows: The gas sensing module adopts a four-array design of double-channel electrochemical sensors, infrared sensors, and PID photoionization sensors, and is equipped with a sensor health monitoring circuit to collect zero voltage, response time, and drift in real time; The environmental perception module includes temperature and humidity sensors and three-dimensional wind speed and direction sensors for collecting pipeline surface temperature. The visual perception module is provided with an industrial camera and an infrared thermal imager, the lens is provided with an anti-fog coating and an automatic cleaning device, and three types of leakage-related visual features of pipeline cracks, valve corrosion, and condensation traces are extracted in real time; The positioning reference module adopts a three-source fusion design of UWB ultra-wideband positioning, laser radar SLAM, and high-precision IMU, dynamically builds a high-risk area coordinate map, and outputs the real-time pose of the robot. The data processing and communication module is provided with an edge computing node and a 5G industrial gateway. The edge node is responsible for local processing of multi-modal data preprocessing, weight calculation, and positioning iteration, and only uploads the optimized path and leakage source results to the cloud. The collaborative control module adds a multi-robot collaborative communication interface.

[0008] Preferably, the data preprocessing is specifically as follows: After the multi-source data collection is completed, the gas data preprocessing uses Kalman filtering to filter out sensor drift noise, and corrects the data through a temperature and humidity and gas type double-factor correction formula combined with the environmental perception module data, compensates for the concentration error, and introduces a sensor health compensation mechanism. When the sensor drift reaches 80% of the threshold, the backup sensor data is called, and the LSTM model trained by historical data generates an auxiliary correction value; In the visual data preprocessing, the industrial camera image is de-fogged using the dark channel prior algorithm, and the pipeline cracks and valve corrosion areas are extracted by Canny edge detection. The infrared thermal imaging image is segmented using an adaptive temperature threshold to extract the suspected leakage point high-temperature area and label the area size and center point coordinates. The positioning data preprocessing uses an unscented Kalman filter to realize the fusion of IMU, UWB and SLAM three-source data, and dynamically adjusts the weight of each source according to the robot motion state. Finally, the timestamps of the positioning reference module are used as the basis for time alignment of the gas and visual data, and all data are standardized to the [0, 1] interval.

[0009] Preferably, the multi-modal feature fusion is as follows: After the perception data preprocessing is completed, first, the weight self-adaptive rule is designed, including the environmental dimension, gas characteristic dimension, sensor health dimension, and feature reliability dimension. Then, the improved D-S evidence theory is used to define three core evidence bodies: gas concentration peak greater than threshold C0, visual recognition of leakage trace confidence greater than 0.6, and positioning coordinates located in a high-risk area. A conflict coefficient threshold λ = 0.8 is introduced. When the conflict between two evidence bodies is greater than λ, the sensor health data is called to arbitrate. When the conflict is less than or equal to λ, the fusion confidence is calculated according to the classical D-S synthesis rule. Finally, the suspected leakage source is selected according to the fusion confidence.

[0010] Preferably, the leakage source positioning is as follows: After the suspected leakage source is determined, a dynamic Gaussian plume diffusion model is used to calculate the diffusion coefficient based on the robot's real-time pose, motion direction, and environmental parameters. The output is the predicted gas concentration at different times and positions, which is compared with the actual sampling concentration of the robot. The leakage point size identified by the industrial camera is combined with the fluid mechanics formula to back-calculate the leakage rate Q. The real-time input of Q into the diffusion model corrects the parameters, and the actual concentration gradient of the robot at three consecutive sampling points is calculated. The model prediction gradient is compared, and if the deviation is greater than 20%, the resampling weight of the particle filter is adjusted. The improved particle filter positioning is centered on the suspected leakage source, 1000 particles are generated within a range of 5m*5m, the initial weight of the particles is related to the multi-modal fusion confidence, the particle position is updated in combination with the robot motion data in each iteration, the particle weight is adjusted according to the deviation between the actual concentration and the predicted concentration, the particles with a weight less than 0.05 are removed and new particles are supplemented, and the particle with a weight in the top 20% after iteration is retained, and the cluster center is the coordinate of the leakage source.

[0011] Preferably, the initial path planning is specifically as follows: After obtaining the leakage source coordinates, a four-dimensional priority matrix of concentration level*distance to the core device*leakage rate*diffusion area growth rate is established for leakage priority division, the weight of each dimension is dynamically adjusted according to the scene, the priority score is calculated and divided into special, high, medium and low levels according to the score, wherein the special priority leakage source weight*5, the high level*3, the medium level*2, and the low level*1 to guide the path planning priority; The improved A path planning algorithm designs a comprehensive cost function, so that the path preferentially covers the high-priority and high-risk areas, and finally takes the current position of the robot as the starting point, calculates the cost of each node according to the cost function, and selects the path with the minimum cost as the initial inspection path.

[0012] Preferably, the feedback real-time path optimization is specifically as follows: After the initial inspection path planning is completed, the optimization trigger conditions include triggering emergency path insertion and preferentially scheduling the robot to go to the new leakage source area when a new suspected leakage source is detected, triggering local path re-planning when encountering an unplanned obstacle, and adopting a circular arc bypass strategy to avoid collision, and triggering priority iteration and path adjustment when the concentration of the leakage source changes by more than a threshold value after inspection; The multi-objective optimization algorithm adopts the NSGA-II non-dominated sorting genetic algorithm, sets four optimization objectives of the shortest total path length, the shortest response time of special and high priority leakage sources, the fastest leakage source concentration drop rate, and the lowest path repetition rate, generates a Pareto optimal path set in each iteration, and the robot selects the optimal path according to the real-time environment; In multi-robot cooperative optimization, when a single robot detects more than 2 special priority leakage sources, it sends a cooperative request containing the leakage source coordinates, concentration, and priority information to the adjacent robots through the 5G gateway, and allocates tasks based on distance, concentration, and robot load.

[0013] Preferably, the positioning and path optimization effect verification is specifically as follows: The test scene design covers three typical scenes of chemical industry park pipeline area, underground pipe gallery and open-air tank area, and the scene parameters are matched with the hardware adaptation requirements; The core verification indexes include dynamic positioning accuracy of a leakage source positioning error of robot movement of less than or equal to ± 30 cm, emergency response time of a special priority leakage source of less than or equal to 30 s from detection to first inspection, long-term stability of sensor drift of less than or equal to 3 % FS and backup sensor switching success rate of 100 % without detection interruption, path optimization effect of invalid inspection path ratio of less than or equal to 15 %, and cooperation efficiency of a total response time of less than or equal to 45 s when two robots cooperatively process three special leakage sources; The parameter iterative optimization establishes an environment parameter, fusion weight and model parameter mapping database based on multi-scene test data, collects real-time environment parameters through the environment perception module after the robot is started, automatically matches the optimal parameters in the database without manual debugging, stores new data into the database after completing each inspection, and updates the mapping relationship through a gradient descent algorithm.

[0014] The present application has the following advantages: 1. The present application constructs a four-dimensional weight model of environment, gas characteristics, sensor health and feature reliability, and designs a weight self-adaptive rule. For example, in a high humidity or high wind speed environment, the weight proportion of gas concentration data is dynamically reduced, and the weight of visual feature data is increased. When the sensor drift reaches a set threshold, not only the weight of the sensor data is reduced, but also a backup sensor is called, and an auxiliary correction value is generated by combining the LSTM model trained by historical data. In addition, by improving the D-S evidence theory, a conflict coefficient threshold is introduced. When there is a conflict between different evidence bodies, the sensor health data is called for arbitration, and then the fusion confidence is screened to screen the leakage source. This design can dynamically match the reliability of each modal data, effectively avoid the limitations of fixed weight, and significantly reduce the misjudgment rate in complex environments and sensor abnormal scenes.

[0015] 2. The present application uses a dynamic Gaussian plume model, taking real-time robot pose, motion direction, speed and turning angle as input, and calculates the dynamic diffusion coefficient through the optimized formula, avoiding misalignment between static parameters and robot movement sampling process. At the same time, the leakage rate is back calculated combined with the leakage point size recognized by the industrial camera, and the actual concentration gradient of multiple consecutive sampling points is compared with the model predicted gradient. If the deviation exceeds the set range, the resampling weight of particle filtering is adjusted in time to realize multi-source data calibration. The improved particle filtering generates particles with the suspected leakage source as the center, the initial weight of the particle is related to the multi-modal fusion confidence, low weight particles are removed and new particles are supplemented in the iteration process, and finally the clustering center of high weight particles is taken as the leakage source coordinate. This design can dynamically correct model parameters, eliminate interference caused by robot movement, accurately distinguish sampling concentration fluctuation and real leakage change, and control the positioning error within a reasonable range.

[0016] 3, The application constructs a feedback real-time path optimization mechanism, designs three types of triggering scenarios: when a new suspected leakage source is detected, trigger emergency path insertion and prioritize the robot to go; When encountering an unplanned obstacle, trigger local path re-planning and adopt a circular arc bypass strategy to avoid collision; When the concentration of the leakage source changes beyond the threshold after inspection, trigger leakage priority iteration and path adjustment; Adopt NSGA-II non-dominated sorting genetic algorithm, set the optimization goals of path total length, special and high priority leakage source response time, leakage source concentration drop rate, path repetition rate, etc. Generate a set of Pareto optimal paths for the robot to choose; When a single robot detects multiple special priority leakage sources, send a coordination request to nearby robots through the 5G gateway, and distribute tasks based on distance, concentration and robot load; This design effectively reduces invalid inspection paths, significantly shortens the response time of special priority leakage sources, significantly improves the efficiency of multi-robot cooperative processing of leakage, and further ensures the safety of inspection in high-risk scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0017] Fig. 1 The flow chart of the gas inspection robot target positioning and path optimization method based on multi-modal perception of the application; Fig. 2 The flow chart of the adaptive fusion and suspected leakage source screening of the application; Fig. 3 The flow chart of the leakage source positioning and path optimization of the application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0019] As shown in the drawings, Figs. 1 to 3 The embodiment of the application provides a gas inspection robot target positioning and path optimization method based on multi-modal perception, and the specific steps of the method are as follows: Build a hardware system: including gas perception module, environment perception module, visual perception module, positioning reference module, data processing and communication module, cooperative control module, for data acquisition; Data preprocessing: Kalman filter is used for gas data to eliminate noise and double-factor correction, dark channel prior algorithm is used for visual data to remove fog, and untraceable Kalman filter is used for positioning data to fuse three source data and adjust weight, finally time alignment is performed on gas and visual data; Multi-modal feature fusion: Establish a four-dimensional weight model of environment, gas characteristics, sensor health, and feature reliability, design weight self-adaptive rules, use improved D-S evidence theory, define three core evidence bodies, and screen suspected leakage sources; Leakage source positioning: Based on the suspected leakage sources, use a dynamic Gaussian plume model to calculate the diffusion coefficient, correct the parameters through multi-source calibration, and combine the improved particle filter to locate the leakage source coordinates; Initial path planning: Leak priority division establishes a four-dimensional priority matrix, uses an improved A path planning algorithm to design a comprehensive cost function, and plans the optimal path; Feedback real-time path optimization: Design three types of triggering scenarios, use NSGA-II non-dominated sorting genetic algorithm, set four optimization objectives, select the optimal path, and distribute tasks in multi-robot cooperation; Positioning and path optimization effect verification: Design three types of typical scene tests to verify the core indicators including dynamic positioning accuracy and response time, and establish a mapping database of environmental parameters, fusion weights, and model parameters.

[0020] The hardware system is built as follows: Around the core needs of gas inspection in high-risk scenes such as chemical industry parks and underground pipe galleries, a hardware system of redundant perception, edge computing, and collaborative communication is constructed to provide a foundation for subsequent data collection and processing.

[0021] The gas perception module adopts a four-array design of dual-channel electrochemical sensors, infrared sensors, and PID photoionization sensors, with a sampling frequency of 1Hz~5Hz adjustable, and a supporting sensor health monitoring circuit to collect zero voltage, response time, and drift in real time; The dual-channel electrochemical sensors detect methane (0~100%LEL) and hydrogen sulfide (0~100ppm), the infrared sensor detects carbon dioxide (0~5%vol), and the PID photoionization sensor detects VOCs (0~2000ppm); The environmental perception module includes high-precision temperature and humidity sensors and three-dimensional wind speed and direction sensors to collect pipeline surface temperature and provide environmental parameters for subsequent gas concentration correction and diffusion model; The visual perception module is equipped with an industrial camera and a high-definition infrared thermal imager, with anti-fog coating and automatic cleaning device on the lens to adapt to the high-humidity and dusty environment of underground pipe galleries, and real-time extraction of three types of leakage-related visual features of pipeline cracks, valve corrosion, and condensation traces; The positioning reference module adopts a three-source fusion design of UWB ultra-wideband positioning, laser radar SLAM and high-precision IMU, and labels the coordinates of high-risk areas such as pipeline joints, valves and flanges during dynamic mapping, and outputs the real-time pose of the robot; The data processing and communication module is provided with an edge computing node and a 5G industrial gateway, the edge node is responsible for local processing of multi-modal data preprocessing, weight calculation, positioning iteration, and only the optimized path and leakage source results are uploaded to the cloud, thereby avoiding the real-time deficiency caused by dependence on the cloud; The collaborative control module adds a multi-robot collaborative communication interface, which can send task allocation instructions to adjacent robots through the interface when a single robot detects multi-source high-risk leakage, realizing cross-area collaborative inspection. The temperature and humidity sensor measurement range is -20~60℃ (accuracy ±0.5℃), 0~100%RH (accuracy ±3%RH); the three-dimensional wind speed and direction sensor measurement range is 0~15m / s (accuracy ±0.1m / s); the UWB positioning accuracy is ±15cm, and the IMU angular velocity accuracy is ±0.1° / h; The data preprocessing is as follows: After completing multi-source data acquisition, the gas data preprocessing uses Kalman filtering to filter out sensor drift noise, and corrects it through a temperature and humidity and gas type double-factor correction formula combined with the data of the environment perception module, compensates for the concentration error, and introduces a sensor health compensation mechanism, when the sensor drift reaches a threshold of 80%, the backup sensor data is called, and the LSTM model trained by historical data generates auxiliary correction values, thereby ensuring the continuity of the concentration data; The LSTM model historical data comes from 1000+ hours of sensor data under the same working conditions accumulated by the robot, including concentration measured values under different temperature and humidity and drift amounts; the input variables are sensor drift, real-time temperature and real-time humidity, and the output variable is the gas concentration correction value; the model structure is 3 layers of hidden layers, the optimizer uses Adam, and the loss function is mean square error (MSE); In the visual data preprocessing, after the industrial camera image is de-fogged by the dark channel prior algorithm, the pipeline cracks and valve rust areas are extracted by Canny edge detection, the suspected leakage point high temperature area is extracted by adaptive temperature threshold segmentation of the infrared thermal imaging image, and the area and center point coordinates are labeled; The positioning data preprocessing adopts unscented Kalman filtering to realize the fusion of IMU, UWB and SLAM three-source data, and dynamically adjusts the weight of each source according to the motion state of the robot; When accelerating, the IMU weight is 0.6, the UWB weight is 0.2, and the SLAM weight is 0.2; when uniform speed, the UWB weight is 0.5, the SLAM weight is 0.4, and the IMU weight is 0.1; when turning, the SLAM weight is 0.5, the IMU weight is 0.3, and the UWB weight is 0.2, to ensure the dynamic pose accuracy; Finally, the gas and visual data are time-aligned based on the timestamp of the positioning reference module, and all data are standardized to the interval [0, 1].

[0022] Gas concentration two-factor correction formula: ; In the formula: represents the gas concentration after two-factor correction, with units of ppm or vol%, which is adapted according to the gas type; represents the original concentration data collected directly by the gas sensor, without eliminating noise and environmental interference; represents the temperature correction coefficient, which is determined by sensor calibration experiments, and is used to compensate for the influence of temperature changes on concentration detection; represents the current temperature, with units of °C, which is collected in real time by the temperature and humidity sensor of the environmental perception module and needs to be aligned with the gas collection time; represents the standard ambient temperature of sensor calibration, with units of °C, which is usually 25°C and is a fixed value; represents the humidity correction coefficient, which is determined by sensor calibration experiments, and is used to compensate for the influence of humidity changes on concentration detection; represents the current relative humidity, with units of %RH, which is collected in real time by the temperature and humidity sensor of the environmental perception module and needs to be aligned with the gas collection time; represents the standard ambient relative humidity of sensor calibration, with units of %RH, which is usually 50%RH and is a fixed value; represents the gas type correction coefficient, which is a preset correction coefficient for different gases (such as methane, VOCs, and toxic gases) and is obtained by fitting experimental data; represents the gas type identification parameter, which is assigned according to the gas category, such as methane = 1, VOCs = 2, and toxic gas = 3, which is used to match the corresponding value.

[0023] Among them, the multi-modal feature fusion is as follows: After completing the perception data preprocessing, first, the weight self-adaptive rule is designed: in high humidity or high wind speed environment, the gas concentration weight is down-regulated by 20%-40%, and the visual feature weight is up-regulated by 30%-50%; in low humidity and low wind speed environment, the gas concentration weight is up-regulated by 10%-20%, and the visual weight is down-regulated by 5%-15%. Gas characteristic dimension: toxic gas concentration gradient > 50 ppm / m, gas weight is additionally increased by 15-20%, combustible gas concentration close to 1 / 3 of the lower explosive limit triggers "gas and visual" dual-mode weight locking, VOCs concentration > 100 ppm, PID sensor data weight ≥ 0.6 and other gas sensor weight is reduced; Sensor health dimension: when the gas sensor drift exceeds the threshold, the weight is reduced to 0.2 or less, when it reaches 80%-100% of the threshold, the weight is reduced to 0.3-0.2, and when it is less than 80% of the threshold, it is adjusted linearly according to "1-drift / threshold"; Feature reliability dimension: when the visual recognition crack / corrosion confidence is greater than 0.8, the weight is increased by 10-15%, when it is between 0.6 and 0.8, it remains at the baseline value, and when it is less than 0.6, it is reduced by 20-30% and the cooperative weight of gas concentration and positioning information is simultaneously increased; Then, improved D-S evidence theory is used to define three core evidence bodies: gas concentration peak greater than threshold C0, visual recognition of leakage trace confidence greater than 0.6, and positioning coordinates in high-risk area. A conflict coefficient threshold λ = 0.8 is introduced. When the conflict between two evidence bodies is greater than λ, the sensor health data is used for arbitration. When the conflict is less than or equal to λ, the fusion confidence is calculated according to the classical D-S synthesis rule. Finally, the suspected leakage source is screened according to the fusion confidence; When the confidence is greater than 0.85, it is a high-confidence leakage source and directly enters the positioning link. When it is between 0.6 and 0.85, it is a leakage source to be verified and triggers secondary sampling verification. When it is less than 0.6, it is excluded to avoid false positives; The gas concentration peak threshold C0 is set according to the type of gas detected. For methane, C0 = 5000 ppm (corresponding to 10% of the 5% vol lower explosive limit), and for VOCs, C0 = 200 ppm (referring to the safety limit value in GB27632-2011); The leakage rate is calculated using the orifice outflow formula Q = Cd × A × √(2ΔP / ρ), where Cd is the flow coefficient (0.6-0.8, determined according to the shape of the leakage hole), A is the area of the leakage hole (calculated from the crack size identified by the industrial camera), ΔP is the pressure difference between the inside and outside of the pipeline (obtained from the process data of the park), and ρ is the gas density (standard value according to gas type).

[0024] The leakage source positioning is as follows: After determining the suspected leakage source, the dynamic Gaussian plume diffusion model takes the real-time pose, motion direction, and environmental parameters of the robot as input, calculates the diffusion coefficient using the optimized diffusion coefficient formula, outputs the predicted gas concentration at different times and positions, and compares it with the actual sampling concentration of the robot to form a benchmark; In the multi-source data calibration link, the size of the leakage point identified by the industrial camera is combined with the fluid mechanics formula to back-calculate the leakage rate Q. Q is input into the diffusion model in real time to correct the parameters. Meanwhile, the actual concentration gradient of the three consecutive sampling points of the robot is calculated and compared with the model predicted gradient. If the deviation is greater than 20%, the resampling weight of the particle filter is adjusted to ensure that the model matches the actual concentration distribution. The improved particle filter positioning is centered on the suspected leakage source, 1000 particles are generated within a 5m x 5m range, and the initial weight of the particles is related to the multi-modal fusion confidence. In each iteration, the particle position is updated combined with the robot motion data, the particle weight is adjusted according to the deviation between the actual concentration and the predicted concentration, the particles with a weight less than 0.05 are removed and new particles are supplemented. After iteration, the top 20% of particles are retained, and the cluster center is the leakage source coordinates and controls the positioning error. Dynamic diffusion coefficient calculation formula: ; In the formula: D(t) represents the dynamic gas diffusion coefficient at time t, with a unit of , which changes with time and is used to predict the gas concentration at different positions in the Gaussian plume model; D0 represents the static gas diffusion coefficient, with a unit of , which refers to the diffusion coefficient in a standard environment and is determined by the physical properties of the gas (such as small molecule gas in air), and is a fixed value; V represents the velocity influence coefficient, which is an experimental fitting parameter, used to quantify the interference of robot motion speed on gas diffusion, and usually takes a value of 0.02~0.05; V(t) represents the real-time motion speed of the robot at time t, with a unit of , which is obtained from the motion state data collected by the positioning reference module (IMU or SLAM); θ(t) represents the angle between the robot motion direction and the dominant direction of gas diffusion at time t, with a unit of rad, which is determined by the three-dimensional wind speed and direction sensor of the environment perception module, and is calculated based on the robot motion direction; φ represents the turning angle influence coefficient, which is an experimental fitting parameter, used to quantify the interference of robot turning on gas diffusion, and usually takes a value of 0.01~0.03; φ(t) represents the rate of change of the robot turning angle at time t, with a unit of , which is collected by the IMU of the positioning reference module and reflects the severity of the robot turning, and is used to compensate for the sampling point deviation caused by turning.

[0025] Wherein, the initial path planning is specifically as follows: After obtaining the leakage source coordinates, the leakage priority is divided to establish a four-dimensional priority matrix of concentration level x distance from core equipment x leakage rate x diffusion area growth rate, and the weight of each dimension is dynamically adjusted according to the scene, Concentration level > lower explosive limit 1 / 2 5 points, 1 / 3~1 / 2 4 points, distance from core equipment <5m 5 points, 5~10m 4 points, leakage rate > 0.1m 3 / s 5 points, 0.05~0.1m 3 / s 4 points, diffusion area growth rate > 1m 2 / min 5 points, 0.5~1m 2 / min 4 points; The priority score is calculated and divided into special, high, medium and low levels according to the score, wherein the special priority leakage source weight x 5, the high level x 3, the medium level x 2, and the low level x 1 to guide the priority of path planning; The improved A path planning algorithm designs a comprehensive cost function, so that the path preferentially covers high priority and high risk areas, and finally takes the current position of the robot as the starting point, calculates the cost of each node according to the cost function, and selects the path with the minimum cost as the initial inspection path to meet the real-time demand.

[0026] Path planning comprehensive cost function: ; =1; In the formula: represents the comprehensive cost of the path node, and the smaller the cost, the better the path, which is used for A* algorithm to select the optimal path node; represents the path length weight, which can be dynamically adjusted, such as open area , complex obstacle area , used to balance the length of path and inspection efficiency; represents the path length from the current node to the target node, and the unit is m, which is calculated based on the prior map of the inspection area and the real-time pose of the robot; represents the leakage priority weight, which can be dynamically adjusted, such as special leakage area, , low level leakage area , to ensure that the path preferentially covers high-risk leakage sources; represents the leakage source priority score, which is calculated by concentration level distance from core equipment leakage rate The four-dimensional matrix calculation of the diffusion area growth rate shows that the score range is 1-20, the special level is Z16, the high level is 12-15, the medium level is 8-11, and the low level is 7; The obstacle avoidance cost weight is represented, which can be dynamically adjusted, such as in the obstacle dense area =0.3, and the obstacle-free area =0.1, which is used to avoid collision between the path and the obstacle; The obstacle avoidance cost is represented, which is calculated based on the laser radar SLAM mapping data, the closer the obstacle is, the larger the volume is, The higher the value is, the range is 1-5; The diffusion risk cost weight is represented, which can be dynamically adjusted, such as in the area with fast diffusion speed =0.3, and the area with slow diffusion =0.1, which is used to preferentially process the leakage source with aggravated risk; The diffusion risk cost is represented, which is calculated based on the gas diffusion area growth rate, and the growth rate is >1 When min, R=5, 0.5 When min, R=3, When min, R=1.

[0027] The leakage priority score calculation formula is: ; In the formula, The total score of the leakage source priority is represented, and the score determines the priority level of the leakage source, the special level is ≥16, the high level is 12-15, the medium level is 8-11, and the low level 7; i takes values 1, 2, 3, and 4, respectively, representing the concentration, distance, rate, and diffusion of the four core risk dimensions; The basic score of the dimension is represented, and the score standards of the four dimensions are respectively: 1. Concentration level The lower explosive limit is 5 points, 1 / 3-1 / 2 is 4 points, 1 / 4-1 / 3 is 3 points, and <1 / 4 is 2 points; 2. Distance to core equipment 5 points are given to 5-10 m, 4 points are given to 10-15 m, 3 points are given to 15-20 m, and 2 points are given to >20 m; 3. Leakage rate 5 points are given to 0.1-0.2 4 points are given to 0.05-0.1 3 points are given to 0.01-0.05 2 points are given to <0.01 4. Diffusion area growth rate (m2 / min): 5 points for >1 m2 / min, 4 points for 0.5 m2 / min, 3 points for 0.1-0.5 m2 / min, and 2 points for <0.1 m2 / min; Wi represents the weight of the ith dimension, which can be dynamically adjusted, such as the core area of a chemical industrial park = 0.4, and the open storage tank area = 0.4), and satisfies = 1.

[0028] The feedback real-time path optimization is as follows: After the initial inspection path planning is completed, the optimization triggering conditions include triggering emergency path insertion and prioritizing the robot to go to the new leakage source area when a new suspected leakage source is detected, triggering local path re-planning when an unexpected obstacle is encountered, and adopting a circular arc bypass strategy to avoid collision, triggering priority iteration and path adjustment when the concentration of the leakage source changes beyond the threshold after inspection; The circular arc radius = robot body width (assuming 0.5 m) + 0.3 m safety distance, i.e., radius ≥ 0.8 m, to ensure that the robot does not collide with obstacles when bypassing; Robot load = total score of allocated leakage source priority (special class × 5 + high class × 3 + medium class × 2 + low class × 1), and when coordinating allocation, new leakage sources are preferentially allocated to robots with a load <10 points and the closest distance; When the leakage intensifies, i.e., the concentration rises by more than 20% in 5 minutes, the priority is increased by 2 levels and the robot performs secondary inspection within 10 minutes, and the path priority is raised to the highest level. When the leakage is alleviated, i.e., the concentration drops by more than 30% in 10 minutes, the priority is reduced by 1 level and the inspection interval is extended to 1.5 times the original interval, and the path is delayed. When suspected misjudgment occurs, i.e., the concentration does not fluctuate after 3 rounds of inspection, it is marked as "to be verified" and combined with visual rejudgment. If there is no leakage after rejudgment, the path is excluded. When the leakage is stable, i.e., the concentration fluctuates by less than 10% in 10 minutes, the original level is maintained and the inspection is performed at the original interval; The multi-objective optimization algorithm uses the NSGA-II non-dominated sorting genetic algorithm, sets four optimization objectives: shortest total path length, shortest response time for special and high priority leakage sources, fastest concentration drop rate for leakage sources, and lowest path repetition rate, and sets the algorithm parameters as population size 50, iteration number 30, crossover probability 0.8, and mutation probability 0.1. The Pareto optimal path set is generated each iteration, and the robot selects the optimal path based on the real-time environment; In multi-robot cooperative optimization, when a single robot detects more than 2 special priority leakage sources, it sends a cooperation request containing the leakage source coordinates, concentration, and priority information to nearby robots through the 5G gateway, assigns tasks based on distance, concentration, and robot load to ensure synchronous response to multi-source leakage and improve inspection efficiency.​

[0029] The positioning and path optimization effect verification are specifically as follows: The test scene design covers three typical scenes of chemical industry park pipeline area (preset 3 leakage sources, environment temperature 28℃, humidity 65%, wind speed 2m / s), underground pipe gallery (preset 2 dynamic leakage sources, environment temperature 22℃, humidity 90%, low wind speed, and much dust), and open-air storage tank area (preset 2 leakage sources, environment temperature 35℃, humidity 40%, high wind speed, and strong sunlight), and the scene parameters are matched with the hardware adaptation requirements. The core verification indexes include dynamic positioning accuracy of the robot moving, i.e., the positioning error of the leakage source is less than or equal to ±30cm, emergency response time of the special priority leakage source, i.e., the time from detection to the first inspection is less than or equal to 30s, long-term stability of the sensor drift, i.e., the drift is less than or equal to 3%FS and the standby sensor switching success rate is 100% without detection interruption, path optimization effect, i.e., the proportion of invalid inspection paths is less than or equal to 15%, and the cooperation efficiency of two robots in processing three special leakage sources, i.e., the total response time is less than or equal to 45s. The parameter iterative optimization establishes an environment parameter, fusion weight, and model parameter mapping database based on multi-scene test data, the robot collects real-time environment parameters through the environment perception module after starting, automatically matches the optimal parameters in the database without manual debugging, stores new data (environment parameters, optimized parameters, and verification indexes) into the database after completing each inspection, updates the mapping relationship through the gradient descent algorithm, and ensures the performance stability in different scenes.

[0030] It should be noted that, in this text, relational terms such as first and second are used merely to distinguish one entity or action from another, without necessarily requiring or implying any such actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof are intended to cover non-exclusive inclusions, so that a process, method, article, or apparatus including a series of elements includes not only those elements, but also other elements not explicitly listed, or inherent to such a process, method, article, or apparatus.

[0031] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made thereto without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A target localization and path optimization method for a gas inspection robot based on multimodal perception, characterized in that: The specific steps of this method are as follows: The hardware system includes a gas sensing module, an environmental sensing module, a visual sensing module, a positioning reference module, a data processing and communication module, and a collaborative control module for data acquisition. Data preprocessing: Kalman filtering was used to eliminate noise and perform two-factor correction on gas data; dark channel prior algorithm was used to remove fog from visual data; and unscented Kalman filtering was used to fuse the three-source data and adjust the weights on the localization data. Finally, time alignment was performed on the gas and visual data. Multimodal feature fusion: A four-dimensional weighted model of environment, gas characteristics, sensor health and feature reliability is established, weight adaptive rules are designed, and an improved DS evidence theory is adopted to define three core evidence bodies to screen suspected leakage sources. Leakage source location: Based on the suspected leakage source, the diffusion coefficient is calculated using a dynamic Gaussian plume model, and the parameters are corrected through multi-source calibration. Combined with an improved particle filter, the coordinates of the leakage source are located. Initial path planning: A four-dimensional priority matrix is ​​established based on leakage priority division, and an improved A-path planning algorithm is used to design a comprehensive cost function to plan the optimal path; Feedback-based real-time path optimization: Three types of triggering scenarios are designed, the NSGA-II non-dominated sorting genetic algorithm is adopted, four optimization objectives are set, the optimal path is selected, and multiple robots collaboratively allocate tasks; Verification of localization and path optimization effects: Design three typical scenario tests to verify core indicators including dynamic positioning accuracy and response time, and establish a mapping database of environmental parameters, fusion weights and model parameters.

2. The target localization and path optimization method for gas inspection robots based on multimodal perception according to claim 1, characterized in that: The specific hardware system setup is as follows: The gas sensing module adopts a four-array design consisting of a dual-channel electrochemical sensor, an infrared sensor, and a PID photoionization sensor, and is equipped with a sensor health monitoring circuit to collect zero-point voltage, response time, and drift in real time. The environmental perception module includes a temperature and humidity sensor and a three-dimensional wind speed and direction sensor, used to collect the surface temperature of the pipeline; the visual perception module is equipped with an industrial camera and an infrared thermal imager, with the lens equipped with an anti-fog coating and an automatic cleaning device, to extract the visual features associated with three types of leaks in real time: pipeline cracks, valve corrosion, and condensation. The positioning reference module adopts a three-source fusion design of UWB ultra-wideband positioning, LiDAR SLAM and high-precision IMU. During dynamic mapping, the coordinates of high-risk areas are marked and the robot's real-time pose is output. The data processing and communication module is equipped with edge computing nodes and a 5G industrial gateway. The edge nodes are responsible for localized processing of multimodal data preprocessing, weight calculation and positioning iteration, and only upload the optimized path and leakage source results to the cloud. The collaborative control module adds a multi-robot collaborative communication interface.

3. The target localization and path optimization method for gas inspection robots based on multimodal perception according to claim 2, characterized in that: The data preprocessing is as follows: After completing the multi-source data acquisition, the gas data preprocessing uses Kalman filtering to remove sensor drift noise. Combined with the environmental sensing module data, it is corrected using a two-factor correction formula of temperature, humidity and gas type to compensate for concentration error. At the same time, a sensor health compensation mechanism is introduced. When the sensor drift reaches a threshold of 80%, backup sensor data is called. An auxiliary correction value is generated by an LSTM model trained with historical data. In visual data preprocessing, industrial camera images are dehazed using a dark channel prior algorithm, and Canny edge detection is used to extract pipe cracks and valve corrosion areas. Infrared thermal images are segmented using an adaptive temperature threshold to extract high-temperature areas of suspected leaks and to mark the area area and center point coordinates. In localization data preprocessing, unscented Kalman filtering is used to fuse IMU, UWB, and SLAM data, and the weights of each source are dynamically adjusted in conjunction with the robot's motion state. Finally, the gas and visual data are time-aligned based on the timestamp of the localization reference module, and all data are standardized to the [0, 1] interval.

4. The target localization and path optimization method for gas inspection robots based on multimodal perception according to claim 3, characterized in that: The multimodal feature fusion is described in detail below: After completing the preprocessing of the sensing data, the first step is to design a weighted adaptive rule, including environmental dimensions, gas characteristic dimensions, sensor health dimensions, and feature reliability dimensions. Then, an improved DS evidence theory is adopted, defining three core evidence bodies: gas concentration peak value greater than threshold C0, visual identification of leak trace confidence greater than 0.6, and location coordinates located in a high-risk area. A conflict coefficient threshold λ=0.8 is introduced. When the conflict between two evidence bodies is greater than λ, sensor health data arbitration is invoked. When the conflict is ≤λ, the fusion confidence is calculated according to the classic DS synthesis rule. Finally, suspected leak sources are screened based on the fusion confidence.

5. The target localization and path optimization method for a gas inspection robot based on multimodal perception according to claim 4, characterized in that: The specific location of the leakage source is as follows: After identifying the suspected leak source, the dynamic Gaussian plume diffusion model takes the robot's real-time pose, motion direction, and environmental parameters as input, calculates the diffusion coefficient through the optimized diffusion coefficient formula, outputs the predicted gas concentration values ​​at different times and locations, and forms a comparison benchmark with the robot's actual sampling concentration. The leakage rate Q is inferred by combining the size of the leakage point identified by the industrial camera with the fluid dynamics formula. Q is then input into the diffusion model in real time to correct the parameters. At the same time, the actual concentration gradient of three consecutive sampling points of the robot is calculated and compared with the gradient predicted by the model. If the deviation is greater than 20%, the resampling weight of the particle filter is adjusted. The improved particle filtering localization method generates 1000 particles within a 5m×5m range, centered on the suspected leakage source. The initial particle weights are related to the multimodal fusion confidence. In each iteration, the particle positions are updated in conjunction with robot motion data. The particle weights are adjusted according to the deviation between the actual concentration and the predicted concentration. Particles with weights less than 0.05 are removed and new particles are added. After iteration, the top 20% of particles with the highest weights are retained, and their cluster centers are the coordinates of the leakage source.

6. The target localization and path optimization method for a gas inspection robot based on multimodal perception according to claim 5, characterized in that: The initial path planning is as follows: After obtaining the coordinates of the leak source, a four-dimensional priority matrix is ​​established for leak priority classification, which is based on concentration level × distance from core equipment × leak rate × diffusion area growth rate. The weight of each dimension is dynamically adjusted according to the scenario. Priority scores are calculated and classified into top-level, high-level, medium-level, and low-level according to the scores. The weight of the leak source is multiplied by 5 for top-level priority, 3 for high-level, 2 for medium-level, and 1 for low-level to guide the priority of path planning. The improved A-path planning algorithm is designed with a comprehensive cost function to prioritize the coverage of high-priority and high-risk areas. Finally, starting from the robot's current position, the cost of each node is calculated according to the cost function, and the path with the lowest cost is selected as the initial inspection path.

7. The target localization and path optimization method for a gas inspection robot based on multimodal perception according to claim 6, characterized in that: The feedback-based real-time path optimization is as follows: After the initial inspection path planning is completed, the optimized triggering conditions include triggering emergency path insertion and prioritizing the dispatch of the robot to the new leak source area when a new suspected leak source is detected; triggering local path replanning when encountering unpreset obstacles and adopting an arc detour strategy to avoid collisions; and triggering priority iteration and path adjustment when the leak source concentration changes beyond the threshold after inspection. The multi-objective optimization algorithm adopts the NSGA-II non-dominated sorting genetic algorithm, which sets four optimization objectives: shortest total path length, shortest response time of super-level and high-level leakage sources, fastest rate of decrease of leakage source concentration, and lowest path repetition rate. Each iteration generates a Pareto optimal path set, and the robot selects the optimal path according to the real-time environment. In multi-robot collaborative optimization, when a single robot detects more than two high-priority leakage sources, it sends a collaborative request containing the coordinates, concentration, and priority information of the leakage sources to neighboring robots through the 5G gateway, and allocates tasks based on distance, concentration, and robot load.

8. The target localization and path optimization method for a gas inspection robot based on multimodal perception according to claim 7, characterized in that: The verification of the positioning and path optimization effect is as follows: The test scenario design covers three typical scenarios: pipeline area in chemical industrial park, underground pipe gallery, and open-air storage tank area. The scenario parameters are matched with hardware adaptation requirements. The core verification indicators include dynamic positioning accuracy with a leakage source positioning error of ≤±30cm when the robot moves, emergency response time of ≤30s from detection to first inspection of a top-priority leakage source, long-term stability with sensor drift of ≤3%FS and 100% success rate of backup sensor switching without detection interruption, path optimization effect with invalid inspection path ratio of ≤15%, and collaborative efficiency with a total response time of ≤45s when two robots work together to handle three top-priority leakage sources. The parameter iterative optimization establishes an environmental parameter, fusion weight, and model parameter mapping database based on multi-scenario test data. After the robot is powered on, it collects real-time environmental parameters through the environmental perception module, automatically matches the optimal parameters in the database, and does not require manual debugging. After each inspection is completed, new data is stored in the database, and the mapping relationship is updated through the gradient descent algorithm.

Images