A gas inspection robot target positioning and path optimization method based on multi-modal perception

Abstract

The application belongs to the technical field of gas inspection robot positioning and path optimization, and discloses a gas inspection robot target positioning and path optimization method based on multi-modal perception, a four-dimensional weight model of environment, gas characteristics, sensor health degree and feature reliability is constructed, and a weight self-adaptive rule is designed; for example, in a high-humidity or high-wind environment, the weight proportion of gas concentration data will be dynamically reduced, and the weight of visual feature data will be increased at the same time; when the sensor drift reaches the set threshold, not only the weight of the sensor data will be reduced, but also a standby sensor will be called, and an auxiliary correction value is generated by combining the LSTM model trained by historical data; by improving the D-S evidence theory, a conflict coefficient threshold is introduced, when different evidence bodies exist conflict, the sensor health data is called for arbitration, and then the leakage source is screened according to the fusion confidence, which effectively avoids the limitation of fixed weight, and reduces the misjudgment rate under complex environment and sensor abnormal scene.

Description

Technical Field

[0001] This invention belongs to the field of gas inspection robot localization and path optimization technology, specifically a target localization and path optimization method for gas inspection robots based on multimodal perception. Background Technology

[0002] Chemical industrial parks and underground utility tunnels often store and transport flammable, explosive, or toxic gases. Leaks can easily lead to serious accidents such as explosions and poisoning. Manual inspections suffer from high safety risks, limited coverage, and poor real-time data. Therefore, gas inspection robots have become the mainstream solution. Currently, the industry mostly adopts a multimodal technology approach that integrates gas, vision, and positioning modules. However, the following technical problems still exist in practical applications:

[0003] Existing solutions often use preset fixed weights, such as 0.6 for gas concentration and 0.4 for visual features, without considering the impact of dynamic environment on modal reliability: in high humidity environments with humidity exceeding 85%RH, the response delay of electrochemical sensors and the concentration detection error increase, and the fixed weights still forcibly rely on gas data, leading to an increase in the false positive rate of suspected leak sources; moreover, when the sensor experiences zero drift exceeding 5%FS, there is a lack of dynamic weight reduction mechanism, which further amplifies the error.

[0004] While some solutions incorporate the Gaussian Plume diffusion model, key parameters such as leakage rate and diffusion coefficient are mostly statically preset, failing to incorporate real-time motion data such as dynamic acceleration and turning angle collected by the IMU. When the robot accelerates, the sampling points are prone to misalignment with the concentration distribution predicted by the model, leading to increased positioning errors. Furthermore, it cannot distinguish between concentration fluctuations from moving samples and actual concentration changes at the leakage source, affecting positioning accuracy.

[0005] The existing dynamic path planning is only triggered when a new leak is detected or an obstacle is avoided, and does not take the change in the concentration of the leak source after the inspection as feedback: when the leak is mitigated, the inspection priority is not reduced and it is still repeatedly covered; when the leak is aggravated, the secondary inspection cannot be scheduled first, forming a planning-execution-no-feedback breakpoint, resulting in a high proportion of invalid inspection paths and long response delays at core leak points. Summary of the Invention

[0006] The purpose of this invention is to provide a target localization and path optimization method for gas inspection robots based on multimodal perception, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a target localization and path optimization method for a gas inspection robot based on multimodal perception, the specific steps of which are as follows:

[0008] The hardware system includes a gas sensing module, an environmental sensing module, a visual sensing module, a positioning reference module, a data processing and communication module, and a collaborative control module for data acquisition.

[0009] Data preprocessing: Kalman filtering was used to eliminate noise and perform two-factor correction on gas data; dark channel prior algorithm was used to remove fog from visual data; and unscented Kalman filtering was used to fuse the three-source data and adjust the weights on the localization data. Finally, time alignment was performed on the gas and visual data.

[0010] Multimodal feature fusion: A four-dimensional weighted model of environment, gas characteristics, sensor health and feature reliability is established, weight adaptive rules are designed, and an improved DS evidence theory is adopted to define three core evidence bodies to screen suspected leakage sources.

[0011] Leakage source location: Based on the suspected leakage source, the diffusion coefficient is calculated using a dynamic Gaussian plume model, and the parameters are corrected through multi-source calibration. Combined with an improved particle filter, the coordinates of the leakage source are located.

[0012] Initial path planning: A four-dimensional priority matrix is ​​established based on leakage priority division, and an improved A-path planning algorithm is used to design a comprehensive cost function to plan the optimal path;

[0013] Feedback-based real-time path optimization: Three types of triggering scenarios are designed, the NSGA-II non-dominated sorting genetic algorithm is adopted, four optimization objectives are set, the optimal path is selected, and multiple robots collaboratively allocate tasks;

[0014] Verification of localization and path optimization effects: Design three typical scenario tests to verify core indicators including dynamic positioning accuracy and response time, and establish a mapping database of environmental parameters, fusion weights and model parameters.

[0015] Preferably, the hardware system is constructed as follows:

[0016] The gas sensing module adopts a four-array design consisting of a dual-channel electrochemical sensor, an infrared sensor, and a PID photoionization sensor, and is equipped with a sensor health monitoring circuit to collect zero-point voltage, response time, and drift in real time.

[0017] The environmental perception module includes a temperature and humidity sensor and a three-dimensional wind speed and direction sensor, used to collect the surface temperature of the pipeline; the visual perception module is equipped with an industrial camera and an infrared thermal imager, with the lens equipped with an anti-fog coating and an automatic cleaning device, to extract the visual features associated with three types of leaks in real time: pipeline cracks, valve corrosion, and condensation.

[0018] The positioning reference module adopts a three-source fusion design of UWB ultra-wideband positioning, LiDAR SLAM and high-precision IMU. During dynamic mapping, the coordinates of high-risk areas are marked and the robot's real-time pose is output. The data processing and communication module is equipped with edge computing nodes and a 5G industrial gateway. The edge nodes are responsible for localized processing of multimodal data preprocessing, weight calculation and positioning iteration, and only upload the optimized path and leakage source results to the cloud. The collaborative control module adds a multi-robot collaborative communication interface.

[0019] Preferably, the data preprocessing is as follows:

[0020] After completing the multi-source data acquisition, the gas data preprocessing uses Kalman filtering to remove sensor drift noise. Combined with the environmental sensing module data, it is corrected using a two-factor correction formula of temperature, humidity and gas type to compensate for concentration error. At the same time, a sensor health compensation mechanism is introduced. When the sensor drift reaches a threshold of 80%, backup sensor data is called. An auxiliary correction value is generated by an LSTM model trained with historical data.

[0021] In visual data preprocessing, industrial camera images are dehazed using a dark channel prior algorithm, and Canny edge detection is used to extract pipe cracks and valve corrosion areas. Infrared thermal images are segmented using an adaptive temperature threshold to extract high-temperature areas of suspected leaks and to mark the area area and center point coordinates. In localization data preprocessing, unscented Kalman filtering is used to fuse IMU, UWB, and SLAM data, and the weights of each source are dynamically adjusted in conjunction with the robot's motion state. Finally, the gas and visual data are time-aligned based on the timestamp of the localization reference module, and all data are standardized to the [0, 1] interval.

[0022] Preferably, the multimodal feature fusion is specifically as follows:

[0023] After completing the preprocessing of the sensing data, the first step is to design a weighted adaptive rule, including environmental dimensions, gas characteristic dimensions, sensor health dimensions, and feature reliability dimensions. Then, an improved DS evidence theory is adopted, defining three core evidence bodies: gas concentration peak value greater than threshold C0, visual identification of leak trace confidence greater than 0.6, and location coordinates located in a high-risk area. A conflict coefficient threshold λ=0.8 is introduced. When the conflict between two evidence bodies is greater than λ, sensor health data arbitration is invoked. When the conflict is ≤λ, the fusion confidence is calculated according to the classic DS synthesis rule. Finally, suspected leak sources are screened based on the fusion confidence.

[0024] Preferably, the location of the leakage source is as follows:

[0025] After identifying the suspected leak source, the dynamic Gaussian plume diffusion model takes the robot's real-time pose, motion direction, and environmental parameters as input, calculates the diffusion coefficient through the optimized diffusion coefficient formula, outputs the predicted gas concentration values ​​at different times and locations, and forms a comparison benchmark with the robot's actual sampling concentration.

[0026] The leakage rate Q is inferred by combining the size of the leakage point identified by the industrial camera with the fluid dynamics formula. Q is then input into the diffusion model in real time to correct the parameters. At the same time, the actual concentration gradient of three consecutive sampling points of the robot is calculated and compared with the gradient predicted by the model. If the deviation is greater than 20%, the resampling weight of the particle filter is adjusted.

[0027] The improved particle filtering localization method generates 1000 particles within a 5m×5m range, centered on the suspected leakage source. The initial particle weights are related to the multimodal fusion confidence. In each iteration, the particle positions are updated in conjunction with robot motion data. The particle weights are adjusted according to the deviation between the actual concentration and the predicted concentration. Particles with weights less than 0.05 are removed and new particles are added. After iteration, the top 20% of particles with the highest weights are retained, and their cluster centers are the coordinates of the leakage source.

[0028] Preferably, the initial path planning is as follows:

[0029] After obtaining the coordinates of the leak source, a four-dimensional priority matrix is ​​established for leak priority classification, which is based on concentration level × distance from core equipment × leak rate × diffusion area growth rate. The weight of each dimension is dynamically adjusted according to the scenario. Priority scores are calculated and classified into top-level, high-level, medium-level, and low-level according to the scores. The weight of the leak source is multiplied by 5 for top-level priority, 3 for high-level, 2 for medium-level, and 1 for low-level to guide the priority of path planning.

[0030] The improved A-path planning algorithm is designed with a comprehensive cost function to prioritize the coverage of high-priority and high-risk areas. Finally, starting from the robot's current position, the cost of each node is calculated according to the cost function, and the path with the lowest cost is selected as the initial inspection path.

[0031] Preferably, the feedback-based real-time path optimization is as follows:

[0032] After the initial inspection path planning is completed, the optimized triggering conditions include triggering emergency path insertion and prioritizing the dispatch of the robot to the new leak source area when a new suspected leak source is detected; triggering local path replanning when encountering unpreset obstacles and adopting an arc detour strategy to avoid collisions; and triggering priority iteration and path adjustment when the leak source concentration changes beyond the threshold after inspection.

[0033] The multi-objective optimization algorithm adopts the NSGA-II non-dominated sorting genetic algorithm, which sets four optimization objectives: shortest total path length, shortest response time of super-level and high-level leakage sources, fastest rate of decrease of leakage source concentration, and lowest path repetition rate. Each iteration generates a Pareto optimal path set, and the robot selects the optimal path according to the real-time environment.

[0034] In multi-robot collaborative optimization, when a single robot detects more than two high-priority leakage sources, it sends a collaborative request containing the coordinates, concentration, and priority information of the leakage sources to neighboring robots through the 5G gateway, and allocates tasks based on distance, concentration, and robot load.

[0035] Preferably, the verification of the positioning and path optimization effect is as follows:

[0036] The test scenario design covers three typical scenarios: pipeline area in chemical industrial park, underground pipe gallery, and open-air storage tank area. The scenario parameters are matched with hardware adaptation requirements.

[0037] The core verification indicators include dynamic positioning accuracy with a leakage source positioning error of ≤±30cm when the robot moves, emergency response time of ≤30s from detection to first inspection of a top-priority leakage source, long-term stability with sensor drift of ≤3%FS and 100% success rate of backup sensor switching without detection interruption, path optimization effect with invalid inspection path ratio of ≤15%, and collaborative efficiency with a total response time of ≤45s when two robots work together to handle three top-priority leakage sources.

[0038] The parameter iterative optimization establishes an environmental parameter, fusion weight, and model parameter mapping database based on multi-scenario test data. After the robot is powered on, it collects real-time environmental parameters through the environmental perception module, automatically matches the optimal parameters in the database, and does not require manual debugging. After each inspection is completed, new data is stored in the database, and the mapping relationship is updated through the gradient descent algorithm.

[0039] The beneficial effects of this invention are as follows:

[0040] 1. This invention constructs a four-dimensional weighted model encompassing environment, gas characteristics, sensor health, and feature reliability, and designs adaptive weighting rules. For example, in high humidity or high wind speed environments, the weight of gas concentration data is dynamically reduced while the weight of visual feature data is increased. When sensor drift reaches a set threshold, not only is the weight of that sensor data reduced, but a backup sensor is also invoked, and an auxiliary correction value is generated using an LSTM model trained with historical data. Furthermore, by improving the DS evidence theory and introducing a conflict coefficient threshold, when different pieces of evidence conflict, sensor health data is used for arbitration, and the leakage source is then screened based on the fusion confidence level. This design can dynamically match the reliability of each modality of data, effectively avoiding the limitations of fixed weights and significantly reducing the misjudgment rate in complex environments and sensor anomaly scenarios.

[0041] 2. This invention employs a dynamic Gaussian plume model, using real-time robot pose, direction of motion, speed, and turning angle as input. An optimized formula is used to calculate the dynamic diffusion coefficient, preventing misalignment between static parameters and the robot's sampling process. Simultaneously, the leakage rate is inferred from the size of the leak point identified by the industrial camera. Furthermore, the actual concentration gradient of multiple consecutive sampling points is compared with the model's predicted gradient. If the deviation exceeds a set range, the resampling weights of the particle filter are adjusted promptly to achieve multi-source data calibration. The improved particle filter generates particles centered on suspected leak sources. The initial particle weights are related to the multimodal fusion confidence level. During iteration, low-weight particles are removed and new particles are added. Finally, the cluster center of high-weight particles is used as the leak source coordinates. This design dynamically corrects model parameters, eliminates interference from robot motion, accurately distinguishes between sampling concentration fluctuations and actual leakage changes, and keeps positioning errors within a reasonable range.

[0042] 3. This invention constructs a feedback-based real-time path optimization mechanism and designs three trigger scenarios: when a new suspected leak source is detected, an emergency path is inserted and the robot is prioritized to proceed; when encountering unpreset obstacles, local path replanning is triggered and a circular detour strategy is adopted to avoid collisions; when the leak source concentration change exceeds a threshold after inspection, leak priority iteration and path adjustment are triggered; using the NSGA-II non-dominated sorting genetic algorithm, optimization objectives such as total path length, response time of top-priority and high-priority leak sources, leak source concentration decrease rate, and path repetition rate are set to generate a Pareto optimal path set for the robot to choose from; when a single robot detects multiple top-priority leak sources, a collaborative request is sent to neighboring robots through a 5G gateway, and tasks are allocated based on distance, concentration, and robot load; this design effectively reduces invalid inspection paths, significantly shortens the response time of top-priority leak sources, significantly improves the efficiency of multi-robot collaborative leak handling, and further ensures the inspection safety of high-risk scenarios. Attached Figure Description

[0043] Figure 1 This is a flowchart of the target localization and path optimization method for a gas inspection robot based on multimodal perception, as described in this invention.

[0044] Figure 2 This is a flowchart of the adaptive fusion and suspected leakage source screening process of the present invention;

[0045] Figure 3 This is a flowchart of the leakage source location and path optimization process of the present invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] like Figures 1 to 3 As shown in the figure, this embodiment of the invention provides a target localization and path optimization method for a gas inspection robot based on multimodal perception. The specific steps of the method are as follows:

[0048] The hardware system includes a gas sensing module, an environmental sensing module, a visual sensing module, a positioning reference module, a data processing and communication module, and a collaborative control module for data acquisition.

[0049] Data preprocessing: Kalman filtering was used to eliminate noise and perform two-factor correction on gas data; dark channel prior algorithm was used to remove fog from visual data; and unscented Kalman filtering was used to fuse the three-source data and adjust the weights on the localization data. Finally, time alignment was performed on the gas and visual data.

[0050] Multimodal feature fusion: A four-dimensional weighted model of environment, gas characteristics, sensor health and feature reliability is established, weight adaptive rules are designed, and an improved DS evidence theory is adopted to define three core evidence bodies to screen suspected leakage sources.

[0051] Leakage source location: Based on the suspected leakage source, the diffusion coefficient is calculated using a dynamic Gaussian plume model, and the parameters are corrected through multi-source calibration. Combined with an improved particle filter, the coordinates of the leakage source are located.

[0052] Initial path planning: A four-dimensional priority matrix is ​​established based on leakage priority division, and an improved A-path planning algorithm is used to design a comprehensive cost function to plan the optimal path;

[0053] Feedback-based real-time path optimization: Three types of triggering scenarios are designed, the NSGA-II non-dominated sorting genetic algorithm is adopted, four optimization objectives are set, the optimal path is selected, and multiple robots collaboratively allocate tasks;

[0054] Verification of localization and path optimization effects: Design three typical scenario tests to verify core indicators including dynamic positioning accuracy and response time, and establish a mapping database of environmental parameters, fusion weights and model parameters.

[0055] The specific details of building the hardware system are as follows:

[0056] To meet the core needs of gas inspection in high-risk scenarios such as chemical industrial parks and underground pipe corridors, a hardware system combining redundant sensing, edge computing, and collaborative communication is constructed to provide basic support for subsequent data collection and processing.

[0057] The gas sensing module adopts a four-array design consisting of a dual-channel electrochemical sensor, an infrared sensor, and a PID photoionization sensor. The sampling frequency is adjustable from 1Hz to 5Hz. The accompanying sensor health monitoring circuit collects zero-point voltage, response time, and drift in real time.

[0058] A dual-channel electrochemical sensor detects methane (0~100% LEL) and hydrogen sulfide (0~100 ppm), an infrared sensor detects carbon dioxide (0~5% vol), and a PID photoionization sensor detects VOCs (0~2000 ppm).

[0059] The environmental perception module includes a high-precision temperature and humidity sensor and a three-dimensional wind speed and direction sensor, which are used to collect the surface temperature of the pipeline and provide environmental parameters for subsequent gas concentration correction and diffusion model; the visual perception module is equipped with an industrial camera and a high-definition infrared thermal imager, and the lens is equipped with an anti-fog coating and an automatic cleaning device to adapt to the high humidity and dusty environment of the underground pipe gallery, and extracts the three types of leakage-related visual features in real time: pipe cracks, valve corrosion and condensation water stains.

[0060] The positioning reference module adopts a three-source fusion design of UWB ultra-wideband positioning, LiDAR SLAM, and high-precision IMU. During dynamic mapping, it marks the coordinates of high-risk areas such as pipe joints, valves, and flanges, and outputs the robot's real-time pose. The data processing and communication module is equipped with edge computing nodes and a 5G industrial gateway. The edge nodes are responsible for localized processing of multimodal data preprocessing, weight calculation, and positioning iteration. Only the optimized path and leakage source results are uploaded to the cloud, thereby avoiding the lack of real-time performance caused by cloud reliance. The collaborative control module adds a multi-robot collaborative communication interface. When a single robot detects a multi-source high-risk leakage, it can send task allocation instructions to neighboring robots through this interface to achieve cross-regional collaborative inspection.

[0061] Temperature and humidity sensor measurement range -20~60℃ (accuracy ±0.5℃), 0~100%RH (accuracy ±3%RH); 3D wind speed and direction sensor measurement range 0~15m / s (accuracy ±0.1m / s); UWB positioning accuracy ±15cm, IMU angular velocity accuracy ±0.1° / h;

[0062] The data preprocessing is specifically as follows:

[0063] After completing the multi-source data acquisition, the gas data preprocessing uses Kalman filtering to remove sensor drift noise. Combined with the environmental sensing module data, it is corrected using a two-factor correction formula of temperature, humidity and gas type to compensate for concentration error. At the same time, a sensor health compensation mechanism is introduced. When the sensor drift reaches a threshold of 80%, backup sensor data is called. An auxiliary correction value is generated by an LSTM model trained with historical data to ensure the continuity of concentration data.

[0064] The historical data for the LSTM model comes from over 1000 hours of sensor data accumulated by the robot under the same working conditions, including measured concentration values ​​under different temperatures, humidity levels, and drift amounts. The input variables are sensor drift amount, real-time temperature, and real-time humidity, and the output variable is the gas concentration correction value. The model structure has 3 hidden layers, the optimizer is Adam, and the loss function is mean squared error (MSE).

[0065] In visual data preprocessing, industrial camera images are dehazed using a dark channel prior algorithm, and Canny edge detection is used to extract pipe cracks and valve corrosion areas. Infrared thermal images are segmented using an adaptive temperature threshold to extract high-temperature areas of suspected leaks and to mark the area area and center point coordinates. In localization data preprocessing, unscented Kalman filtering is used to achieve the fusion of IMU, UWB and SLAM three-source data, and the weights of each source are dynamically adjusted in combination with the robot's motion state.

[0066] During acceleration, the weights are: IMU 0.6, UWB 0.2, and SLAM 0.2; during constant speed, the weights are: UWB 0.5, SLAM 0.4, and IMU 0.1; and during turning, the weights are: SLAM 0.5, IMU 0.3, and UWB 0.2, to ensure dynamic pose accuracy.

[0067] Finally, the gas and visual data are time-aligned based on the timestamp of the positioning reference module, and all data are standardized to the [0, 1] interval.

[0068] Two-factor correction formula for gas concentration:

[0069] ;

[0070] In the formula: This indicates the gas concentration after two-factor correction, in units of ppm or vol%, depending on the gas type.

[0071] This indicates that the raw concentration data was directly collected by the gas sensor without noise and environmental interference being eliminated.

[0072] This represents the temperature correction coefficient, determined by the sensor calibration experiment, used to compensate for the effect of temperature changes on concentration detection.

[0073] The current temperature is displayed in °C and is collected in real time by the temperature and humidity sensor of the environmental sensing module. It needs to be aligned with the gas collection time.

[0074] This indicates the standard ambient temperature at which the sensor is calibrated, in °C, typically 25°C, and is a fixed value.

[0075] This represents the humidity correction coefficient, determined by the sensor calibration experiment, used to compensate for the impact of humidity changes on concentration detection.

[0076] This indicates the current relative humidity, expressed in %RH. It is collected in real time by the temperature and humidity sensors in the environmental sensing module and needs to be aligned with the gas collection time.

[0077] This indicates the standard ambient relative humidity for sensor calibration, expressed in %RH, typically 50%RH, which is a fixed value.

[0078] This represents the gas type correction coefficient, a preset correction coefficient for different gases (such as methane, VOCs, and toxic gases), obtained by fitting experimental data.

[0079] This parameter represents the gas type identifier, assigned values ​​according to the gas category, such as methane=1, VOCs=2, toxic gases=3, used to match the corresponding... value.

[0080] The multimodal feature fusion is specifically as follows:

[0081] After completing the preprocessing of the perception data, the first step is to design adaptive weight rules. In the environmental dimension, the weight of gas concentration is reduced by 20%-40% and the weight of visual features is increased by 30%-50% in high humidity or high wind speed environments, while the weight of gas concentration is increased by 10%-20% and the weight of visual features is reduced by 5%-15% in low humidity and low wind speed environments.

[0082] Gas characteristics dimension: When the concentration gradient of toxic gas is >50ppm / m, the gas weight is increased by an additional 15%-20%; when the concentration of combustible gas is close to 1 / 3 of the lower explosive limit, the weight lock of the "gas and vision" dual-modality is triggered; when the VOCs concentration is >100ppm, the weight of PID sensor data is ≥0.6 and the weight of other gas sensors is reduced.

[0083] Sensor health dimension: When the gas sensor drift exceeds the threshold, the weight drops to below 0.2; when it reaches 80%~100% of the threshold, the weight drops to 0.3~0.2; when it is <80% of the threshold, it is linearly adjusted according to "1 - drift amount / threshold".

[0084] Feature reliability dimension: When the confidence level of visual recognition of cracks / corrosion is >0.8, the weight is increased by 10%-15%; when it is 0.6~0.8, the baseline value is maintained; when it is <0.6, the weight is decreased by 20%-30%, and the collaborative weight of gas concentration and positioning information is increased simultaneously.

[0085] Subsequently, an improved DS evidence theory was adopted, defining three core evidence bodies: gas concentration peak value greater than threshold C0, visual identification of leak trace confidence value greater than 0.6, and location coordinates located in a high-risk area. A conflict coefficient threshold λ=0.8 was introduced. When the conflict between two evidence bodies is greater than λ, sensor health data arbitration is called. When the conflict is ≤λ, the fusion confidence value is calculated according to the classic DS synthesis rule. Finally, suspected leak sources are screened based on the fusion confidence value.

[0086] A confidence level > 0.85 indicates a high-confidence leak source that directly enters the localization process; a confidence level between 0.6 and 0.85 indicates a leak source to be verified, triggering secondary sampling verification; and a confidence level < 0.6 indicates a leak source that is excluded to avoid misjudgment.

[0087] The peak gas concentration threshold C0 is set according to the type of gas being detected. For methane, C0 = 5000 ppm (corresponding to 10% of its lower explosive limit of 5% vol), and for VOCs, C0 = 200 ppm (refer to the safety limits in GB27632-2011).

[0088] The leakage rate is calculated using the orifice outflow formula Q=Cd×A×√(2ΔP / ρ), where Cd is the flow coefficient (taken as 0.6~0.8, determined according to the shape of the leakage orifice), A is the area of ​​the leakage orifice (calculated from the crack size identified by the industrial camera), ΔP is the pressure difference between the inside and outside of the pipeline (obtained from the industrial park's process data), and ρ is the gas density (the standard value is looked up by gas type).

[0089] The specific location of the leakage source is as follows:

[0090] After identifying the suspected leak source, the dynamic Gaussian plume diffusion model takes the robot's real-time pose, motion direction, and environmental parameters as input, calculates the diffusion coefficient through the optimized diffusion coefficient formula, outputs the predicted gas concentration values ​​at different times and locations, and forms a comparison benchmark with the robot's actual sampling concentration.

[0091] In the multi-source data calibration process, the leakage rate Q is inferred by combining the size of the leakage point identified by the industrial camera with the fluid dynamics formula. Q is then input into the diffusion model correction parameters in real time. At the same time, the actual concentration gradient of three consecutive sampling points of the robot is calculated and compared with the model prediction gradient. If the deviation is greater than 20%, the resampling weight of the particle filter is adjusted to ensure that the model matches the actual concentration distribution.

[0092] The improved particle filtering localization method generates 1000 particles within a 5m×5m range, centered on the suspected leak source. The initial weight of the particles is related to the confidence level of multimodal fusion. In each iteration, the particle position is updated in combination with robot motion data. The particle weight is adjusted according to the deviation between the actual concentration and the predicted concentration. Particles with a weight less than 0.05 are removed and new particles are added. After iteration, the top 20% of the particles with the highest weights are retained. Their cluster center is the coordinate of the leak source, and the localization error is controlled.

[0093] Formula for calculating dynamic diffusion coefficient:

[0094] ;

[0095] In the formula: This represents the dynamic gas diffusion coefficient at time t, with units of . The concentration of gas at different locations varies over time and is used in Gaussian plume models to predict the gas concentration at different locations.

[0096] Represents the static gas diffusion coefficient, with units of . The diffusion coefficient refers to the diffusion coefficient under standard conditions, which is determined by the physical properties of the gas (such as small molecule gases in the air). ), which is a fixed value;

[0097] This represents the velocity influence coefficient, an experimental fitting parameter used to quantify the interference of robot motion speed on gas diffusion, typically ranging from 0.02 to 0.05.

[0098] This represents the robot's real-time velocity at time t, in units of . It is obtained from motion state data collected by the positioning reference module (IMU or SLAM);

[0099] The angle between the robot's motion direction and the dominant gas diffusion direction at time t is expressed in rad. It is calculated by combining the three-dimensional wind speed and direction sensor of the environmental perception module to determine the dominant diffusion direction with the robot's motion direction.

[0100] This represents the steering angle influence coefficient, an experimental fitting parameter used to quantify the interference of robot steering on gas diffusion, typically ranging from 0.01 to 0.03.

[0101] This represents the rate of change of the robot's steering angle at time t, in units of . The data is collected by the IMU of the positioning reference module, reflecting the severity of the robot's turning, and is used to compensate for the offset of the sampling points caused by the turning.

[0102] The initial path planning is as follows:

[0103] After obtaining the coordinates of the leak source, a four-dimensional priority matrix is ​​established for leak priority classification, which is based on concentration level × distance from core equipment × leak rate × diffusion area and growth rate. The weight of each dimension is dynamically adjusted according to the scenario.

[0104] Concentration level > 1 / 2 of the lower explosive limit: 5 points; 1 / 3 to 1 / 2: 4 points; Distance from core equipment < 5m: 5 points; 5 to 10m: 4 points; Leakage rate > 0.1m³ / h 3 / s scores 5 points, 0.05~0.1m 3 / s scores 4 points, diffusion area growth rate > 1m 2 5 points for a speed of 0.5~1m 2 / min earns 4 points;

[0105] Calculate priority scores and classify them into top-priority, high-priority, medium-priority, and low-priority categories based on the scores. The weight of the leakage source is multiplied by 5 for top-priority, 3 for high-priority, 2 for medium-priority, and 1 for low-priority to guide path planning priority.

[0106] The improved A-path planning algorithm is designed with a comprehensive cost function to prioritize the coverage of high-priority and high-risk areas. Finally, starting from the robot's current position, the cost of each node is calculated according to the cost function, and the path with the lowest cost is selected as the initial inspection path to meet real-time requirements.

[0107] Path planning comprehensive cost function:

[0108] ;

[0109] =1;

[0110] In the formula: This represents the total cost of a path node. The lower the cost, the better the path. It is used in the A* algorithm to select the optimal path node.

[0111] The path length weight can be dynamically adjusted, such as in open areas. Complex obstruction area This is used to balance path length and inspection efficiency;

[0112] This represents the path length from the current node to the target node, in meters, calculated based on the prior map of the inspection area and the robot's real-time pose.

[0113] This indicates the priority weight of the leak, which can be dynamically adjusted, such as a critical leak area. Low-level leakage area Ensure that the path prioritizes coverage of high-risk leakage sources;

[0114] The score indicates the priority of the leak source, based on the concentration level. Distance from core equipment Leakage rate The diffusion area growth rate is calculated using a four-dimensional matrix, with scores ranging from 1 to 20: Special Grade (z16), Advanced Grade (z2-15), Intermediate Grade (z8-11), and Low Grade (z10-20). 7;

[0115] This indicates the weight of obstacle avoidance costs, which can be dynamically adjusted, such as in areas with dense obstacles. =0.3, unobstructed area =0.1, used to avoid collisions between the path and obstacles;

[0116] The cost of obstacle avoidance is calculated based on LiDAR SLAM mapping data; the closer the obstacle and the larger its size, the higher the cost. The higher the value, the higher the range (1-5).

[0117] This represents the risk-cost weighting of diffusion, which can be dynamically adjusted, such as in areas with rapid diffusion. =0.3, slow diffusion region =0.1, used to prioritize addressing leak sources that pose an increased risk;

[0118] The cost of diffusion risk is calculated based on the growth rate of the gas diffusion area; if the growth rate is greater than 1... At the minimum, R=5, 0.5~ When R=3, At min time, R=1.

[0119] Leakage priority score calculation formula:

[0120] ;

[0121] In the formula: This represents the total score indicating the priority of the leak source. The score determines the priority level of the leak source: Top priority ≥ 16, High priority 12~15, Medium priority 8~11, Low priority 7;

[0122] The values ​​of i are 1, 2, 3, and 4, representing the four core risk dimensions of concentration, distance, rate, and diffusion, respectively.

[0123] Indicates the first The base scores for each of the four dimensions are as follows:

[0124] 1. Concentration level 5 points for 1 / 2 of the lower limit of explosion, 4 points for 1 / 3 to 1 / 2, 3 points for 1 / 4 to 1 / 3, and 2 points for <1 / 4;

[0125] 2. Distance from core equipment ( 5 points for m, 4 points for 510m, 3 points for 1015m, and 2 points for >15m;

[0126] 3. Leakage rate ( 5 points, 0.05~0.1 4 points, 0.01~0.05 3 points, <0.01 2 points

[0127] 4. Growth rate of diffusion area ( (5 points for >1 m² / min, 4 points for 0.5-1 m² / min, 3 points for 0.1-0.5 m² / min, and 2 points for <0.1 m² / min)

[0128] This represents the weight of the i-th dimension, which can be dynamically adjusted, such as the core area of ​​a chemical industrial park. =0.4, open-air tank area =0.4), and satisfies =1.

[0129] The feedback-based real-time path optimization is specifically as follows:

[0130] After the initial inspection path planning is completed, the optimized triggering conditions include triggering emergency path insertion and prioritizing the dispatch of the robot to the new leak source area when a new suspected leak source is detected; triggering local path replanning when encountering unpreset obstacles and adopting an arc detour strategy to avoid collisions; and triggering priority iteration and path adjustment when the leak source concentration changes beyond the threshold after inspection.

[0131] The radius of the arc = robot body width (assuming 0.5m) + 0.3m safety distance, that is, the radius ≥ 0.8m, to ensure that the robot does not collide with obstacles when going around them;

[0132] Robot load = Sum of assigned leak source priority scores (Special Grade × 5 + High Grade × 3 + Medium Grade × 2 + Low Grade × 1). During collaborative allocation, new leak sources are preferentially assigned to robots with load < 10 points and the closest proximity.

[0133] If the leakage worsens (i.e., the concentration increases by more than 20% / 5min), the priority is increased by 2 levels and a second inspection is conducted within 10min, and the path priority is increased to the highest level. If the leakage is mitigated (i.e., the concentration decreases by more than 30% / 10min), the priority is reduced by 1 level and the inspection interval is extended to 1.5 times the original interval, and the path is delayed. If there is a suspected misjudgment (i.e., the concentration does not fluctuate after 3 rounds of inspections), it is marked as "to be verified" and visual re-judgment is combined. If no leakage is found after re-judgment, the path is removed. If the leakage is stable (i.e., the concentration fluctuation is <10% / 10min), the original level is maintained and inspections are conducted at the original interval.

[0134] The multi-objective optimization algorithm adopts the NSGA-II non-dominated sorting genetic algorithm, which sets four optimization objectives: shortest total path length, shortest response time of super-grade and high-grade leakage sources, fastest rate of decrease of leakage source concentration, and lowest path repetition rate. The algorithm parameters are set as follows: population size 50, number of iterations 30, crossover probability 0.8, and mutation probability 0.1. Each iteration generates a Pareto optimal path set, and the robot selects the optimal path according to the real-time environment.

[0135] In multi-robot collaborative optimization, when a single robot detects more than two high-priority leakage sources, it sends a collaborative request containing the coordinates, concentration, and priority information of the leakage sources to neighboring robots through the 5G gateway. Based on distance, concentration, and robot load, tasks are allocated to ensure synchronous response to multiple leakage sources and improve inspection efficiency.

[0136] The verification of the positioning and path optimization effect is as follows:

[0137] The test scenario design covers three typical scenarios: pipeline area in chemical industrial park (3 preset leak sources, ambient temperature 28℃, humidity 65%, wind speed 2m / s), underground pipe gallery (2 preset dynamic leak sources, ambient temperature 22℃, humidity 90%, low wind speed, high dust), and open-air storage tank area (2 preset leak sources, ambient temperature 35℃, humidity 40%, high wind speed, strong sunlight). The scenario parameters are matched with the hardware adaptation requirements.

[0138] The core verification indicators include dynamic positioning accuracy with a leakage source positioning error of ≤±30cm when the robot moves, emergency response time of ≤30s from detection to first inspection of a top-priority leakage source, long-term stability with sensor drift of ≤3%FS and 100% success rate of backup sensor switching without detection interruption, path optimization effect with invalid inspection path ratio of ≤15%, and collaborative efficiency with a total response time of ≤45s when two robots work together to handle three top-priority leakage sources.

[0139] The parameter iterative optimization establishes an environmental parameter, fusion weight, and model parameter mapping database based on multi-scenario test data. After the robot is powered on, it collects real-time environmental parameters through the environmental perception module, automatically matches the optimal parameters in the database, and does not require manual debugging. After each inspection, new data (environmental parameters, optimized parameters, and verification indicators) are stored in the database, and the mapping relationship is updated through the gradient descent algorithm to ensure performance stability under different scenarios.

[0140] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0141] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A target localization and path optimization method for a gas inspection robot based on multimodal perception, characterized in that: The specific steps of this method are as follows: The hardware system includes a gas sensing module, an environmental sensing module, a visual sensing module, a positioning reference module, a data processing and communication module, and a collaborative control module for data acquisition. Data preprocessing: Kalman filtering is used to eliminate sensor drift noise for gas data, and a two-factor correction formula based on temperature, humidity, and gas type is used to compensate for concentration errors; visual data is defogging using a dark channel prior algorithm, and then visual features associated with three types of leaks—pipe cracks, valve corrosion, and condensate stains—are extracted; localization data is fused using unscented Kalman filtering, integrating three types of source data: UWB ultra-wideband localization, LiDAR SLAM, and high-precision IMU, with the weights of the three types of source data dynamically adjusted based on the robot's motion state; finally, the gas and visual data are time-aligned using the timestamp of the localization reference module, and all data are standardized to the 0-1 range. Multimodal feature fusion: A four-dimensional weighted model is established, encompassing environment, gas characteristics, sensor health, and feature reliability. An adaptive weighting rule is designed to dynamically lower the weight of gas data and increase the weight of visual data under high humidity or high wind speed conditions. When sensor drift reaches a set threshold, the weight of the corresponding sensor data is lowered, and a backup sensor is invoked. An improved DS evidence theory is adopted, defining three core evidence bodies: peak gas concentration greater than threshold C0, visual leakage trace confidence greater than 0.6, and location coordinates in a high-risk area. A conflict coefficient threshold λ=0.8 is introduced. When the conflict between two evidence bodies exceeds λ, sensor health data arbitration is invoked; when the conflict is ≤λ, the multimodal fusion confidence is calculated according to the classic DS synthesis rule. Suspected leakage sources are then screened based on the multimodal fusion confidence. Leakage source localization: Based on the suspected leakage source, a dynamic Gaussian plume diffusion model is used to calculate the dynamic diffusion coefficient with the robot's real-time pose, motion direction, and environmental parameters as inputs, and outputs the predicted gas concentration values ​​at different times and locations; the leakage rate Q is inversely calculated by combining the size of the leakage point identified by the industrial camera with fluid dynamics formulas, and Q is input into the dynamic Gaussian plume diffusion model in real time to correct the model parameters; the particle filter resampling weight is adjusted by combining the deviation between the actual sampling concentration gradient of continuous sampling points and the predicted gas concentration gradient; the improved particle filter is used to locate the leakage source coordinates. The improved particle filter generates particles centered on the suspected leakage source, and the initial particle weight is related to the multimodal fusion confidence. After iteration, the top 20% of the particles with the highest weights are retained, and their cluster centers are the leakage source coordinates; Initial path planning: Leakage priority classification establishes a four-dimensional priority matrix based on concentration level, distance from core equipment, leakage rate, and diffusion area growth rate. The weights of each dimension are dynamically adjusted according to the application scenarios of pipeline areas, underground pipe corridors, and open-air storage tank areas in chemical industrial parks. Priority scores are calculated and classified into special, high, medium, and low levels according to the scores, and corresponding grade weights are set to guide the priority of path planning. An improved A-path planning algorithm is used to design a comprehensive cost function and plan the optimal path; Feedback-based real-time path optimization: Three types of path optimization trigger scenarios are designed, including emergency path insertion when a new suspected leak source is detected, local path replanning when encountering unpreset obstacles, and priority iteration and path adjustment when the leak source concentration change exceeds a threshold after inspection. The NSGA-II non-dominated sorting genetic algorithm is used to set four optimization objectives: shortest total path length, shortest response time for top-priority and high-priority leak sources, fastest rate of decrease in leak source concentration, and lowest path repetition rate. The Pareto optimal path set is generated and the optimal path is selected. When a single robot detects more than two top-priority leak sources, a cooperation request is sent to neighboring robots, and tasks are allocated based on distance, concentration, and robot load. Verification of Positioning and Path Optimization Effects: Three typical test scenarios were designed: pipeline area in a chemical industrial park, underground pipe gallery, and open-air storage tank area. Core indicators, including dynamic positioning accuracy and response time of high-risk and high-risk leakage sources, were verified. A mapping database was established for environmental parameters, multimodal feature fusion weights, dynamic Gaussian Plume diffusion model parameters, and four-dimensional weight model parameters. The multimodal feature fusion weights are dynamic weighting coefficients for fusing three types of evidence: peak gas concentration, visually identified leakage traces, and positioning coordinates located in high-risk areas. These weights are adaptively calculated by a four-dimensional weight model that considers environmental factors, gas characteristics, sensor health, and feature reliability.

2. The target localization and path optimization method for gas inspection robots based on multimodal perception according to claim 1, characterized in that: The specific hardware system setup is as follows: The gas sensing module adopts a four-array design consisting of a dual-channel electrochemical sensor, an infrared sensor, and a PID photoionization sensor, and is equipped with a sensor health monitoring circuit to collect zero-point voltage, sensor step response time, and drift in real time. The environmental perception module includes a temperature and humidity sensor and a three-dimensional wind speed and direction sensor, used to collect the surface temperature of the pipeline; the visual perception module is equipped with an industrial camera and an infrared thermal imager, with the lens equipped with an anti-fog coating and an automatic cleaning device, to extract the visual features associated with three types of leaks in real time: pipeline cracks, valve corrosion, and condensation. The positioning reference module adopts a three-source fusion design of UWB ultra-wideband positioning, LiDAR SLAM and high-precision IMU. During dynamic mapping, it marks the coordinates of high-risk areas such as pipe joints, valves and flanges, and outputs the robot's real-time pose. The data processing and communication module is equipped with edge computing nodes and a 5G industrial gateway. The edge nodes are responsible for localized data preprocessing, weight calculation and positioning iteration, and only upload the optimized path and leakage source results to the cloud. The collaborative control module adds a multi-robot collaborative communication interface.

3. The target localization and path optimization method for gas inspection robots based on multimodal perception according to claim 2, characterized in that: The data preprocessing is as follows: After completing the multi-source data acquisition, the gas data preprocessing uses Kalman filtering to remove sensor drift noise. Combined with the environmental sensing module data, it is corrected using a two-factor correction formula of temperature, humidity and gas type to compensate for concentration error. At the same time, a sensor health compensation mechanism is introduced. When the sensor drift reaches a threshold of 80%, backup sensor data is called. An auxiliary correction value is generated by an LSTM model trained with historical data. In visual data preprocessing, industrial camera images are dehazed using a dark channel prior algorithm, and Canny edge detection is used to extract pipe cracks and valve corrosion areas. Infrared thermal images are segmented using adaptive temperature thresholding to extract high-temperature areas of suspected leaks, and the area and center point coordinates are marked. In localization data preprocessing, unscented Kalman filtering is used to fuse IMU, UWB, and SLAM data, and the weights of each source are dynamically adjusted in conjunction with the robot's motion state. Finally, the gas and visual data are time-aligned based on the timestamp of the localization reference module, and all data are standardized to the 0 to 1 range.

4. The target localization and path optimization method for gas inspection robots based on multimodal perception according to claim 3, characterized in that: The specific location of the leakage source is as follows: After identifying the suspected leak source, the dynamic Gaussian plume diffusion model takes the robot's real-time pose, motion direction, and environmental parameters as input, calculates the diffusion coefficient through the optimized diffusion coefficient formula, outputs the predicted gas concentration values ​​at different times and locations, and forms a comparison benchmark with the robot's actual sampling concentration. The leakage rate Q is inferred by combining the size of the leakage point identified by the industrial camera with the fluid dynamics formula. Q is then input into the dynamic Gaussian plume diffusion model in real time to correct the model parameters. At the same time, the actual sampling concentration gradient of three consecutive sampling points of the robot is calculated and compared with the predicted gas concentration gradient. If the deviation is greater than 20%, the particle filter resampling weight is adjusted. The improved particle filtering localization method generates 1000 particles within a 5m×5m range, centered on the suspected leak source. The initial particle weights are related to the multimodal fusion confidence. In each iteration, the particle positions are updated in conjunction with robot motion data. The particle weights are adjusted according to the deviation between the actual sampled concentration and the predicted gas concentration. Particles with weights less than 0.05 are removed and new particles are added. After iteration, the top 20% of particles by weight are retained, and their cluster centers are the coordinates of the leak source.

5. The target localization and path optimization method for a gas inspection robot based on multimodal perception according to claim 4, characterized in that: The initial path planning is as follows: After obtaining the coordinates of the leak source, a four-dimensional priority matrix is ​​established for leak priority classification based on concentration level, distance from core equipment, leak rate, and diffusion area growth rate. The weight of each dimension is dynamically adjusted according to the application scenarios of pipeline area, underground pipe gallery, and open-air storage tank area in chemical industrial park. Priority scores are calculated and classified into special, high, medium, and low levels according to the scores. The weight of the leak source in special priority is multiplied by 5, high priority by 3, medium priority by 2, and low priority by 1 to guide the priority of path planning. The improved A-path planning algorithm is designed with a comprehensive cost function to prioritize the coverage of high-priority leakage sources and high-risk areas. Finally, starting from the robot's current position, the cost of each node is calculated according to the comprehensive cost function, and the path with the lowest cost is selected as the initial inspection path.

6. The target localization and path optimization method for a gas inspection robot based on multimodal perception according to claim 5, characterized in that: The feedback-based real-time path optimization is as follows: After the initial inspection path planning is completed, the path optimization trigger conditions include triggering emergency path insertion and prioritizing the dispatch of the robot to the new leak source area when a new suspected leak source is detected; triggering local path replanning when encountering no preset obstacles and adopting an arc detour strategy to avoid collisions; and triggering priority iteration and path adjustment when the leak source concentration changes beyond the threshold after inspection. The multi-objective optimization algorithm adopts the NSGA-II non-dominated sorting genetic algorithm, which sets four optimization objectives: shortest total path length, shortest response time of super-level and high-level leakage sources, fastest rate of decrease of leakage source concentration, and lowest path repetition rate. Each iteration generates a Pareto optimal path set, and the robot selects the optimal path according to the real-time environment. In multi-robot collaborative optimization, when a single robot detects more than two high-priority leakage sources, it sends a collaborative request containing the coordinates, concentration, and priority information of the leakage sources to neighboring robots through the 5G industrial gateway, and allocates tasks based on distance, concentration, and robot load.

7. The target localization and path optimization method for a gas inspection robot based on multimodal perception according to claim 6, characterized in that: The verification of the positioning and path optimization effect is as follows: The test scenario design covers three typical test scenarios: pipeline area in chemical industrial park, underground pipe gallery, and open-air storage tank area. The scenario parameters are matched with hardware adaptation requirements. The core verification indicators include dynamic positioning accuracy with a leakage source positioning error of ≤±30cm when the robot moves, emergency response time of ≤30s from detection to first inspection of a top-priority leakage source, long-term stability with sensor drift of ≤3%FS and 100% success rate of backup sensor switching without detection interruption, path optimization effect with invalid inspection path ratio of ≤15%, and collaborative efficiency with a total response time of ≤45s when two robots work together to handle three top-priority leakage sources. The parameter iterative optimization establishes a mapping database based on multi-scenario test data, including environmental parameters, multimodal feature fusion weights, dynamic Gaussian plume diffusion model parameters, and four-dimensional weight model parameters. After the robot is powered on, it collects real-time environmental parameters through the environmental perception module, automatically matches the optimal parameters in the database, and does not require manual debugging. After each inspection is completed, new data is stored in the database, and the mapping relationship is updated through the gradient descent algorithm.

Images