Poisonous and harmful substance leakage reinspection method, device and equipment based on unmanned aerial vehicle and storage medium

Abstract

The invention discloses a toxic and harmful substance leakage reinspection method, device and equipment based on an unmanned aerial vehicle, and a storage medium, relates to the technical field of Internet of Things and intelligent factories, and aims to solve the problem that the detection accuracy and efficiency of toxic and harmful substances are low. The method comprises the following steps: dividing a target area into a plurality of grids, and acquiring initial gas concentration data of each grid in real time through a fixed foundation sensing array; the grids with the initial gas concentration data larger than the preset concentration are determined as abnormal grids, and a reinspection flight path is dynamically generated according to the unmanned aerial vehicle parking position, the abnormal grid position and the environment parameters; the unmanned aerial vehicle is controlled to fly to the abnormal grid according to the reinspection flight path, and multi-dimensional data including visible light images, infrared thermal imaging data, gas component spectrums and space coordinate data of the abnormal grid are collected; and fusing the initial gas concentration data and the multi-dimensional data of the abnormal grid, constructing a three-dimensional dynamic concentration field model, and outputting leakage source position information and a diffusion trend.

Description

Technical Field

[0001] The present application relates to the technical field of the Internet of Things and smart factories, and in particular to a method, device, equipment and storage medium for re-inspecting leakage of toxic and hazardous substances based on drones. Background Art

[0002] In high-risk industries such as chemical and energy, rapid detection and precise location of leaks of toxic and hazardous substances are core issues of industrial safety.

[0003] Currently, traditional detection methods primarily rely on fixed gas sensor networks, which collect gas concentration data in real time through pre-installed probes on the ground to detect leaks. However, this fixed detection method has significant drawbacks: First, the deployment of fixed probes is limited by physical space and installation costs, making it difficult to cover all blind spots in complex factory areas, especially in areas with dense storage tanks and pipelines. Second, single gas concentration data lacks spatial correlation and cannot intuitively reflect the three-dimensional distribution and diffusion trend of the leak source, resulting in delayed emergency response. Therefore, existing detection methods for toxic and hazardous substances suffer from low detection accuracy and efficiency. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, equipment and storage medium for leakage re-inspection of toxic and hazardous substances based on drones, aiming to solve the problems of low detection accuracy and efficiency of toxic and hazardous substances.

[0005] To achieve the above objectives, this application adopts the following technical solutions: The present application provides a leak re-inspection method for toxic and hazardous substances based on a drone, the method comprising: dividing a target area into multiple grids, and acquiring initial gas concentration data of each grid in real time through a fixed ground-based sensor array; determining a grid whose initial gas concentration data is greater than a preset concentration as an abnormal grid, and dynamically generating a re-inspection flight path based on the drone's parking position, the abnormal grid's position, and environmental parameters; controlling the drone to fly to the abnormal grid along the re-inspection flight path, and synchronously collecting multi-dimensional data of the abnormal grid, the multi-dimensional data including visible light images, infrared thermal imaging data, gas composition spectra, and spatial coordinate data; fusing the initial gas concentration data of the abnormal grid with the multi-dimensional data to construct a three-dimensional dynamic concentration field model, and outputting leakage source location information and diffusion trend.

[0006] The embodiment of the present application provides a drone-based leakage re-inspection method for toxic and hazardous substances. Through grid partitioning and real-time monitoring of a fixed ground-based sensor array, it achieves full-coverage concentration scanning of the target area, reduces monitoring blind spots, and dynamically generates a re-inspection flight path based on the drone's parking position, abnormal grid position, and environmental parameters to ensure that the drone preferentially covers high-risk areas and avoids obstacles, thereby improving path planning efficiency. The drone is then controlled to synchronously collect multi-dimensional data such as visible light images, infrared thermal imaging, gas composition spectra, and spatial coordinates, and the comprehensiveness of leakage feature identification is enhanced through multi-source data complementarity. Finally, the ground-based sensor data and the drone's multi-dimensional data are integrated to construct a three-dimensional dynamic concentration field model, accurately restoring the three-dimensional spatial distribution and diffusion trend of the leakage source, thereby improving the detection accuracy and efficiency of toxic and hazardous substances.

[0007] In some embodiments, the above-mentioned environmental parameters include real-time wind direction data, obstacle distribution information and communication signal strength; the above-mentioned dynamic generation of the re-inspection flight path based on the drone parking position, abnormal grid position and environmental parameters includes: generating an initial obstacle avoidance path based on the drone parking position, abnormal grid position and obstacle distribution information, and adjusting the initial obstacle avoidance path in combination with real-time wind direction data and communication signal strength to obtain a re-inspection flight route. The re-inspection flight route is a continuous non-collision route with the strongest communication signal strength between the drone parking position and the abnormal grid position, and covering the upwind area of ​​the leakage source.

[0008] Based on this, this application dynamically optimizes the path by integrating wind direction, obstacles and signal strength, thereby improving the coverage efficiency of drones upwind of the leakage source, reducing the risk of communication interruption and enhancing the accuracy of re-inspection.

[0009] In some embodiments, the above-mentioned drones are multiple drones; the above-mentioned collection of multi-dimensional data of the abnormal grid includes: photographing the abnormal grid through a dual-spectral camera carried by the drone to obtain visible light images and infrared thermal imaging data; detecting the gas composition and concentration gradient in the abnormal grid through a gas mass spectrometer carried by the drone to generate a gas composition spectrum; controlling the laser radars carried by multiple drones to scan the abnormal grid from different angles to generate spatial coordinate data.

[0010] Based on this, this application realizes multimodal data complementarity and improves the integrity and timeliness of data collection by dividing the work of collecting spectral, mass spectrometric and three-dimensional coordinate data.

[0011] In some embodiments, the above-mentioned fusion of the initial gas concentration data of the abnormal grid and the multi-dimensional data to construct a three-dimensional dynamic concentration field model and output the leakage source location information and diffusion trend includes: denoising, coordinate normalization and feature extraction of the multi-dimensional data to generate a standardized data set; using a spatiotemporal interpolation algorithm to fuse the standardized data set with the initial gas concentration data to generate a dynamic concentration matrix; iteratively optimizing the dynamic concentration matrix based on the gas diffusion equation to construct a three-dimensional dynamic concentration field model, and outputting the leakage source location information and diffusion trend.

[0012] Based on this, this application avoids static model errors by combining spatiotemporal interpolation with physical equations, accurately restores the three-dimensional distribution of leakage, and improves the reliability of diffusion trend prediction.

[0013] In some embodiments, the above-mentioned denoising, coordinate normalization and feature extraction of multi-dimensional data to generate a standardized data set include: extracting regional edge features of the abnormal grid from the visible light image; identifying the temperature abnormality area in the abnormal grid from the infrared thermal imaging data; performing baseline correction and characteristic wavelength extraction on the gas component spectrum to obtain the gas component type; performing coordinate system normalization on the spatial coordinate data to obtain unified coordinate data; generating a standardized data set based on the regional edge features of the abnormal grid, the temperature abnormality area, the gas component type and the unified coordinate data.

[0014] Based on this, this application eliminates the heterogeneity of multi-source data through feature extraction and coordinate unification, enhances the consistency of model input, and reduces the impact of noise interference on positioning accuracy.

[0015] In some embodiments, the fixed ground-based sensor array includes at least two groups of sensors, which are diagonally distributed downwind of the dominant wind in the target area; and the spacing between the at least two groups of sensors is determined according to the type of toxic and hazardous substances.

[0016] Based on this, this application arranges sensors diagonally according to the dominant wind direction, and adjusts the spacing based on the material type to reduce missed detections caused by environmental interference and improve the accuracy of abnormal grid determination.

[0017] In some embodiments, the toxic and hazardous substances include at least ammonia and chlorine; the fixed ground-based sensing array includes an electrochemical sensor and an infrared spectral sensor; the electrochemical sensor is used to detect the ammonia concentration according to a first sampling frequency; the infrared spectral sensor is used to detect the chlorine concentration according to a second sampling frequency.

[0018] Based on this, this application customizes the sensor type and sampling frequency according to the characteristics of ammonia and chlorine to ensure rapid response to different leakage scenarios and improve concentration detection sensitivity.

[0019] The present application provides a leak re-inspection device for toxic and hazardous substances based on a drone, which includes: an acquisition unit, which is used to divide a target area into multiple grids and obtain the initial gas concentration data of each grid in real time through a fixed ground-based sensor array; a generation unit, which is used to determine a grid with initial gas concentration data greater than a preset concentration as an abnormal grid, and dynamically generate a re-inspection flight path based on the drone parking position, the abnormal grid position and environmental parameters; a control unit, which is used to control the drone to fly to the abnormal grid according to the re-inspection flight path, and synchronously collect multi-dimensional data of the abnormal grid, the multi-dimensional data including visible light images, infrared thermal imaging data, gas composition spectra and spatial coordinate data; and a construction unit, which is used to fuse the initial gas concentration data of the abnormal grid with the multi-dimensional data, construct a three-dimensional dynamic concentration field model, and output leakage source location information and diffusion trend.

[0020] In some embodiments, the above-mentioned environmental parameters include real-time wind direction data, obstacle distribution information and communication signal strength; the above-mentioned generation unit is specifically used to: generate an initial obstacle avoidance path based on the drone parking position, abnormal grid position and obstacle distribution information, and adjust the initial obstacle avoidance path in combination with the real-time wind direction data and communication signal strength to obtain a re-inspection flight route. The re-inspection flight route is a continuous non-collision route with the strongest communication signal strength between the drone parking position and the abnormal grid position, and covers the upwind area of ​​the leakage source.

[0021] In some embodiments, the above-mentioned drones are multiple drones; the above-mentioned control unit is specifically used to: photograph the abnormal grid through the dual-spectral camera carried by the drone to obtain visible light images and infrared thermal imaging data; detect the gas composition and concentration gradient in the abnormal grid through the gas mass spectrometer carried by the drone to generate a gas composition spectrum; control the laser radars carried by multiple drones to scan the abnormal grid from different angles to generate spatial coordinate data.

[0022] In some embodiments, the above-mentioned construction unit is specifically used to: denoise, normalize coordinates and extract features of multi-dimensional data to generate a standardized data set; use a spatiotemporal interpolation algorithm to fuse the standardized data set with the initial gas concentration data to generate a dynamic concentration matrix; iteratively optimize the dynamic concentration matrix based on the gas diffusion equation, construct a three-dimensional dynamic concentration field model, and output leakage source location information and diffusion trend.

[0023] In some embodiments, the above-mentioned construction unit is specifically used to: extract regional edge features of the abnormal grid from the visible light image; identify the temperature abnormality area in the abnormal grid from the infrared thermal imaging data; perform baseline correction and characteristic wavelength extraction on the gas component spectrum to obtain the gas component type; perform coordinate system normalization on the spatial coordinate data to obtain unified coordinate data; generate a standardized data set based on the regional edge features of the abnormal grid, the temperature abnormality area, the gas component type and the unified coordinate data.

[0024] In some embodiments, the fixed ground-based sensor array includes at least two groups of sensors, which are diagonally distributed downwind of the dominant wind in the target area; and the spacing between the at least two groups of sensors is determined according to the type of toxic and hazardous substances.

[0025] In some embodiments, the toxic and hazardous substances include ammonia and chlorine; the fixed ground-based sensing array includes an electrochemical sensor and an infrared spectral sensor; the electrochemical sensor is used to detect the ammonia concentration according to a first sampling frequency; the infrared spectral sensor is used to detect the chlorine concentration according to a second sampling frequency.

[0026] The present application provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement the above-described drone-based toxic and hazardous substance leakage re-inspection method.

[0027] The present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal, the terminal executes the above-described drone-based toxic and hazardous substance leakage re-inspection method.

[0028] The present application provides a computer program product comprising instructions. When a computer executes the instructions, the computer executes the above-described drone-based toxic and hazardous substance leakage re-inspection method.

[0029] The present application provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled, and the processor is used to run computer programs or instructions to implement the above-described drone-based toxic and hazardous substance leakage re-inspection method.

[0030] Specifically, the chip provided in the embodiment of the present application also includes a memory for storing computer programs or instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0032] Figure 1 A flowchart of a method for re-inspecting a toxic or hazardous substance leak using a drone provided in an embodiment of the present application; Figure 2 A schematic diagram of an abnormal grid used in a drone-based toxic and hazardous substance leakage re-inspection method provided in an embodiment of the present application; Figure 3 A structural diagram of a toxic and hazardous substance leakage re-inspection device based on a drone provided in an embodiment of the present application; Figure 4 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0034] In the description of this application, it should be understood that the terms "upper," "lower," "left," "right," "front," "back," "inner," "outer," and the like, indicating directions or positional relationships, are based on the directions or relative positional relationships shown in the accompanying drawings and are intended solely to facilitate the description of this application and simplify the description. They do not indicate or imply that the devices or components referred to must have a specific direction, be constructed, or operate in a specific direction. Therefore, they should not be construed as limitations on this application. Unless otherwise specified, the above-mentioned directionality descriptions may be flexibly set in actual application, provided that the relative positional relationships shown in the accompanying drawings are met.

[0035] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature specified as "first" or "second" may explicitly or implicitly include one or more of such features. Throughout this application, unless otherwise specified, "plurality" means two or more.

[0036] In the description of this application, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connected," and "connected" should be understood broadly. For example, they may refer to fixed connections, detachable connections, or integral connections. They may be directly connected, indirectly connected through an intermediary, or internally connected between two components. Those skilled in the art will understand the specific meanings of these terms in this application based on the specific circumstances.

[0037] In some embodiments, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, article, or apparatus that includes the element.

[0038] In some embodiments, words such as "exemplary" or "for example" are used to indicate an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0039] In the description of this specification, specific features, structures, materials or characteristics may be combined in an appropriate manner in any one or more embodiments or examples.

[0040] In recent years, drone technology has been introduced to the leak detection field, enabling mobile inspections by carrying gas sensors and cameras. However, existing solutions often rely on pre-set flight paths or manual control, lacking coordination with fixed monitoring networks, resulting in low detection efficiency. For example, when fixed sensors detect an anomaly, drones often need to scan the entire area, making it difficult to quickly identify the suspected leak. Furthermore, multi-dimensional data collected by drones, such as visible light images and infrared thermal imaging, is typically analyzed independently and not deeply integrated with ground-based sensor data, resulting in low detection accuracy.

[0041] Furthermore, existing methods for locating leak sources rely heavily on two-dimensional data, failing to accurately reproduce the concentration gradient of the leaked material in three dimensions, increasing the risk of misjudgment. Existing methods for predicting diffusion trends are often based on static meteorological parameters, lacking the dynamic correction of real-time multi-source data. This can lead to significant deviations between predicted results and actual diffusion trajectories.

[0042] Therefore, existing detection methods have the problem of low detection accuracy and efficiency for toxic and hazardous substances.

[0043] Against this backdrop, and to address the low accuracy and efficiency of toxic and hazardous substance detection in related technologies, this application provides a drone-based leak re-inspection method, device, equipment, and storage medium for toxic and hazardous substances. By integrating a fixed monitoring network with dynamic drone re-inspection, achieving multi-dimensional data fusion, and supporting three-dimensional leak modeling, the method improves the accuracy and efficiency of toxic and hazardous substance detection.

[0044] Refer to the following Figure 1 and Figure 2 The present invention describes a method for re-inspecting leakage of toxic and hazardous substances based on a drone, as provided in an embodiment of the present application.

[0045] Figure 1 The embodiment of the present application provides a method flow chart of a method for re-inspecting leakage of toxic and hazardous substances based on drones. The subject executing the method can be an electronic device or various devices / modules in an electronic device, such as an integrated circuit or chip, and the embodiment of the present application does not make specific limitations on this.

[0046] For example, Figure 1 As shown, the toxic and hazardous substance leakage re-inspection method based on a drone provided in an embodiment of the present application may include the following S101 to S104: S101, dividing the target area into multiple grids, and acquiring the initial gas concentration data of each grid in real time through a fixed ground-based sensor array.

[0047] In some embodiments, the grid density can be dynamically adjusted based on the topology of the geographic information system (GIS) according to the distribution of equipment within the plant and the risk level. For example, a high-density grid of 5m×5m can be used in the tank area, while a standard grid of 10m×10m can be used in open areas.

[0048] For example, in a chemical park, a 100m x 100m tank area can be divided into 20 x 20 5m x 5m grids, with sensor nodes deployed at the center of each grid. A fixed ground-based sensor array uses the long-range radio (LoRa) communication protocol to upload chlorine concentration data from each grid to a central control system at a 1Hz frequency.

[0049] In an embodiment of the present application, the fixed ground-based sensor array includes at least two groups of sensors.

[0050] Optionally, at least two groups of sensors are diagonally distributed downwind of the dominant wind in the target area.

[0051] For example, Figure 2 As shown in the figure, the target area is area A, and at least two sensor groups include sensor X and sensor Y. If the dominant wind direction in area A is from left to right, then the downwind direction is the right side of area A. In this case, sensor X can be deployed at point P, the upper right corner of area A, and sensor Y can be deployed at point Q, the lower right corner of area A.

[0052] Optionally, the distance between the at least two groups of sensors is determined according to the type of the toxic and hazardous substances.

[0053] In some embodiments, the spacing can be dynamically adjusted according to the diffusion coefficient of the toxic and hazardous substances.

[0054] For example, the distance L can be calculated using formula (1).

[0055] Formula (1) Where k is the correction coefficient; t is the monitoring period; and D is the diffusion coefficient.

[0056] Specifically, if the toxic and hazardous substance is ammonia, the diffusion coefficient D=0.28m² / s. If k=1.2 and the monitoring period t=60s, then the sensor spacing If chlorine is detected at the same time, the diffusion coefficient D=0.15m² / s, and the corresponding spacing is adjusted to .

[0057] In this way, the present application arranges sensors diagonally according to the dominant wind direction, and adjusts the spacing according to the material type to reduce missed detections caused by environmental interference and improve the accuracy of abnormal grid determination.

[0058] Optionally, in the embodiment of the present application, the sensor type can be matched according to gas characteristics.

[0059] In some embodiments, when the toxic and hazardous substances include at least ammonia and chlorine, the fixed ground-based sensor array includes an electrochemical gas sensor (EGS) and an infrared spectroscopy sensor (IRS). The electrochemical sensor is used to detect ammonia concentration at a first sampling frequency, and the infrared spectroscopy sensor is used to detect chlorine concentration at a second sampling frequency.

[0060] In the embodiment of the present application, the first sampling frequency and the second sampling frequency may be equal or unequal.

[0061] Optionally, both the first sampling frequency and the second sampling frequency can be flexibly adjusted according to environmental parameters.

[0062] For example, the first sampling frequency can be adjusted according to the wind speed. If the initial wind speed is 2 m / s, the default sampling frequency is 1 Hz. When the wind speed reaches 5 m / s, the sampling frequency can be increased (for example, adjusted to 4 Hz).

[0063] For example, the second sampling frequency can be adjusted according to the temperature. If the initial temperature is 25° C., the default sampling frequency is 0.5 Hz. When the temperature rises to 30° C., the sampling frequency can be increased (eg, adjusted to 1 Hz).

[0064] In this way, this application ensures rapid response to different leakage scenarios and improves concentration detection sensitivity by customizing the sensor type and sampling frequency according to the characteristics of ammonia and chlorine.

[0065] S102: Determine the grid whose initial gas concentration data is greater than the preset concentration as an abnormal grid, and dynamically generate a re-inspection flight path based on the drone parking location, the abnormal grid location and environmental parameters.

[0066] In some embodiments, the preset concentration may be set according to the lower explosive limit (LEL) of the gas. For example, the first level alarm threshold for ammonia is 50% of the LEL (corresponding to a volume concentration of 1.5%).

[0067] For example, taking the preset concentration as 2%, if the monitored ammonia concentration of grid A is 2.3%, then grid A is determined to be an abnormal grid.

[0068] In an embodiment of the present application, environmental parameters include real-time wind direction data, obstacle distribution information, and communication signal strength.

[0069] For example, real-time wind direction data can be obtained through a three-dimensional ultrasonic anemometer; three-dimensional maps (such as building information modeling (BIM)) can be constructed through lidar point cloud data to obtain obstacle distribution information; and communication signal strength can be queried through base station signal quality indicators.

[0070] In some embodiments, an initial obstacle avoidance path may be generated based on the parking location of the drone, abnormal grid location, and obstacle distribution information.

[0071] For example, the A* algorithm (such as the A-star algorithm) known in the field of path planning can be used, with the drone's parking position as the starting point, the center point of the abnormal grid as the end point, and the Euclidean distance as the heuristic function to avoid obstacles marked in the BIM model (such as buildings and tall equipment) to obtain an initial obstacle avoidance path.

[0072] Specifically, for example, if the 3D coordinates of the drone's parking location are (100, 200, 5) and the 3D coordinates of the center of the abnormal grid are (300, 400, 0), if the 3D coordinates of the obstacle distribution area are (200, 300, 0)-(250, 350, 10), the 3D coordinates of the initial obstacle avoidance path can be generated as (100, 200, 5)→(150, 300, 15)→(280, 380, 10)→(300, 400, 0) to avoid the obstacle.

[0073] Furthermore, the initial obstacle avoidance path is adjusted based on the real-time wind direction data and communication signal strength to obtain a rechecked flight route.

[0074] Among them, the re-inspection flight route is a continuous non-collision route with the strongest communication signal strength between the drone parking location and the abnormal grid location, and covering the upwind area of ​​the leakage source.

[0075] For example, a wind direction weight factor can be introduced to correct the path direction of the initial obstacle avoidance path so that the drone covers the upwind area of ​​the leakage source; at the same time, the communication signal strength of the initial obstacle avoidance path is measured, and the path corresponding to the communication signal strength in the initial obstacle avoidance path is adjusted to be less than the preset strength.

[0076] Specifically, taking the preset strength of -70dBm as an example, if the wind direction data indicates that the wind direction is from south to north, in order to allow the drone to cover the area upwind of the leak source, the initial obstacle avoidance path can be offset 10° to the south. At this time, if the communication signal strength of a portion of the initial obstacle avoidance path (for example, 50 meters) after the offset is -80dBm (lower than the preset strength of -70dBm), the portion of the path is offset 10° toward the base station to increase the signal strength to -65dBm (higher than the preset strength of -70dBm).

[0077] In this way, the present application dynamically optimizes the path by integrating wind direction, obstacles and signal strength, thereby improving the coverage efficiency of drones upwind of the leakage source, reducing the risk of communication interruption and enhancing the accuracy of re-inspection.

[0078] S103: Control the UAV to fly to the abnormal grid according to the re-inspection flight path, and simultaneously collect multi-dimensional data of the abnormal grid.

[0079] Among them, multi-dimensional data includes visible light images, infrared thermal imaging data, gas composition spectra and spatial coordinate data.

[0080] For example, visible light images are used to identify the surface morphology of leaking liquid in abnormal grids; infrared thermal imaging data are used to detect temperature abnormality areas in abnormal grids; gas spectral analysis components are used to identify the type of regional leaking liquid; and spatial coordinate data are used to construct a three-dimensional leakage distribution model.

[0081] Optionally, the drone in the embodiment of the present application may be multiple drones, and the multiple drones may form a drone array.

[0082] Optionally, in the case where there are multiple drones, the above-mentioned collection of multi-dimensional data of abnormal grids can be achieved in the following manner.

[0083] In some embodiments, for visible light images and infrared thermal imaging data, the abnormal grid can be photographed by a dual-spectrum camera carried by a drone to obtain visible light images and infrared thermal imaging data.

[0084] For example, dual-spectral imaging technology can be used to simultaneously acquire the visual and temperature characteristics of abnormal grids. The dual-spectral camera can use a coaxial optical design for visible light and infrared thermal imaging, and achieve pixel-level alignment through image registration algorithms.

[0085] Furthermore, under low-light conditions, the visible light camera can automatically switch to the starlight-level sensor, and the infrared thermal imaging module can enable noise-equivalent temperature difference.

[0086] In some embodiments, for the gas composition spectrum, a gas mass spectrometer carried by a drone can be used to detect the gas composition and concentration gradient in the abnormal grid to generate a gas composition spectrum.

[0087] For example, quantitative and qualitative detection of gas components can be achieved through mass spectrometry. A bulk mass spectrometer uses a quadrupole mass analyzer and ion mobility spectrometry (IMS) pre-separation technology to monitor different leaked gases (such as ammonia (molecular weight 17) and chlorine (molecular weight 71)).

[0088] In some embodiments, for spatial coordinate data, laser radars carried by multiple drones can be controlled to scan abnormal grids from different angles to generate spatial coordinate data.

[0089] For example, multi-view 3D reconstruction can be used to obtain the spatial coordinates of the leak area within the anomaly grid. Specifically, four drones can be deployed in an orthogonal array, each equipped with a 16-line lidar (ranging accuracy ±2cm). Using the iterative closest point (ICP) algorithm for point cloud registration, a 3D model with a spatial resolution of 5mm can be constructed to determine the spatial coordinates of the leak source.

[0090] In this way, the present application realizes multimodal data complementarity and improves the integrity and timeliness of data collection by dividing the work of collecting spectral, mass spectrometric and three-dimensional coordinate data.

[0091] S104. Fusing the initial gas concentration data of the abnormal grid with the multi-dimensional data, constructing a three-dimensional dynamic concentration field model, and outputting the leakage source location information and diffusion trend.

[0092] Optionally, before constructing the three-dimensional dynamic concentration field model, the multi-dimensional data may be preprocessed to eliminate invalid data.

[0093] In some embodiments, multi-dimensional data may be subjected to denoising, coordinate normalization, and feature extraction to generate a standardized data set.

[0094] For example, the regional edge features of the abnormal grid can be extracted from the visible light image; the temperature abnormality area in the abnormal grid can be identified from the infrared thermal imaging data; the gas component spectrum can be baseline corrected and the characteristic wavelength extracted to obtain the gas component type; the spatial coordinate data can be normalized to obtain unified coordinate data.

[0095] Specifically, the U-Net++ neural network (segmentation accuracy of 92.3%) can be used to perform semantic segmentation of visible light images and extract the edges of leakage areas; the threshold segmentation method can be used to identify low-temperature areas in infrared thermal imaging data (such as low-temperature areas less than -15°C); the wavelet transform can be used to perform baseline calibration on the gas component spectrum, and the machine learning algorithm can be combined to identify characteristic wavelengths (such as the characteristic wavelength of the ammonia spectrum is 2.26μm); the coordinate system of the spatial coordinate data can be normalized through the coordinate transformation matrix and converted to the global coordinate system.

[0096] Furthermore, a standardized data set can be generated based on the regional edge features, temperature anomaly areas, gas composition types and unified coordinate data obtained after preprocessing.

[0097] In this way, the present application eliminates the heterogeneity of multi-source data through feature extraction and coordinate unification, enhances the consistency of model input, and reduces the impact of noise interference on positioning accuracy.

[0098] Optionally, after obtaining the preprocessed standardized data set, a spatiotemporal interpolation algorithm can be used to fuse the standardized data set with the initial gas concentration data to generate a dynamic concentration matrix; then, the dynamic concentration matrix is ​​iteratively optimized based on the gas diffusion equation to construct a three-dimensional dynamic concentration field model, and the leakage source location information and diffusion trend are output.

[0099] In some embodiments, since the initial gas concentration points collected by the fixed ground-based sensor array are sparse and the standardized data set collected by the drone is discontinuous in time, the standardized data set collected and processed by the drone can be fused with the initial gas concentration collected by the fixed ground-based sensor array to output a dynamic concentration matrix covering the entire area.

[0100] For example, the spatiotemporal data fusion can be performed using the Kriging interpolation method. The discrete initial gas concentration data is interpolated into a continuous concentration field based on the standardized data set. The concentration matrix is ​​then iteratively optimized based on the Gaussian diffusion model (GDM). The Gaussian diffusion model can be represented by Formula (2).

[0101] Formula (2) in, for The gas concentration at the location where the leak occurs; Q is the leak source intensity, which indicates the mass of material released from the leak source per unit time; u is the average wind speed; is the horizontal diffusion coefficient, which is used to describe the diffusion degree of gas in the horizontal direction (y-axis); is the vertical diffusion coefficient, which is used to describe the diffusion degree of gas in the vertical direction (z-axis).

[0102] Specifically, taking an ammonia leak scenario as an example, initial gas concentration data from a fixed ground-based sensor array (e.g., three point concentrations of 200 ppm, 280 ppm, and 350 ppm) is fused with the 520 ppm data from drone detection through kriging interpolation to generate a 1m×1m×1m concentration matrix. Using GDM to iteratively optimize the concentration matrix, with an input wind speed of 6 m / s and a stability level of D, the iterative optimization generates a three-dimensional concentration field. The three-dimensional coordinates of the leak source are predicted to be (305, 405, 2), and the concentration at 200 meters downwind will reach 100 ppm in the next 10 minutes.

[0103] In the UAV-based leakage re-inspection method for toxic and hazardous substances provided in the embodiment of the present application, through grid partitioning and real-time monitoring of a fixed ground-based sensor array, full coverage concentration scanning of the target area is achieved, reducing monitoring blind spots, and then the re-inspection flight path is dynamically generated in combination with the UAV parking position, abnormal grid position and environmental parameters to ensure that the UAV preferentially covers high-risk areas and avoids obstacles, thereby improving path planning efficiency; then the UAV is controlled to synchronously collect multi-dimensional data such as visible light images, infrared thermal imaging, gas composition spectra and spatial coordinates, and the comprehensiveness of leakage feature identification is enhanced through multi-source data complementarity; finally, the ground-based sensor data and the UAV multi-dimensional data are integrated to construct a three-dimensional dynamic concentration field model, accurately restoring the three-dimensional spatial distribution and diffusion trend of the leakage source, thereby improving the detection accuracy and efficiency of toxic and hazardous substances.

[0104] Optionally, the electronic equipment and physical components involved in the embodiments of the present application (such as sensors in a fixed ground-based sensor array and drones) are all anti-static and explosion-proof equipment, and will not generate ignition sources such as static electricity, open flames, and electric sparks during actual operation.

[0105] In this way, the present application improves production safety by using anti-static and explosion-proof equipment to avoid ignition sources such as static electricity generated during operation, which may cause fires, explosions, and other hazards.

[0106] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, the leakage re-inspection device or electronic device for toxic and hazardous substances based on drones includes hardware structures and / or software modules corresponding to the execution of each function. It should be easy for those skilled in the art to realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0107] The embodiment of the present application can, according to the above method, exemplarily divide the functional modules of the leakage re-inspection device or electronic device based on drone for toxic and hazardous substances. For example, the leakage re-inspection device or electronic device based on drone for toxic and hazardous substances may include various functional modules corresponding to the various functional divisions, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0108] Figure 3 This is a structural diagram of a drone-based toxic and hazardous substance leakage re-inspection device provided in an embodiment of the present application. The drone-based toxic and hazardous substance leakage re-inspection device 300 includes: an acquisition unit 301, a generation unit 302, a control unit 303, and a construction unit 304.

[0109] Among them: the acquisition unit 301 is used to divide the target area into multiple grids and obtain the initial gas concentration data of each grid in real time through a fixed ground-based sensor array; the generation unit 302 is used to determine the grid whose initial gas concentration data is greater than the preset concentration as an abnormal grid, and dynamically generate a re-inspection flight path based on the parking position of the UAV, the position of the abnormal grid and the environmental parameters; the control unit 303 is used to control the UAV to fly to the abnormal grid according to the re-inspection flight path, and synchronously collect multi-dimensional data of the abnormal grid, the multi-dimensional data including visible light images, infrared thermal imaging data, gas composition spectra and spatial coordinate data; the construction unit 304 is used to fuse the initial gas concentration data of the abnormal grid with the multi-dimensional data, construct a three-dimensional dynamic concentration field model, and output the leakage source location information and diffusion trend.

[0110] In some embodiments, the above-mentioned environmental parameters include real-time wind direction data, obstacle distribution information and communication signal strength; the above-mentioned generation unit 302 is specifically used to: generate an initial obstacle avoidance path based on the drone parking position, abnormal grid position and obstacle distribution information, and adjust the initial obstacle avoidance path in combination with the real-time wind direction data and communication signal strength to obtain a re-inspection flight route. The re-inspection flight route is a continuous non-collision route with the strongest communication signal strength between the drone parking position and the abnormal grid position, and covers the upwind area of ​​the leakage source.

[0111] In some embodiments, the above-mentioned drones are multiple drones; the above-mentioned control unit 303 is specifically used to: photograph the abnormal grid through the dual-spectral camera carried by the drone to obtain visible light images and infrared thermal imaging data; detect the gas composition and concentration gradient in the abnormal grid through the gas mass spectrometer carried by the drone to generate a gas composition spectrum; control the laser radars carried by multiple drones to scan the abnormal grid from different angles to generate spatial coordinate data.

[0112] In some embodiments, the construction unit 304 is specifically used to: denoise, normalize coordinates, and extract features from multi-dimensional data to generate a standardized data set; fuse the standardized data set with the initial gas concentration data using a spatiotemporal interpolation algorithm to generate a dynamic concentration matrix; iteratively optimize the dynamic concentration matrix based on the gas diffusion equation to construct a three-dimensional dynamic concentration field model, and output leakage source location information and diffusion trends.

[0113] In some embodiments, the construction unit 304 is specifically used to: extract regional edge features of the abnormal grid from the visible light image; identify temperature anomaly areas in the abnormal grid from infrared thermal imaging data; perform baseline correction and characteristic wavelength extraction on the gas component spectrum to obtain the gas component type; perform coordinate system normalization on the spatial coordinate data to obtain unified coordinate data; and generate a standardized data set based on the regional edge features of the abnormal grid, the temperature anomaly area, the gas component type, and the unified coordinate data.

[0114] In some embodiments, the fixed ground-based sensor array includes at least two groups of sensors, which are diagonally distributed downwind of the dominant wind in the target area; and the spacing between the at least two groups of sensors is determined according to the type of toxic and hazardous substances.

[0115] In some embodiments, the toxic and hazardous substances include ammonia and chlorine; the fixed ground-based sensing array includes an electrochemical sensor and an infrared spectral sensor; the electrochemical sensor is used to detect the ammonia concentration according to a first sampling frequency; the infrared spectral sensor is used to detect the chlorine concentration according to a second sampling frequency.

[0116] In the drone-based toxic and hazardous substance leakage re-inspection device provided in the embodiment of the present application, through grid partitioning and real-time monitoring of a fixed ground-based sensor array, full coverage concentration scanning of the target area is achieved, reducing monitoring blind spots, and then the re-inspection flight path is dynamically generated in combination with the drone parking position, abnormal grid position and environmental parameters to ensure that the drone preferentially covers high-risk areas and avoids obstacles, thereby improving path planning efficiency; then the drone is controlled to synchronously collect multi-dimensional data such as visible light images, infrared thermal imaging, gas composition spectra and spatial coordinates, and the comprehensiveness of leakage feature identification is enhanced through multi-source data complementarity; finally, the ground-based sensor data and the drone's multi-dimensional data are integrated to construct a three-dimensional dynamic concentration field model, accurately restoring the three-dimensional spatial distribution and diffusion trend of the leakage source, thereby improving the detection accuracy and efficiency of toxic and hazardous substances.

[0117] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0118] Figure 4 This is a structural diagram of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device 400 includes but is not limited to: a processor 401 and a memory 402 .

[0119] The memory 402 is used to store executable instructions of the processor 401. It is understood that the processor 401 is configured to execute instructions to implement the UAV-based toxic and hazardous substance leakage re-inspection method in the above embodiment.

[0120] It should be noted that those skilled in the art can understand that Figure 4 The electronic device structure shown in the figure does not limit the electronic device, and the electronic device may include Figure 4 More or fewer components may be shown, or certain components may be combined, or the components may be arranged differently.

[0121] The processor 401 is the control center of the electronic device. It connects the various parts of the entire electronic device using various interfaces and lines. By running or executing software programs and / or modules stored in the memory 402 and accessing data stored in the memory 402, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. The processor 401 may include one or more processing units. Optionally, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly handles wireless communications. It is understood that the modem processor may not be integrated into the processor 401.

[0122] Memory 402 can be used to store software programs and various data. Memory 402 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and application programs required by at least one functional module (such as a determination unit, a processing unit, etc.). Furthermore, memory 402 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0123] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 402 including instructions. The above instructions can be executed by the processor 401 of the electronic device 400 to implement the drone-based toxic and hazardous substance leakage re-inspection method in the above embodiment.

[0124] In actual implementation, Figure 3 The steps performed by the acquisition unit 301, the generation unit 302, the control unit 303 and the construction unit 304 in Figure 4 The processor 401 in the embodiment calls the computer program stored in the memory 402. The specific execution process can be referred to the description of the method part in the above embodiment, which will not be repeated here.

[0125] Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0126] In an exemplary embodiment, the present application also provides a computer program product comprising one or more instructions, which can be executed by the processor 401 of the electronic device to complete the drone-based toxic and hazardous substance leakage re-inspection method in the above embodiment.

[0127] It should be noted that when the instructions in the above-mentioned computer-readable storage medium or one or more instructions in the computer program product are executed by the processor of the electronic device, the various processes of the above-mentioned method embodiment are implemented and the same technical effect as the above-mentioned method can be achieved. To avoid repetition, they will not be repeated here.

[0128] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete the full classification or partial functions described above.

[0129] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0130] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0131] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0132] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the prior art, or the entire classification part or part of the technical solution can be embodied in the form of a software product. The software product is stored in a storage medium and includes a number of instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute the entire classification part or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM, RAM, disk or optical disk, etc., various media that can store program code.

[0133] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for leak re-inspection of toxic and hazardous substances based on drones, characterized in that: The method comprises: The target area is divided into multiple grids, and the initial gas concentration data of each grid is obtained in real time through a fixed ground-based sensor array; The grids where the initial gas concentration data is greater than the preset concentration are determined as abnormal grids, and a re-inspection flight path is dynamically generated based on the parking location of the UAV, the location of the abnormal grid and the environmental parameters; Controlling the UAV to fly to the abnormal grid according to the re-inspection flight path, and synchronously collecting multi-dimensional data of the abnormal grid, wherein the multi-dimensional data includes visible light images, infrared thermal imaging data, gas composition spectra, and spatial coordinate data; The initial gas concentration data of the abnormal grid is integrated with the multi-dimensional data to construct a three-dimensional dynamic concentration field model, and the leakage source location information and diffusion trend are output.

2. The method according to claim 1, characterized in that The environmental parameters include real-time wind direction data, obstacle distribution information, and communication signal strength; the dynamic generation of the recheck flight path based on the drone's parking location, abnormal grid location, and environmental parameters includes: Generate an initial obstacle avoidance path based on the parking position of the UAV, the abnormal grid position and the obstacle distribution information; Combined with the real-time wind direction data and the communication signal strength, the initial obstacle avoidance path is adjusted to obtain the re-inspection flight route. The re-inspection flight route is a continuous non-collision route with the strongest communication signal strength between the UAV parking position and the abnormal grid position, and covering the upwind area of ​​the leakage source.

3. The method according to claim 2, characterized in that The drones are multiple drones; The collecting of multi-dimensional data of the abnormal grid includes: photographing the abnormal grid using a dual-spectrum camera carried by the drone to obtain the visible light image and the infrared thermal imaging data; Detecting the gas composition and concentration gradient in the abnormal grid by a gas mass spectrometer carried by the drone to generate the gas composition spectrum; The laser radars carried by the multiple drones are controlled to scan the abnormal grid from different angles to generate the spatial coordinate data.

4. The method according to any one of claims 1 to 3, characterized in that The fusion of the initial gas concentration data of the abnormal grid and the multi-dimensional data to construct a three-dimensional dynamic concentration field model and output the leakage source location information and diffusion trend includes: Performing denoising, coordinate normalization, and feature extraction on the multi-dimensional data to generate a standardized data set; fusing the standardized data set with the initial gas concentration data using a spatiotemporal interpolation algorithm to generate a dynamic concentration matrix; The dynamic concentration matrix is ​​iteratively optimized based on the gas diffusion equation, the three-dimensional dynamic concentration field model is constructed, and the leakage source location information and diffusion trend are output.

5. The method according to claim 4, characterized in that The step of performing denoising, coordinate normalization, and feature extraction on the multi-dimensional data to generate a standardized data set includes: Extracting regional edge features of the abnormal grid from the visible light image; identifying temperature anomaly areas in the anomaly grid from the infrared thermal imaging data; Performing baseline correction and characteristic wavelength extraction on the gas component spectrum to obtain the gas component type; Normalizing the spatial coordinate data to obtain unified coordinate data; The standardized data set is generated according to the regional edge features, the temperature anomaly region, the gas component type and the unified coordinate data.

6. The method according to claim 1, characterized in that The fixed ground-based sensor array includes at least two groups of sensors, which are diagonally distributed downwind of the dominant wind in the target area; and the spacing between the at least two groups of sensors is determined according to the type of the toxic and hazardous substances.

7. The method according to claim 1 or 6, characterized in that The toxic and hazardous substances include at least ammonia and chlorine; The fixed ground-based sensor array includes an electrochemical sensor and an infrared spectrum sensor; The electrochemical sensor is used to detect the ammonia concentration according to a first sampling frequency; The infrared spectrum sensor is used to detect the chlorine concentration according to a second sampling frequency.

8. A toxic and hazardous substance leakage re-inspection device based on drone, characterized in that: The device comprises: An acquisition unit is used to divide the target area into multiple grids and acquire the initial gas concentration data of each grid in real time through a fixed ground-based sensor array; a generating unit, configured to determine a grid whose initial gas concentration data is greater than a preset concentration as an abnormal grid, and dynamically generate a re-inspection flight path based on the parking location of the UAV, the location of the abnormal grid, and environmental parameters; a control unit, configured to control the UAV to fly to the abnormal grid along the re-inspection flight path and simultaneously collect multi-dimensional data of the abnormal grid, wherein the multi-dimensional data includes visible light images, infrared thermal imaging data, gas composition spectra, and spatial coordinate data; A construction unit is used to fuse the initial gas concentration data of the abnormal grid with the multi-dimensional data, construct a three-dimensional dynamic concentration field model, and output leakage source location information and diffusion trend.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing instructions, characterized in that: When a computer executes the instruction, the computer performs the method according to any one of claims 1 to 7.

Images