Unmanned aerial vehicle-based method and device for re-inspecting leakage of toxic and harmful substances, equipment and storage medium

Abstract

The application discloses a toxic and harmful substance leakage re-inspection method and device based on a UAV, an equipment and a storage medium, relates to the technical field of Internet of Things and intelligent factory, and aims to solve the problems of low detection accuracy and low efficiency of toxic and harmful substances. The method comprises the following steps: dividing a target area into a plurality of grids, acquiring initial gas concentration data of each grid in real time through a fixed ground sensor array; determining a grid with initial gas concentration data greater than a preset concentration as an abnormal grid, dynamically generating a re-inspection flight path according to a UAV parking position, an abnormal grid position and environmental parameters; controlling the UAV to fly to the abnormal grid according to the re-inspection flight path, collecting multi-dimensional data of the abnormal grid, including visible light images, infrared thermal imaging data, gas component spectra and spatial coordinate data; fusing the initial gas concentration data of the abnormal grid and the multi-dimensional data, constructing a three-dimensional dynamic concentration field model, and outputting leakage source position information and diffusion trend.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet of Things and smart factory, and particularly relates to a toxic and harmful substance leakage re-inspection method and device based on a UAV, equipment and a storage medium. BACKGROUND

[0002] In high-risk industries such as chemical industry and energy industry, the rapid detection and accurate positioning of toxic and harmful substance leakage is a core problem of industrial safety.

[0003] At present, the traditional detection means mainly relies on a fixed gas sensor network to realize leakage detection by collecting gas concentration data in real time through a pre-installed probe on the ground. However, this fixed detection method has significant defects: first, the deployment position of the fixed probe is limited by physical space and installation cost, and it is difficult to cover all blind areas of a complex plant, especially in the storage tank group and pipeline intensive area, which is prone to form a monitoring dead angle; second, the single gas concentration data lacks spatial correlation and cannot intuitively reflect the three-dimensional distribution and diffusion trend of the leakage source, resulting in delayed emergency response. Therefore, the existing detection means has the problems of low detection accuracy and low detection efficiency for toxic and harmful substances. SUMMARY

[0004] The purpose of the present application is to provide a toxic and harmful substance leakage re-inspection method, device, equipment and storage medium based on a UAV, which aims to solve the problem of low detection accuracy and low detection efficiency for toxic and harmful substances.

[0005] To achieve the above purpose, the present application adopts the following technical solutions:

[0006] The present application provides a toxic and harmful substance leakage re-inspection method based on a UAV, which comprises: dividing a target area into a plurality of grids, and acquiring initial gas concentration data of each grid in real time through a fixed ground sensor array; determining a grid with initial gas concentration data greater than a preset concentration as an abnormal grid, and dynamically generating a re-inspection flight path according to the UAV parking position, the abnormal grid position and the environmental parameters; controlling the UAV to fly to the abnormal grid according to the re-inspection flight path, synchronously collecting multi-dimensional data of the abnormal grid, the multi-dimensional data including visible light images, infrared thermal imaging data, gas component spectrum and spatial coordinate data; fusing the initial gas concentration data and the multi-dimensional data of the abnormal grid to construct a three-dimensional dynamic concentration field model, and outputting the leakage source position information and the diffusion trend.

[0007] The method for re-inspecting leakage of toxic and harmful substances based on a UAV provided by the embodiments of the present application realizes full-coverage concentration scanning of a target area by real-time monitoring of the grid partition and the fixed ground-based sensing array, reduces the monitoring blind area, dynamically generates a re-inspection flight path in combination with the UAV parking position, the abnormal grid position and the environmental parameters, ensures that the UAV preferentially covers the high-risk area and avoids obstacles, and improves the path planning efficiency; then the UAV is controlled to synchronously collect multi-dimensional data such as visible light images, infrared thermal images, gas component spectra and spatial coordinates, the comprehensiveness of leakage feature identification is enhanced through multi-source data complementation, finally, a three-dimensional dynamic concentration field model is constructed by fusing the ground-based sensing data and the multi-dimensional data of the UAV, the three-dimensional spatial distribution and diffusion trend of the leakage source are accurately restored, and therefore the detection accuracy and efficiency of the toxic and harmful substances are improved.

[0008] In some embodiments, the environmental parameters include real-time wind direction data, obstacle distribution information and communication signal strength; and the dynamic generation of the re-inspection flight path according to the UAV parking position, the abnormal grid position and the environmental parameters comprises: generating an initial obstacle avoidance path based on the UAV parking position, the abnormal grid position and the obstacle distribution information, adjusting the initial obstacle avoidance path in combination with the real-time wind direction data and the communication signal strength to obtain a re-inspection flight route, and the re-inspection flight route is a continuous non-collision route between the UAV parking position and the abnormal grid position with the strongest communication signal strength and covering the upwind area of the leakage source.

[0009] Based on this, the present application dynamically optimizes the path by fusing the wind direction, obstacles and signal strength, improves the coverage efficiency of the UAV on the upwind of the leakage source, reduces the risk of communication interruption, and enhances the re-inspection accuracy.

[0010] In some embodiments, the UAV is a plurality of UAVs; and the collection of the multi-dimensional data of the abnormal grid comprises: photographing the abnormal grid by a dual-spectrum camera carried by the UAV to obtain visible light images and infrared thermal imaging data; detecting the gas components and concentration gradient in the abnormal grid by a gas mass spectrometer carried by the UAV to generate a gas component spectrum; and controlling a laser radar carried by the plurality of UAVs to scan the abnormal grid from different angles to generate spatial coordinate data.

[0011] Based on this, the present application realizes multi-modal data complementation by dividing the collection of spectral, mass spectral and three-dimensional coordinate data, improves the data collection integrity and timeliness.

[0012] In some embodiments, the initial gas concentration data of the fusion abnormal grid and the multi-dimensional data are used to construct a three-dimensional dynamic concentration field model, and the leakage source position information and diffusion trend are output, including: denoising, coordinate normalization and feature extraction are performed on the multi-dimensional data to generate a standardized data set; the standardized data set and the initial gas concentration data are fused by using a space-time interpolation algorithm to generate a dynamic concentration matrix; the dynamic concentration matrix is iteratively optimized based on a gas diffusion equation to construct a three-dimensional dynamic concentration field model, and the leakage source position information and diffusion trend are output.

[0013] Based on this, the application combines space-time interpolation and physical equations to avoid static model errors, accurately restore the three-dimensional distribution of leakage, and improve the reliability of diffusion trend prediction.

[0014] In some embodiments, the denoising, coordinate normalization and feature extraction of the multi-dimensional data to generate a standardized data set include: extracting the regional edge features of the abnormal grid from the visible light image; identifying the temperature abnormal area in the abnormal grid from the infrared thermal imaging data; performing baseline correction and feature wavelength extraction on the gas component spectrum to obtain the gas component type; performing coordinate system normalization on the spatial coordinate data to obtain unified coordinate data; and generating a standardized data set according to the regional edge features of the abnormal grid, the temperature abnormal area, the gas component type and the unified coordinate data.

[0015] Based on this, the application eliminates the heterogeneity of multi-source data through feature extraction and coordinate unification, enhances the consistency of model input, and reduces the influence of noise interference on positioning accuracy.

[0016] In some embodiments, the fixed ground-based sensing array includes at least two groups of sensors, and the at least two groups of sensors are diagonally distributed according to the dominant wind direction of the target area; the spacing of the at least two groups of sensors is determined according to the type of toxic and harmful substances.

[0017] Based on this, the application reduces missed detection of environmental interference and improves the accuracy of abnormal grid determination by diagonally arranging sensors according to the dominant wind direction and adjusting the spacing according to the type of substances.

[0018] In some embodiments, the toxic and harmful substances at least include ammonia and chlorine; the fixed ground-based sensing array includes an electrochemical sensor and an infrared spectrum sensor; the electrochemical sensor is used to detect the ammonia concentration according to a first sampling frequency; and the infrared spectrum sensor is used to detect the chlorine concentration according to a second sampling frequency.

[0019] Based on this, the application customizes the sensor type and sampling frequency according to the characteristics of ammonia and chlorine to ensure rapid response to different leakage scenarios and improve the sensitivity of concentration detection.

[0020] The application provides a toxic and harmful substance leakage re-inspection device based on a UAV, which comprises: an acquisition unit, configured to divide a target area into a plurality of grids and acquire initial gas concentration data of each grid in real time through a fixed ground sensor array; a generation unit, configured to determine a grid with initial gas concentration data greater than a preset concentration as an abnormal grid, and dynamically generate a re-inspection flight path according to a UAV parking position, an abnormal grid position and environmental parameters; a control unit, configured to control the UAV to fly to the abnormal grid according to the re-inspection flight path, and synchronously collect multi-dimensional data of the abnormal grid, wherein the multi-dimensional data comprises visible light images, infrared thermal imaging data, gas component spectra and spatial coordinate data; and a construction unit, configured to fuse the initial gas concentration data of the abnormal grid and the multi-dimensional data, construct a three-dimensional dynamic concentration field model, and output leakage source position information and diffusion trend.

[0021] In some embodiments, the environmental parameters comprise real-time wind direction data, obstacle distribution information and communication signal strength; and the generation unit is specifically configured to generate an initial obstacle avoidance path based on the UAV parking position, the abnormal grid position and the obstacle distribution information, adjust the initial obstacle avoidance path in combination with the real-time wind direction data and the communication signal strength to obtain a re-inspection flight route, and the re-inspection flight route is a continuous non-collision route with the strongest communication signal strength between the UAV parking position and the abnormal grid position and covering the upwind area of the leakage source.

[0022] In some embodiments, the UAV is a plurality of UAVs; and the control unit is specifically configured to control a dual-spectrum camera carried by the UAV to take pictures of the abnormal grid to obtain visible light images and infrared thermal imaging data, control a gas mass spectrometer carried by the UAV to detect gas components and concentration gradients in the abnormal grid to generate gas component spectra, and control a laser radar carried by the plurality of UAVs to scan the abnormal grid from different angles to generate spatial coordinate data.

[0023] In some embodiments, the construction unit is specifically configured to perform denoising, coordinate normalization and feature extraction on the multi-dimensional data to generate a standardized data set, fuse the standardized data set and the initial gas concentration data by using a space-time interpolation algorithm to generate a dynamic concentration matrix, iteratively optimize the dynamic concentration matrix based on a gas diffusion equation to construct a three-dimensional dynamic concentration field model, and output leakage source position information and diffusion trend.

[0024] In some embodiments, the construction unit described above is specifically configured to: extract region edge features of the abnormal grid from the visible light image; identify temperature abnormal regions in the abnormal grid from the infrared thermal imaging data; perform baseline correction and feature wavelength extraction on the gas component spectrum to obtain a gas component type; perform coordinate system normalization on the spatial coordinate data to obtain unified coordinate data; and generate a standardized data set according to the region edge features of the abnormal grid, the temperature abnormal regions, the gas component type, and the unified coordinate data.

[0025] In some embodiments, the fixed ground-based sensing array described above includes at least two groups of sensors, which are diagonally distributed in the downwind direction of the dominant wind direction of the target area; the spacing of the at least two groups of sensors is determined according to the type of the toxic and harmful substance.

[0026] In some embodiments, the toxic and harmful substance includes ammonia and chlorine; the fixed ground-based sensing array includes an electrochemical sensor and an infrared spectrum sensor; the electrochemical sensor is used to detect the concentration of ammonia at a first sampling frequency; and the infrared spectrum sensor is used to detect the concentration of chlorine at a second sampling frequency.

[0027] The present application provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the above-described method for re-inspecting a leak of a toxic and harmful substance based on a UAV.

[0028] The present application provides a computer-readable storage medium having instructions stored therein, which, when executed on a terminal, cause the terminal to perform the above-described method for re-inspecting a leak of a toxic and harmful substance based on a UAV.

[0029] The present application provides a computer program product containing instructions, which, when executed by a computer, cause the computer to perform the above-described method for re-inspecting a leak of a toxic and harmful substance based on a UAV.

[0030] The present application provides a chip, comprising a processor and a communication interface, the communication interface and the processor being coupled, the processor being configured to run a computer program or instructions to implement the above-described method for re-inspecting a leak of a toxic and harmful substance based on a UAV.

[0031] Specifically, the chip provided in the embodiments of the present application further includes a memory for storing the computer program or instructions. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0033] Figure 1 A method flowchart of a toxic and harmful substance leakage re-inspection method based on a UAV provided by an embodiment of the present application;

[0034] Figure 2 A schematic diagram of an abnormal grid applied by a toxic and harmful substance leakage re-inspection method based on a UAV provided by an embodiment of the present application;

[0035] Figure 3 A structural diagram of a toxic and harmful substance leakage re-inspection device based on a UAV provided by an embodiment of the present application;

[0036] Figure 4 A structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present application.

[0038] In the description of the present application, it should be understood that the terms "upper", "lower", "left", "right", "front", "back", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or relative position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element indicated must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. Unless otherwise specified, the above orientation description can be flexibly arranged in the actual application process under the condition of meeting the relative position relationship shown in the drawings.

[0039] The terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, the meaning of "multiple" is two or more.

[0040] In the description of the application, it is necessary to explain that, unless otherwise explicitly specified and limited, the terms "mount", "connect", "connection", "communication" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integrally connected. It can be directly connected, or indirectly connected through an intermediate medium, or the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in this application can be understood according to the specific circumstances.

[0041] In some embodiments, the terms "comprising", "containing" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, article or device. Without more limitation, the element defined by the sentence "including a…" does not exclude the presence of other identical elements in the process, article or device including the element.

[0042] In some embodiments, the words "exemplary" or "for example" are used to mean serving as an example or illustration. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of the words "exemplary" or "for example" is intended to present concepts in a concrete manner.

[0043] In the description of the specification, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0044] In recent years, unmanned aerial vehicle technology has been introduced into the field of leakage detection, and mobile inspection is realized by carrying gas sensors and camera equipment. However, the existing scheme mostly adopts preset flight path or manual control mode, lacks cooperative mechanism with fixed monitoring network, and leads to low detection efficiency. For example, when the fixed sensor detects an anomaly, the unmanned aerial vehicle often needs to scan the whole area, and cannot quickly lock the suspected leakage point; in addition, the multi-dimensional data such as visible light images and infrared thermal images collected by the unmanned aerial vehicle are usually analyzed independently, and are not deeply fused with the ground sensor data, resulting in low detection accuracy.

[0045] In addition, the existing positioning of leakage sources mostly relies on two-dimensional plane data, which cannot accurately restore the concentration gradient change of the leaked substance in three-dimensional space, resulting in increased risk of misjudgment. For the prediction of diffusion trend, the existing method is mostly based on static meteorological parameters, and does not combine real-time collected multi-source data for dynamic correction, so the prediction result deviates greatly from the actual diffusion track.

[0046] Therefore, the existing detection method has the problems of low detection accuracy and low detection efficiency of toxic and harmful substances.

[0047] In this context, to solve the problem of low detection accuracy and low detection efficiency of toxic and harmful substances in the related art, the present application provides a leakage re-inspection method, device, equipment and storage medium for toxic and harmful substances based on a UAV. By integrating a fixed monitoring network and a UAV dynamic re-inspection, multi-dimensional data fusion is realized, and three-dimensional leakage modeling is supported, so as to improve the detection accuracy and detection efficiency of toxic and harmful substances.

[0048] The following refers to the following Figure 1 and Figure 2 The leakage re-inspection method for toxic and harmful substances based on a UAV provided by the embodiments of the present application is described.

[0049] Figure 1 The method flowchart of the leakage re-inspection method for toxic and harmful substances based on a UAV provided by the embodiments of the present application, the subject executing the method can be an electronic device, or each device / module in the electronic device, such as an integrated circuit or a chip, which is not specifically limited by the embodiments of the present application.

[0050] Exemplarily, as shown in Figure 1 The leakage re-inspection method for toxic and harmful substances based on a UAV provided by the embodiments of the present application can include the following S101 to S104:

[0051] S101, divide the target area into a plurality of grids, and obtain initial gas concentration data of each grid in real time through a fixed ground sensor array.

[0052] In some embodiments, the grid density can be dynamically adjusted according to the plant equipment distribution and risk level based on the topological structure of a geographic information system (GIS). For example, a 5m×5m high-density grid is used in the tank area, and a 10m×10m standard grid is used in the open area.

[0053] Exemplarily, taking a certain chemical park as an example, a 100m×100m tank area can be divided into 20×20 5m×5m grids, and a sensor node is deployed at the center of each grid. The fixed ground sensor array uses a long-range radio (LoRa) communication protocol to upload the chlorine concentration data of each grid to the central control system at a frequency of 1Hz.

[0054] In the embodiments of the present application, the fixed ground sensor array includes at least two groups of sensors.

[0055] Optionally, the at least two groups of sensors are diagonally distributed downwind according to the dominant wind direction of the target area.

[0056] For example, Figure 2 As shown in the figure, the target area is area A, and at least two sensor groups include sensor X and sensor Y. If the dominant wind direction in area A is from left to right, then the downwind direction is the right side of area A. In this case, sensor X can be deployed at point P, the upper right corner of area A, and sensor Y can be deployed at point Q, the lower right corner of area A.

[0057] Optionally, the distance between the at least two groups of sensors is determined according to the type of the toxic and hazardous substances.

[0058] In some embodiments, the spacing can be dynamically adjusted according to the diffusion coefficient of the toxic and hazardous substances.

[0059] For example, the distance L can be calculated using formula (1).

[0060] Formula (1)

[0061] Where k is the correction coefficient; t is the monitoring period; and D is the diffusion coefficient.

[0062] Specifically, if the toxic and hazardous substance is ammonia, the diffusion coefficient D=0.28m² / s. If k=1.2 and the monitoring period t=60s, then the sensor spacing If chlorine is detected at the same time, the diffusion coefficient D=0.15m² / s, and the corresponding spacing is adjusted to .

[0063] In this way, the present application arranges sensors diagonally according to the dominant wind direction, and adjusts the spacing according to the material type to reduce missed detections caused by environmental interference and improve the accuracy of abnormal grid determination.

[0064] Optionally, in the embodiment of the present application, the sensor type can be matched according to gas characteristics.

[0065] In some embodiments, when the toxic and hazardous substances include at least ammonia and chlorine, the fixed ground-based sensor array includes an electrochemical gas sensor (EGS) and an infrared spectroscopy sensor (IRS). The electrochemical sensor is used to detect ammonia concentration at a first sampling frequency, and the infrared spectroscopy sensor is used to detect chlorine concentration at a second sampling frequency.

[0066] In the embodiment of the present application, the first sampling frequency and the second sampling frequency may be equal or unequal.

[0067] Optionally, both the first sampling frequency and the second sampling frequency can be flexibly adjusted according to environmental parameters.

[0068] Exemplarily, the first sampling frequency can be adjusted according to the wind speed. If the initial wind speed is 2 m / s, the default sampling frequency is 1 Hz, and when the wind speed becomes 5 m / s, the sampling frequency can be increased (for example, adjusted to 4 Hz).

[0069] Exemplarily, the second sampling frequency can be adjusted according to the temperature. If the initial temperature is 25℃, the default sampling frequency is 0.5 Hz, and when the temperature rises to 30℃, the sampling frequency can be increased (for example, adjusted to 1 Hz).

[0070] In this way, the application customizes the sensor type and sampling frequency according to the characteristics of ammonia and chlorine, ensures fast response to different leakage scenarios, and improves the sensitivity of concentration detection.

[0071] S102, the grid with initial gas concentration data greater than the preset concentration is determined as an abnormal grid, and a re-inspection flight path is dynamically generated according to the UAV parking position, the abnormal grid position and the environmental parameters.

[0072] In some embodiments, the preset concentration can be set according to the lower explosive limit (LEL) of the gas, for example, the first alarm threshold of ammonia is 50% of the LEL (corresponding to 1.5% volume concentration).

[0073] Exemplarily, taking the preset concentration of 2% as an example. If it is monitored that the ammonia concentration of grid A is 2.3%, it is judged that the grid A is an abnormal grid.

[0074] In the embodiments of the application, the environmental parameters include real-time wind direction data, obstacle distribution information and communication signal strength.

[0075] Exemplarily, the real-time wind direction data can be obtained by a three-dimensional ultrasonic anemometer, the obstacle distribution information can be obtained by constructing a three-dimensional map (such as a building information model (BIM)) through laser radar point cloud data, and the communication signal strength can be obtained by querying the base station signal quality index.

[0076] In some embodiments, an initial obstacle avoidance path can be generated based on the UAV parking position, the abnormal grid position and the obstacle distribution information.

[0077] Exemplarily, the A* algorithm (such as A-star algorithm) known in the field of path planning can be used, taking the UAV parking position as the starting point and the center point of the abnormal grid as the end point, taking the Euclidean distance as the heuristic function, and avoiding the obstacles (such as buildings and large equipment) marked in the BIM model to obtain the initial obstacle avoidance path.

[0078] Specifically, taking the three-dimensional coordinates of the UAV parking position as (100, 200, 5) and the center three-dimensional coordinates of the abnormal grid position as (300, 400, 0) as examples. If the three-dimensional coordinates of the obstacle device distribution area are (200, 300, 0)-(250, 350, 10), the three-dimensional coordinates of the initial obstacle avoidance path can be generated as (100, 200, 5)→(150, 300, 15)→(280, 380, 10)→(300, 400, 0) to bypass the obstacle device.

[0079] Further, the initial obstacle avoidance path is adjusted in combination with real-time wind direction data and communication signal strength to obtain a re-inspection flight route.

[0080] The re-inspection flight route is a continuous non-collision route between the UAV parking position and the abnormal grid position with the strongest communication signal strength and covering the upwind area of the leakage source.

[0081] For example, a wind direction weight factor can be introduced to correct the path direction of the initial obstacle avoidance path so that the UAV covers the upwind area of the leakage source; at the same time, the communication signal strength of the initial obstacle avoidance path is measured, and the path corresponding to the communication signal strength less than the preset strength in the initial obstacle avoidance path is adjusted.

[0082] Specifically, taking the preset strength as -70 dBm as an example. If the wind direction data indicates that the wind direction is from south to north, in order to make the UAV cover the upwind area of the leakage source, the initial obstacle avoidance path can be offset by 10° to the south; at this time, if the communication signal strength of part of the path (such as 50 meters) in the offset initial obstacle avoidance path is -80 dBm (lower than the preset strength -70 dBm), the part of the path is offset by 10° to the base station direction, so that the signal strength is increased to -65 dBm (higher than the preset strength -70 dBm).

[0083] In this way, the path is dynamically optimized by fusing wind direction, obstacles and signal strength, which improves the coverage efficiency of the UAV on the upwind of the leakage source, reduces the risk of communication interruption, and enhances the re-inspection accuracy.

[0084] S103, control the UAV to fly to the abnormal grid according to the re-inspection flight path and synchronously collect multi-dimensional data of the abnormal grid.

[0085] The multi-dimensional data includes visible light images, infrared thermal imaging data, gas component spectrum, and spatial coordinate data.

[0086] For example, the visible light image is used to identify the surface morphology of the leaked liquid in the abnormal grid; the infrared thermal imaging data is used to detect the temperature abnormal area in the abnormal grid; the gas spectrum analysis component is used to determine the type of the leaked liquid in the area; and the spatial coordinate data is used to construct a three-dimensional leakage distribution model.

[0087] Optionally, the unmanned aerial vehicle in the embodiments of the present application can be a plurality of unmanned aerial vehicles, which can form an unmanned aerial vehicle array.

[0088] Optionally, in the case of a plurality of unmanned aerial vehicles, the multi-dimensional data of the abnormal grid can be collected by the following method.

[0089] In some embodiments, for visible light images and infrared thermal imaging data, the abnormal grid can be photographed by a dual-spectrum camera carried by the unmanned aerial vehicle to obtain visible light images and infrared thermal imaging data.

[0090] For example, the visual features and temperature features of the abnormal grid can be synchronously obtained by using dual-spectrum imaging technology. The dual-spectrum camera can adopt coaxial optical design of visible light and infrared thermal imaging, and pixel-level alignment can be achieved by image registration algorithm.

[0091] Further, under low illumination conditions, the visible light camera can automatically switch to a starlight-level sensor, and the infrared thermal imaging module can enable noise equivalent temperature difference.

[0092] In some embodiments, for gas component spectrum, the gas components and concentration gradient in the abnormal grid can be detected by a gas mass spectrometer carried by the unmanned aerial vehicle to generate a gas component spectrum.

[0093] For example, quantitative and qualitative detection of gas components can be achieved by mass spectrometry technology. The mass spectrometer adopts a quadrupole mass analyzer, and different leaking gases (such as ammonia (molecular weight 17) and chlorine (molecular weight 71)) are monitored by ion mobility spectrometry (IMS) pre-separation technology.

[0094] In some embodiments, for spatial coordinate data, a plurality of unmanned aerial vehicles can be controlled to carry laser radars to scan the abnormal grid from different angles to generate spatial coordinate data.

[0095] For example, the spatial coordinates of the leaking area in the abnormal grid can be obtained by multi-view three-dimensional reconstruction. Specifically, four unmanned aerial vehicles can form an orthogonal array, each carrying a 16-line laser radar (range finding accuracy ±2 cm). Point cloud registration is performed by iterative closest point (ICP) algorithm to construct a three-dimensional model with a spatial resolution of 5 mm, and the spatial coordinates of the leakage source are obtained.

[0096] In this way, the present application collects spectral, mass spectrometric and three-dimensional coordinate data by division, realizes multi-modal data complementation, and improves data collection integrity and timeliness.

[0097] S104, fuse the initial gas concentration data of the abnormal grid and the multi-dimensional data, construct a three-dimensional dynamic concentration field model, and output the leakage source position information and diffusion trend.

[0098] Optionally, before constructing the three-dimensional dynamic concentration field model, the multi-dimensional data can be pre-processed to eliminate invalid data.

[0099] In some embodiments, the multi-dimensional data can be denoised, coordinate normalized and feature extracted to generate a standardized data set.

[0100] For example, the region edge features of the abnormal grid can be extracted from the visible light image; the temperature abnormal region in the abnormal grid can be identified from the infrared thermal imaging data; the gas component spectrum can be baseline corrected and feature wavelength extracted to obtain the gas component type; the spatial coordinate data can be normalized in the coordinate system to obtain unified coordinate data.

[0101] Specifically, the U-Net++ neural network (segmentation accuracy of 92.3%) can be used for semantic segmentation of the visible light image to extract the edge of the leakage region; the threshold segmentation method can be used to identify the low temperature region (such as a low temperature region less than -15℃) in the infrared thermal imaging data; the wavelet transform can be used to calibrate the baseline of the gas component spectrum, and the machine learning algorithm can be used to identify the feature wavelength (such as the feature wavelength of 2.26μm for ammonia gas spectrum); the coordinate system normalization of the spatial coordinate data can be realized by the coordinate conversion matrix, and converted to the global coordinate system.

[0102] Further, the standardized data set can be generated according to the region edge features, temperature abnormal region, gas component type and unified coordinate data obtained after pre-processing.

[0103] In this way, the present application eliminates the heterogeneity of multi-source data through feature extraction and coordinate unification, enhances the consistency of model input, and reduces the influence of noise interference on positioning accuracy.

[0104] Optionally, after obtaining the pre-processed standardized data set, a spatio-temporal interpolation algorithm can be used to fuse the standardized data set with the initial gas concentration data to generate a dynamic concentration matrix; and then based on the gas diffusion equation, the dynamic concentration matrix is iteratively optimized to construct a three-dimensional dynamic concentration field model, and the leakage source position information and diffusion trend are output.

[0105] In some embodiments, since the initial gas concentration points collected by the fixed ground-based sensing array are sparse, and the standardized data set collected by the unmanned aerial vehicle is not continuous in time, the standardized data set obtained after being collected and processed by the unmanned aerial vehicle can be fused with the initial gas concentration collected by the fixed ground-based sensing array to output a dynamic concentration matrix covering the entire region.

[0106] Exemplarily, the Kriging interpolation method can be used for spatio-temporal data fusion, the discrete initial gas concentration data is interpolated into a continuous concentration field according to a standardized data set, and the concentration matrix is iteratively optimized based on a Gaussian dispersion model (GDM). The Gaussian dispersion model can be formula (two).

[0107] Formula (two)

[0108] wherein, is the gas concentration at the point; Q is the leakage source intensity, used to indicate the mass of the substance released from the leakage source per unit time; u is the average wind speed; is the horizontal diffusion coefficient, used to describe the diffusion degree of the gas in the horizontal direction (y-axis); is the vertical diffusion coefficient, used to describe the diffusion degree of the gas in the vertical direction (z-axis).

[0109] Specifically, taking an ammonia leakage scenario as an example. The initial gas concentration data (such as 3-point concentrations of 200 ppm, 280 ppm and 350 ppm) obtained by the fixed ground sensor array is fused with the 520 ppm data detected by the unmanned aerial vehicle through Kriging interpolation to generate a 1m x 1m x 1m concentration matrix; the concentration matrix is iteratively optimized based on the GDM, the input wind speed is 6 m / s, the stability level is D, and the three-dimensional concentration field is obtained by iterative optimization, the three-dimensional coordinates of the leakage source are predicted to be (305, 405, 2), and the concentration at the downwind 200 meters in the next 10 minutes will reach 100 ppm.

[0110] In the leakage re-inspection method of toxic and harmful substances based on the unmanned aerial vehicle provided in the embodiments of the present application, through the real-time monitoring of the grid partition and the fixed ground sensor array, full-coverage concentration scanning of the target area is realized, the monitoring blind area is reduced, the re-inspection flight path is dynamically generated in combination with the parking position of the unmanned aerial vehicle, the abnormal grid position and the environmental parameters, the high-risk area is preferentially covered by the unmanned aerial vehicle and the obstacles are avoided, and the path planning efficiency is improved; then the unmanned aerial vehicle is controlled to synchronously collect multi-dimensional data such as visible light images, infrared thermal images, gas component spectra and spatial coordinates, and the comprehensiveness of the leakage feature recognition is enhanced through the complementation of multiple data sources; finally, a three-dimensional dynamic concentration field model is constructed by fusing the ground sensor data and the multi-dimensional data of the unmanned aerial vehicle, the three-dimensional spatial distribution and diffusion trend of the leakage source are accurately restored, and thus the detection accuracy and efficiency of the toxic and harmful substances are improved.

[0111] Optionally, the electronic devices, physical components (such as sensors in the fixed ground sensor array and unmanned aerial vehicles) involved in the embodiments of the present application are all anti-static and anti-explosion devices, and no ignition sources such as static electricity, open flames and electric sparks are generated in the actual operation process.

[0112] Thus, the application uses anti-static and explosion-proof equipment to avoid ignition sources such as static electricity generated during operation, thereby preventing hazards such as fires and explosions, and improving production safety.

[0113] The above mainly introduces the scheme provided by the embodiments of the application from the perspective of the method. In order to realize the above functions, the toxic and harmful substance leakage re-inspection device based on a UAV or the electronic device comprises a hardware structure and / or a software module for executing each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is realized in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the application.

[0114] The embodiments of the application can divide the function modules of the toxic and harmful substance leakage re-inspection device based on a UAV or the electronic device according to the above method. For example, the toxic and harmful substance leakage re-inspection device based on a UAV or the electronic device can comprise various function modules corresponding to each function division, or two or more functions can be integrated in one processing module. The above integrated module can be realized in the form of hardware or software function module. It should be noted that the division of the modules in the embodiments of the application is illustrative, and is only a logical function division. Actual implementation can have another division method.

[0115] Figure 3 A structure diagram of a toxic and harmful substance leakage re-inspection device based on a UAV provided by the embodiments of the application. The toxic and harmful substance leakage re-inspection device based on a UAV 300 comprises an acquisition unit 301, a generation unit 302, a control unit 303, and a construction unit 304.

[0116] The acquisition unit 301 is configured to divide a target area into a plurality of grids and acquire initial gas concentration data of each grid in real time through a fixed ground sensing array; the generation unit 302 is configured to determine a grid with initial gas concentration data greater than a preset concentration as an abnormal grid, and dynamically generate a re-inspection flight path according to a UAV parking position, an abnormal grid position and environmental parameters; the control unit 303 is configured to control the UAV to fly to the abnormal grid according to the re-inspection flight path, and synchronously collect multi-dimensional data of the abnormal grid, the multi-dimensional data including visible light images, infrared thermal imaging data, gas component spectra and spatial coordinate data; and the construction unit 304 is configured to fuse the initial gas concentration data of the abnormal grid and the multi-dimensional data, construct a three-dimensional dynamic concentration field model, and output leakage source position information and diffusion trend.

[0117] In some embodiments, the environmental parameters include real-time wind direction data, obstacle distribution information and communication signal strength; and the generation unit 302 is specifically configured to generate an initial obstacle avoidance path based on the UAV parking position, the abnormal grid position and the obstacle distribution information, adjust the initial obstacle avoidance path in combination with the real-time wind direction data and the communication signal strength to obtain a re-inspection flight route, and the re-inspection flight route is a continuous non-collision route with the strongest communication signal strength between the UAV parking position and the abnormal grid position and covering the upwind area of the leakage source.

[0118] In some embodiments, the UAV is a plurality of UAVs; and the control unit 303 is specifically configured to control a dual-spectrum camera carried by the UAV to take pictures of the abnormal grid to obtain visible light images and infrared thermal imaging data, control a gas mass spectrometer carried by the UAV to detect gas components and concentration gradients in the abnormal grid to generate gas component spectra, and control a laser radar carried by the plurality of UAVs to scan the abnormal grid from different angles to generate spatial coordinate data.

[0119] In some embodiments, the construction unit 304 is specifically configured to denoise, normalize coordinates and extract features of the multi-dimensional data to generate a standardized data set, fuse the standardized data set with the initial gas concentration data by using a space-time interpolation algorithm to generate a dynamic concentration matrix, iteratively optimize the dynamic concentration matrix based on a gas diffusion equation to construct a three-dimensional dynamic concentration field model, and output leakage source position information and diffusion trend.

[0120] In some embodiments, the above-mentioned construction unit 304 is specifically configured to: extract region edge features of the abnormal grid from the visible light image; identify a temperature abnormal region in the abnormal grid from the infrared thermal imaging data; perform baseline correction and feature wavelength extraction on the gas component spectrum to obtain a gas component type; perform coordinate system normalization on the spatial coordinate data to obtain unified coordinate data; and generate a standardized data set according to the region edge features of the abnormal grid, the temperature abnormal region, the gas component type, and the unified coordinate data.

[0121] In some embodiments, the above-mentioned fixed ground-based sensing array includes at least two groups of sensors, which are diagonally distributed in the downwind direction of the dominant wind direction of the target area; and the spacing of the at least two groups of sensors is determined according to the type of the toxic and harmful substance.

[0122] In some embodiments, the above-mentioned toxic and harmful substance includes ammonia and chlorine; the fixed ground-based sensing array includes an electrochemical sensor and an infrared spectrum sensor; the electrochemical sensor is configured to detect the concentration of ammonia at a first sampling frequency; and the infrared spectrum sensor is configured to detect the concentration of chlorine at a second sampling frequency.

[0123] In the device for re-inspecting leakage of toxic and harmful substances based on a UAV provided in the embodiments, through grid partitioning and real-time monitoring of the fixed ground-based sensing array, full-coverage concentration scanning of the target area is realized, the monitoring blind area is reduced, and a re-inspection flight path is dynamically generated in combination with the UAV parking position, the abnormal grid position, and the environmental parameters, so as to ensure that the UAV preferentially covers the high-risk area and avoids obstacles, and the path planning efficiency is improved. Then, the UAV is controlled to synchronously collect multi-dimensional data such as visible light images, infrared thermal images, gas component spectra, and spatial coordinates, and the comprehensiveness of leakage feature identification is enhanced through multi-source data complementation. Finally, a three-dimensional dynamic concentration field model is constructed by fusing the ground-based sensing data and the multi-dimensional data of the UAV, and the three-dimensional spatial distribution and diffusion trend of the leakage source are accurately restored, so that the detection accuracy and efficiency of the toxic and harmful substances are improved.

[0124] As to the device in the above-mentioned embodiments, the specific manners in which various modules perform operations have been described in detail in the embodiments of the method, and will not be described here in detail.

[0125] Figure 4 A structural diagram of an electronic device provided in the embodiments is shown in FIG. 4. Figure 4 As shown in FIG. 4, the electronic device 400 includes but is not limited to a processor 401 and a memory 402.

[0126] The memory 402 is configured to store executable instructions of the processor 401. It can be understood that the processor 401 is configured to execute the instructions to implement the method for re-inspecting leakage of toxic and harmful substances based on a UAV in the above-mentioned embodiments.

[0127] It should be noted that those skilled in the art can understand that Figure 4 The electronic device structure shown in the above embodiments does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than those shown in the above embodiments, or combine some components, or arrange different components. Figure 4 The electronic device structure shown in the above embodiments does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than those shown in the above embodiments, or combine some components, or arrange different components.

[0128] The processor 401 is the control center of the electronic device, connects all parts of the electronic device through various interfaces and lines, executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 402 and calling data stored in the memory 402, and thus monitors the entire electronic device. The processor 401 can include one or more processing units. Alternatively, the processor 401 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application program, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 401.

[0129] The memory 402 can be used to store software programs and various data. The memory 402 can mainly include a program storage area and a data storage area, wherein the program storage area can store the operating system, the application programs required by at least one function module (such as the determination unit, the processing unit, etc.), etc. In addition, the memory 402 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0130] In the exemplary embodiments, a computer readable storage medium including instructions is also provided, for example, the memory 402 including instructions, which can be executed by the processor 401 of the electronic device 400 to implement the above-mentioned method for re-inspecting leakage of toxic and harmful substances based on a drone.

[0131] In actual implementation, Figure 3 The steps performed by the acquisition unit 301, the generation unit 302, the control unit 303, and the construction unit 304 in the above embodiments can be implemented by Figure 4 The processor 401 in the above embodiments can call the computer program stored in the memory 402 to implement. The specific execution process can refer to the description of the method part in the above embodiments, which will not be described here.

[0132] Optionally, the computer readable storage medium can be a non-transitory computer readable storage medium, for example, the non-transitory computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0133] In the example embodiment, the embodiment of the application further provides a computer program product comprising one or more instructions executable by the processor 401 of the electronic device to complete the method for re-inspecting leakage of toxic and harmful substances based on a UAV in the above-described embodiment.

[0134] It should be noted that the instructions in the above computer readable storage medium or the one or more instructions in the computer program product are executed by the processor of the electronic device to realize each process of the above method embodiment, and the same technical effects as the above method can be achieved. To avoid repetition, it will not be described here.

[0135] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of functional modules is taken as an example for illustration. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete the above-described full classification or part of the function.

[0136] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the above-described device embodiments are only illustrative, for example, the division of modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be through some interfaces, indirect coupling or communication connection between the units or devices, which can be electrical, mechanical or other forms.

[0137] The units described as separate components can or can not be physically separated, and the components displayed as units can be one physical unit or multiple physical units, that is, they can be located in one place or distributed to multiple different places. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment.

[0138] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0139] If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or say the part that contributes to the prior art or the whole classification part or part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium, including a plurality of instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute the whole classification part or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk and various program code storage media.

[0140] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for leak re-inspection of toxic and hazardous substances based on drones, characterized in that: The method comprises: Based on the topological structure of the Geographic Information System (GIS), the grid density is dynamically adjusted according to the distribution of plant equipment and risk level within the target area, the target area is divided into multiple grids, and the initial gas concentration data of each grid is acquired in real time through a fixed ground-based sensor array using the long-range radio LoRa communication protocol. The fixed ground-based sensor array includes at least two groups of sensors distributed diagonally downwind of the dominant wind direction of the target area. Grids with initial gas concentration data greater than a preset concentration are identified as abnormal grids, and a re-inspection flight path is dynamically generated based on the drone's parking location, the location of the abnormal grid, and environmental parameters, including real-time wind direction data, obstacle distribution information, and communication signal strength; Controlling the UAV to fly to the abnormal grid according to the re-inspection flight path, and synchronously collecting multi-dimensional data of the abnormal grid, wherein the multi-dimensional data includes visible light images, infrared thermal imaging data, gas composition spectra, and spatial coordinate data; Denoising, coordinate normalization, and feature extraction are performed on the multi-dimensional data to generate a standardized data set, and the standardized data set is fused with the initial gas concentration data using a spatiotemporal interpolation algorithm to generate a dynamic concentration matrix; The dynamic concentration matrix is ​​iteratively optimized based on the Gaussian diffusion equation to construct a three-dimensional dynamic concentration field model, and the leakage source location information and diffusion trend are output. The leakage source location information includes the three-dimensional coordinates of the leakage source. The wind speed is introduced into the iterative optimization process to dynamically correct the diffusion coefficient.

2. The method according to claim 1, characterized in that The re-inspection flight path is dynamically generated based on the drone's parking location, abnormal grid location, and environmental parameters, including: Generate an initial obstacle avoidance path based on the parking position of the UAV, the abnormal grid position and the obstacle distribution information; Combined with the real-time wind direction data and the communication signal strength, the initial obstacle avoidance path is adjusted to obtain the re-inspection flight route. The re-inspection flight route is a continuous non-collision route with the strongest communication signal strength between the UAV parking position and the abnormal grid position, and covering the upwind area of ​​the leakage source.

3. The method according to claim 2, characterized in that The drones are multiple drones; The collecting of multi-dimensional data of the abnormal grid includes: photographing the abnormal grid using a dual-spectrum camera carried by the drone to obtain the visible light image and the infrared thermal imaging data; Detecting the gas composition and concentration gradient in the abnormal grid by a gas mass spectrometer carried by the drone to generate the gas composition spectrum; The laser radars carried by the multiple drones are controlled to scan the abnormal grid from different angles to generate the spatial coordinate data.

4. The method according to claim 1, wherein The step of performing denoising, coordinate normalization, and feature extraction on the multi-dimensional data to generate a standardized data set includes: Extracting regional edge features of the abnormal grid from the visible light image; identifying temperature anomaly areas in the anomaly grid from the infrared thermal imaging data; Performing baseline correction and characteristic wavelength extraction on the gas component spectrum to obtain the gas component type; Normalizing the spatial coordinate data to obtain unified coordinate data; The standardized data set is generated according to the regional edge features, the temperature anomaly region, the gas component type and the unified coordinate data.

5. The method according to claim 1, wherein The spacing between sensors in the fixed ground-based sensing array is determined according to the type of the toxic and hazardous substances.

6. The method according to claim 1 or 5, characterized in that The toxic and hazardous substances include at least ammonia and chlorine; The fixed ground-based sensor array includes an electrochemical sensor and an infrared spectrum sensor; The electrochemical sensor is used to detect the ammonia concentration according to a first sampling frequency; The infrared spectrum sensor is used to detect the chlorine concentration according to a second sampling frequency.

7. A toxic and hazardous substance leakage re-inspection device based on drone, characterized in that: The device comprises: an acquisition unit configured to dynamically adjust the grid density based on the topological structure of a geographic information system (GIS) and the distribution of plant equipment and the risk level within the target area, divide the target area into a plurality of grids, and acquire initial gas concentration data for each grid in real time using a fixed ground-based sensor array employing the long-range radio LoRa communication protocol, wherein the fixed ground-based sensor array comprises at least two groups of sensors diagonally distributed downwind of the prevailing wind direction of the target area; a generation unit, configured to identify a grid where the initial gas concentration data is greater than a preset concentration as an abnormal grid, and dynamically generate a recheck flight path based on the parking location of the UAV, the location of the abnormal grid, and environmental parameters, wherein the environmental parameters include real-time wind direction data, obstacle distribution information, and communication signal strength; a control unit, configured to control the UAV to fly to the abnormal grid along the re-inspection flight path and simultaneously collect multi-dimensional data of the abnormal grid, wherein the multi-dimensional data includes visible light images, infrared thermal imaging data, gas composition spectra, and spatial coordinate data; Building blocks for: Denoising, coordinate normalization, and feature extraction are performed on the multi-dimensional data to generate a standardized data set, and the standardized data set is fused with the initial gas concentration data using a spatiotemporal interpolation algorithm to generate a dynamic concentration matrix; The dynamic concentration matrix is ​​iteratively optimized based on the Gaussian diffusion equation to construct a three-dimensional dynamic concentration field model, and the leakage source location information and diffusion trend are output. The leakage source location information includes the three-dimensional coordinates of the leakage source. The wind speed is introduced into the iterative optimization process to dynamically correct the diffusion coefficient.

8. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing instructions, characterized in that: When a computer executes the instruction, the computer performs the method according to any one of claims 1 to 6.

Images