Gas leak detection methods, devices, equipment and media

Abstract

The present invention provides a gas leakage detection method, device, equipment and medium, relating to the field of gas detection technology, wherein the method comprises: obtaining temperature data, acoustic wave data and image data of a target area; based on the temperature data, acoustic wave data and image data, using a gas leakage detection model to determine gas leakage characteristic data of the target area; wherein the gas leakage detection model is a bidirectional feature pyramid network model that receives temperature data, acoustic wave data and image data through an input layer, obtains joint characteristic data based on the temperature data, acoustic wave data and image data through a fusion layer, processes the joint characteristic data through a detection layer, obtains the location coordinates and category of the gas leakage as the gas leakage characteristic data of the target area, and outputs the gas leakage characteristic data of the target area through an output layer, so as to improve the detection accuracy and reliability of gas leakage by unmanned aerial vehicles.

Description

Technical Field

[0001] The present invention relates to the technical field of gas detection, and in particular to a gas leakage detection method, device, equipment and medium. Background Art

[0002] Traditional gas leak detection methods rely primarily on fixed sensors and drones. Fixed sensors have limited coverage, making it difficult to comprehensively monitor large chemical parks. Furthermore, in complex chemical park environments, such as those with equipment obstruction and environmental interference, the detection accuracy of fixed sensors can be severely affected. Drone detection can quickly reach designated areas, enabling comprehensive inspections of chemical parks. However, this method relies solely on a single gas sensor, and its detection capabilities in complex environments remain limited. For example, in the presence of multiple interfering gases, the gas sensor may misjudge. Detection accuracy is also significantly reduced when detecting at long distances. Summary of the Invention

[0003] In view of this, an object of the present invention is to provide a gas leak detection method, device, equipment and medium to improve the accuracy and reliability of gas leak detection by drones.

[0004] In a first aspect, the present invention provides a gas leakage detection method, comprising:

[0005] Acquire temperature data, acoustic wave data, and image data of the target area;

[0006] Based on temperature data, acoustic wave data and image data, a gas leakage detection model is used to determine the gas leakage feature data of the target area; wherein, the gas leakage detection model is a bidirectional feature pyramid network model that receives temperature data, acoustic wave data and image data through an input layer, obtains joint feature data based on the temperature data, acoustic wave data and image data through a fusion layer, and processes the joint feature data through a detection layer to obtain gas leakage feature data, wherein the gas leakage feature data includes the location coordinates of the gas leakage and the category of the gas leakage, and outputs the gas leakage feature data of the target area through an output layer.

[0007] Optionally, after acquiring the temperature data, the sound wave data, and the image data of the target area, the method further includes:

[0008] Preprocess the temperature data to obtain temperature anomaly data;

[0009] Preprocessing the acoustic wave data to obtain acoustic wave abnormality data;

[0010] The image data is preprocessed to obtain leakage port image data.

[0011] Optionally, the method further includes:

[0012] Based on the feature extraction model, determining temperature feature data corresponding to the temperature anomaly data, acoustic feature data corresponding to the acoustic anomaly data, and image feature data corresponding to the leakage port image data;

[0013] Based on the temperature feature data, the acoustic wave feature data and the image feature data, joint feature data corresponding to the temperature feature data, the acoustic wave feature data and the image feature data is determined.

[0014] Optionally, obtaining temperature data, acoustic wave data, and image data of the target area includes:

[0015] Obtain temperature data of the target area from the infrared sensor installed on the drone;

[0016] Acquiring acoustic sensors installed on drones to collect sound wave data of the target area;

[0017] Obtain image data of the target area collected by the image sensor installed on the UAV.

[0018] Optionally, the method further includes:

[0019] Acquire a training data set; wherein the training data set includes a plurality of training sample data; each training sample data includes temperature data, acoustic wave data, image data, and gas leakage characteristic standard data;

[0020] An iterative training operation is performed on the initial gas leak detection model based on the training data set until it is determined that an iterative training termination condition is met, and a gas leak detection model is obtained based on the weights and thresholds of the initial gas leak detection model updated during the last iterative training operation. The iterative training operation includes:

[0021] Select target training sample data from the training data set;

[0022] Inputting temperature data, acoustic wave data, and image data in the target training sample data into the initial gas leakage detection model, so that the initial gas leakage detection model receives the temperature data, acoustic wave data, and image data through the input layer, obtains joint feature data based on the temperature data, acoustic wave data, and image data through the fusion layer, processes the joint feature data through the detection layer to obtain gas leakage feature data, wherein the gas leakage feature data includes the location coordinates of the gas leakage and the category of the gas leakage, and outputs the gas leakage feature data of the target area through the output layer;

[0023] Based on the prediction error between the gas leakage feature data and the gas leakage feature standard data in the target training sample data, the weights and thresholds of the initial gas leakage detection model are updated.

[0024] Optionally, the method further includes:

[0025] During the iterative training operation on the initial gas leakage detection model, the weights and thresholds of the initial gas leakage detection model are optimized.

[0026] Optionally, the method further includes:

[0027] Based on the attention mechanism, the temperature feature data, sound wave feature data and image feature data are updated respectively.

[0028] In a second aspect, the present invention provides a gas leakage detection device, comprising:

[0029] a data acquisition unit, configured to acquire temperature data, acoustic wave data, and image data of a target area;

[0030] The gas detection unit is used to determine the gas leakage characteristic data of the target area based on the temperature data, the acoustic wave data and the image data using a gas leakage detection model; wherein the gas leakage detection model is a bidirectional feature pyramid network model that receives the temperature data, the acoustic wave data and the image data through an input layer, obtains joint characteristic data based on the temperature data, the acoustic wave data and the image data through a fusion layer, processes the joint characteristic data through a detection layer to obtain the gas leakage characteristic data, wherein the gas leakage characteristic data includes the location coordinates of the gas leakage and the category of the gas leakage, and outputs the gas leakage characteristic data of the target area through an output layer.

[0031] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned gas leakage detection method when executing the computer program.

[0032] In a fourth aspect, the present invention further provides a computer-readable storage medium storing computer instructions, which implement the above-mentioned gas leakage detection method when executed by a processor.

[0033] Embodiments of the present invention provide a gas leak detection method, apparatus, device, and medium, which obtain temperature data, acoustic wave data, and image data of a target area; and determine gas leak feature data of the target area using a gas leak detection model based on the temperature data, acoustic wave data, and image data. The gas leak detection model is a bidirectional feature pyramid network model that receives temperature data, acoustic wave data, and image data through an input layer, obtains joint feature data based on the temperature data, acoustic wave data, and image data through a fusion layer, processes the joint feature data through a detection layer, obtains the location coordinates and category of the gas leak as the gas leak feature data of the target area, and outputs the gas leak feature data of the target area through an output layer, thereby improving the accuracy and reliability of drone gas leak detection.

[0034] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 A flow chart of a gas leakage detection method provided by an embodiment of the present invention is shown;

[0037] Figure 2 A flow chart showing a method for training a gas leak detection model provided by an embodiment of the present invention is shown;

[0038] Figure 3 A schematic structural diagram of a gas leakage detection device provided by an embodiment of the present invention is shown;

[0039] Figure 4 A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0041] In order to facilitate those skilled in the art to better understand this application, the technical terms involved in this application are briefly introduced below.

[0042] Temperature characteristic data is data used to reflect the temperature characteristics of the target area. In this application, the temperature characteristic data of the target area is obtained by an infrared sensor installed on the drone.

[0043] Acoustic wave characteristic data is data used to reflect the acoustic wave characteristics of the target area. In this application, the acoustic wave characteristic data of the target area is obtained by an acoustic sensor installed on the drone.

[0044] Image feature data is data used to reflect the image features of the target area. In this application, the image feature data of the target area is obtained through a camera installed on the drone.

[0045] The gas leakage characteristic data is used to reflect the gas leakage characteristic data of the target area. In this application, the gas leakage characteristic data includes the location coordinate data of the gas leakage and the gas leakage category data.

[0046] The gas leak detection model is a bidirectional feature pyramid network (BiFPN) model that detects gas leaks in a target area by learning the mapping relationship between temperature feature data, acoustic feature data, image feature data, and gas leak feature data. In this application, the gas leak detection model includes an input layer, a fusion layer, a processing layer, and an output layer. The data of the input layer and the fusion layer are both at least one input layer. Each fusion layer includes a cross-modal attention mechanism and a weighted feature fusion process.

[0047] The mathematical relationship between the input layer and the fusion layer can be expressed as:

[0048]

[0049] Where, is the joint feature data, is the temperature characteristic data, is the acoustic wave characteristic data, is the image feature data, 、 and is the weight coefficient.

[0050] The term "and / or" used in this application describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0051] After introducing the technical terms involved in this application, the technical solutions provided by this application are described in detail.

[0052] Figure 1 A flow chart of a gas leakage detection method provided by an embodiment of the present invention. Figure 1 As shown, the method comprises at least the following steps:

[0053] Step 110: Acquire temperature data, sound wave data, and image data of the target area.

[0054] In the embodiment of the present application, when acquiring temperature data, sound wave data, and image data of the target area, the following methods may be used, including but not limited to:

[0055] Obtain temperature data of the target area from an infrared sensor installed on the drone; the infrared sensor is an infrared thermal imager, which is an uncooled vanadium oxide focal plane array and can measure temperatures in the range of -40°C to 150°C;

[0056] Acquiring acoustic wave data from the target area using an acoustic sensor installed on the drone. The acoustic sensor is an array of multiple microphones distributed in a circular pattern, capable of collecting sound waves with a frequency response range of 20 Hz to 20 kHz. The collected sound wave data is time series data.

[0057] The image data of the target area is collected by the image sensor installed on the drone, where the image sensor is a high-resolution camera with a pixel resolution of up to 48 million.

[0058] Furthermore, after obtaining the temperature data, acoustic wave data, and image data of the target area, the temperature data, acoustic wave data, and image data are preprocessed to obtain temperature anomaly data, acoustic wave anomaly data, and leakage port image data. This may be done in the following ways, including but not limited to:

[0059] When using infrared thermal imagers to capture temperature data in the target area, when harmful gas leaks, there is a temperature difference between the leaking area and the surrounding environment. After preprocessing the temperature data, including denoising and enhancement operations, the temperature anomaly data of the temperature anomaly area is extracted according to the set temperature threshold. If the temperature value of the temperature anomaly area is set , the average temperature of the surrounding normal area is , then the temperature anomaly data is:

[0060]

[0061] Gas leakage generates high-frequency sound waves. After the acoustic sensor captures the sound wave signal in the target area and filters and removes the interference of environmental noise, the short-time Fourier transform (STFT) is used to convert the time domain sound wave signal into a frequency domain signal to analyze the frequency characteristics of the leakage sound wave. If the frequency domain signal after STFT transformation is , the amplitude value corresponding to the leakage sound wave data is , the average amplitude of the background noise is , then the abnormal sound wave data is:

[0062]

[0063] The image data of the target area is captured by a high-resolution camera. After the target is detected and the location of the leak is located on the collected image data, the shape and texture features of the leak in the image data are extracted. In this application, the local binary pattern (LBP) algorithm can be used to extract the texture features of the leak in the image data. Let the LBP feature vector be After extracting the contour features of the leak port and calculating the perimeter P and area A of the leak port contour, the leak port image data is:

[0064]

[0065] Step 120: Based on the temperature data, the acoustic wave data, and the image data, a gas leakage detection model is used to determine the gas leakage characteristic data of the target area; wherein the gas leakage detection model is a bidirectional feature pyramid network model that receives the temperature data, the acoustic wave data, and the image data through an input layer, obtains joint characteristic data based on the temperature data, the acoustic wave data, and the image data through a fusion layer, processes the joint characteristic data through a detection layer to obtain gas leakage characteristic data, wherein the gas leakage characteristic data includes the location coordinates of the gas leakage and the category of the gas leakage, and outputs the gas leakage characteristic data of the target area through an output layer.

[0066] In an embodiment of the present application, the input data is temperature data, acoustic wave data, and image data; the output data is the location coordinates of a gas leak and the gas leak category. The gas leak detection model receives the temperature data, acoustic wave data, and image data through the input layer, and obtains joint feature data based on the temperature data, acoustic wave data, and image data through the fusion layer. The detection layer processes the joint feature data to obtain the location coordinates and category of the gas leak as the gas leak feature data of the target area, and then outputs the gas leak feature data of the target area through the output layer. In this way, the gas leak detection model based on the characteristics of the bidirectional feature pyramid network model detects the location coordinates and gas leak category of the target area based on the temperature data, acoustic wave data, and image data of the target area, thereby improving the accuracy and reliability of drone gas leak detection.

[0067] In order to meet the needs of multimodal feature fusion, in an embodiment of the present application, after the input layer receives the pre-processed temperature anomaly data, acoustic wave anomaly data and leakage port image data, and based on the feature extraction model, the temperature feature data corresponding to the temperature anomaly data, the acoustic wave feature data corresponding to the acoustic wave anomaly data and the image feature data corresponding to the leakage port image data are determined.

[0068] Specifically, based on the feature extraction model, when determining the temperature feature data corresponding to the temperature anomaly data, the acoustic feature data corresponding to the acoustic anomaly data, and the image feature data corresponding to the leakage port image data, the following methods may be used, but are not limited to:

[0069] For temperature anomaly data, a convolutional neural network (CNN) is used to extract the temperature feature data corresponding to the temperature anomaly data. Specifically, the input layer receives the temperature anomaly data, and extracts the temperature feature data from the temperature anomaly data based on the convolution operation through the hidden layer, and then outputs the temperature feature data through the output layer. Its mathematical expression is:

[0070]

[0071] in, is the temperature characteristic data, is the temperature anomaly data, Convolutional neural network operation for temperature anomaly data;

[0072] For abnormal sound wave data, the time domain feature extraction model based on the convolutional neural network is used to divide the collected abnormal sound wave data into frames. Each frame of data is used as input. After the convolution layer and pooling layer are used to extract the acoustic wave feature data from the abnormal sound wave data, the acoustic wave feature data is output through the output layer. Its mathematical expression is:

[0073]

[0074] Where, is the acoustic wave characteristic data, is the abnormal sound wave data, for convolutional neural network operations on sound wave data;

[0075] For leak image data, a pre-trained convolutional neural network (ResNet) model is used to extract the image feature data corresponding to the leak image data. Specifically, the input layer receives the leak image data, and the hidden layer extracts the leak data from the leak image data as image feature data. The image feature data is then output through the output layer. The mathematical expression is:

[0076]

[0077] Where, is the image feature data, is the leak image data, is the activation function.

[0078] In order to enhance the interaction between different modal features, in an embodiment of the present application, temperature feature data, sound wave feature data and image feature data are updated through a cross-modal attention mechanism.

[0079] Specifically, when updating temperature feature data, sound wave feature data, and image feature data through the cross-modal attention mechanism, the following methods may be used, including but not limited to:

[0080] Taking temperature feature data and image feature data as an example, the attention weight of temperature feature data relative to image feature data is calculated. The mathematical expression is:

[0081]

[0082] Where, is the attention weight, is the normalization function, is the temperature characteristic data that has not been updated. is the image feature data, is the feature dimension;

[0083] Based on the attention weight of the temperature feature data relative to the image feature data, the temperature feature data is weighted updated, and its mathematical expression is:

[0084]

[0085] Where, is the temperature feature data after updating based on the attention weight.

[0086] In an embodiment of the present application, multiple input layers in the gas leak detection model receive temperature anomaly data, acoustic wave anomaly data, and leakage port image data, and extract temperature feature data corresponding to the temperature anomaly data, acoustic wave feature data corresponding to the acoustic wave anomaly data, and image feature data corresponding to the leakage port image data through the input layer; the temperature feature data, acoustic wave feature data, and image feature data are updated based on the attention weights between the temperature feature data, acoustic wave feature data, and image feature data through the fusion layer, and weighted fusion processing is performed based on the updated temperature feature data, acoustic wave feature data, and image feature data to obtain joint feature data; and the gas leakage feature data is determined as the gas leakage feature data of the target area based on the mapping relationship between the joint feature data and the gas leakage category and location through the processing layer, and the gas leakage feature data is output through the output layer to improve the detection accuracy, real-timeness, and robustness of the drone for the target area.

[0087] The following is a detailed description of the training process of the gas leakage model provided in the embodiment of the present application. Figure 2 As shown, the gas leakage model training process provided in the embodiment of the present application is as follows:

[0088] Step 210: Acquire a training data set; wherein the training data set includes a plurality of training sample data; each training sample data includes temperature data, acoustic wave data, image data, and gas leakage characteristic standard data;

[0089] Step 220: Select target training sample data from the training data set;

[0090] Step 230: Input the temperature data, acoustic wave data, and image data in the target training sample data into the initial gas leak detection model, so that the initial gas leak detection model receives the temperature data, acoustic wave data, and image data through the input layer, obtains joint feature data based on the temperature data, acoustic wave data, and image data through the fusion layer, processes the joint feature data through the detection layer to obtain gas leak feature data, wherein the gas leak feature data includes the location coordinates of the gas leak and the category of the gas leak, and outputs the gas leak feature data of the target area through the output layer;

[0091] Step 240: updating the weights and thresholds of the initial gas leakage detection model based on the prediction error between the gas leakage feature data and the gas leakage feature standard data in the target training sample data;

[0092] Step 250, determine whether the iterative training termination condition is met; if so, execute step 260; if not, return to step 220; wherein, the iterative training termination condition is that the number of iterations is not less than the number threshold, or the prediction error is not higher than the error threshold.

[0093] Step 260 : Obtain a gas leakage detection model based on the weights and thresholds of the initial gas leakage detection model updated during the last iterative training operation.

[0094] In order to improve the accuracy of the gas leakage model, in the embodiment of the present application, during the iterative training of the initial gas leakage model, the weights of the initial gas leakage model may be optimized. Specifically, the gas leakage model optimization process provided in the embodiment of the present application is as follows:

[0095] Based on the attention mechanism, the temperature feature data, sound wave feature data and image feature data are updated respectively.

[0096] Specifically, when updating temperature feature data, sound wave feature data, and image feature data through the cross-modal attention mechanism, the following methods may be used, including but not limited to:

[0097] Taking temperature feature data and image feature data as an example, the attention weight of temperature feature data relative to image feature data is calculated. The mathematical expression is:

[0098]

[0099] Where, is the attention weight, is the normalization function, is the temperature characteristic data that has not been updated. is the image feature data, is the feature dimension;

[0100] Based on the attention weight of the temperature feature data relative to the image feature data, the temperature feature data is weighted updated, and its mathematical expression is:

[0101]

[0102] Where, is the temperature feature data after updating based on the attention weight.

[0103] Based on the above embodiments, the present invention provides a gas leakage detection device. Figure 3 As shown, the gas leakage detection device provided in the embodiment of the present application includes at least:

[0104] A data acquisition unit 310 is used to acquire temperature data, acoustic wave data, and image data of a target area;

[0105] The gas detection unit 320 is used to determine the gas leakage characteristic data of the target area based on the temperature data, the acoustic wave data and the image data using a gas leakage detection model; wherein the gas leakage detection model is a bidirectional feature pyramid network model that receives the temperature data, the acoustic wave data and the image data through an input layer, obtains joint characteristic data based on the temperature data, the acoustic wave data and the image data through a fusion layer, processes the joint characteristic data through a detection layer to obtain the gas leakage characteristic data, wherein the gas leakage characteristic data includes the location coordinates of the gas leakage and the category of the gas leakage, and outputs the gas leakage characteristic data of the target area through an output layer.

[0106] In an optional embodiment, the data acquisition unit 310 is further configured to:

[0107] Preprocess the temperature data to obtain temperature anomaly data;

[0108] Preprocessing the acoustic wave data to obtain acoustic wave abnormality data;

[0109] The image data is preprocessed to obtain leakage port image data.

[0110] In an optional embodiment, the gas detection unit 320 is further configured to:

[0111] Based on the feature extraction model, determining temperature feature data corresponding to the temperature anomaly data, acoustic feature data corresponding to the acoustic anomaly data, and image feature data corresponding to the leakage port image data;

[0112] Based on the temperature feature data, the acoustic wave feature data and the image feature data, joint feature data corresponding to the temperature feature data, the acoustic wave feature data and the image feature data is determined.

[0113] In an optional embodiment, the data acquisition unit 310 is further configured to:

[0114] Obtain temperature data of the target area from the infrared sensor installed on the drone;

[0115] Acquiring acoustic sensors installed on drones to collect sound wave data of the target area;

[0116] Obtain image data of the target area collected by the image sensor installed on the UAV.

[0117] In an optional embodiment, the model training unit 330 is configured to:

[0118] Acquire a training data set; wherein the training data set includes a plurality of training sample data; each training sample data includes temperature data, acoustic wave data, image data, and gas leakage characteristic standard data;

[0119] An iterative training operation is performed on the initial gas leak detection model based on the training data set until it is determined that an iterative training termination condition is met, and a gas leak detection model is obtained based on the weights and thresholds of the initial gas leak detection model updated during the last iterative training operation. The iterative training operation includes:

[0120] Select target training sample data from the training data set;

[0121] Inputting temperature data, acoustic wave data, and image data in the target training sample data into the initial gas leakage detection model, so that the initial gas leakage detection model receives the temperature data, acoustic wave data, and image data through the input layer, obtains joint feature data based on the temperature data, acoustic wave data, and image data through the fusion layer, processes the joint feature data through the detection layer to obtain gas leakage feature data, wherein the gas leakage feature data includes the location coordinates of the gas leakage and the category of the gas leakage, and outputs the gas leakage feature data of the target area through the output layer;

[0122] Based on the prediction error between the gas leakage feature data and the gas leakage feature standard data in the target training sample data, the weights and thresholds of the initial gas leakage detection model are updated.

[0123] In an optional embodiment, the model training unit 330 is configured to:

[0124] During the iterative training operation on the initial gas leakage detection model, the weights and thresholds of the initial gas leakage detection model are optimized.

[0125] In an optional embodiment, the model training unit 330 is configured to:

[0126] Based on the attention mechanism, the temperature feature data, sound wave feature data and image feature data are updated respectively.

[0127] The device provided in the embodiment of the present application has the same implementation principle and technical effects as those in the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference can be made to the corresponding content in the aforementioned method embodiment.

[0128] The present application also provides a gas leak detection device, including a drone, and an infrared camera, an acoustic sensor and a camera arranged on the drone, wherein the infrared camera is used to collect temperature data of the target area. In the present application, the infrared camera is an infrared thermal imager, and the infrared thermal imager is an uncooled vanadium oxide focal plane array, which can measure a temperature range of -40°C to 150°C; the acoustic sensor is used to collect sound wave data of the target area. In the present application, the acoustic sensor is an array composed of multiple microphones, which are distributed in a circular shape and can collect sound wave data with a frequency response range of 20Hz to 20kHz. The collected sound wave data is time series data; the camera is used to collect image data of the target area. In the present application, the camera is a high-resolution camera with a pixel count of up to 48 million.

[0129] like Figure 4 As shown, an electronic device 400 provided in an embodiment of the present application includes: a processor 410, a memory 420 and a bus. The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device is running, the processor 410 and the memory 420 communicate through the bus, and the processor 410 executes the machine-readable instructions to perform the steps of the gas leak detection method as described above.

[0130] Specifically, the memory 420 and the processor 410 can be general-purpose memories and processors, which are not specifically limited here. When the processor 410 runs the computer program stored in the memory 420, the gas leakage detection method can be executed.

[0131] The processor 410 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 410 or by instructions in the form of software. The above-mentioned processor 410 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 420, and processor 410 reads information in memory 420 and, in conjunction with its hardware, completes the steps of the above method.

[0132] Corresponding to the above-mentioned gas leakage detection method, an embodiment of the present application also provides a computer-readable storage medium, which stores machine-executable instructions. When the computer-executable instructions are called and executed by the processor, the computer-executable instructions prompt the processor to execute the steps of the above-mentioned gas leakage detection method.

[0133] The gas leakage detection device provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present application has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brief description, any part not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0134] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0135] For another example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of code, and a part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0136] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0137] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0138] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0139] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0140] Finally, it should be noted that the above embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. However, these modifications, changes, or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application. They should all be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A gas leak detection method, characterized in that: include: Acquiring temperature data, acoustic wave data, and image data of a target area, and preprocessing the temperature data to obtain temperature anomaly data; The acoustic wave data is preprocessed to obtain acoustic wave abnormality data; the image data is preprocessed to obtain leakage port image data; wherein the temperature abnormality data for Where, To set the temperature value of the abnormal temperature area, is the average temperature of the normal area; the abnormal sound wave data for: Where, is the amplitude value corresponding to the leakage acoustic wave data, is the average amplitude value of the background noise; the leakage port image data for: Where, is the LBP feature vector, P is the perimeter of the leakage port contour, and A is the area of the leakage port contour; Based on the temperature data, the acoustic wave data, and the image data, a gas leakage detection model is used to determine the gas leakage characteristic data of the target area; wherein the gas leakage detection model is a bidirectional feature pyramid network model that receives the temperature data, the acoustic wave data, and the image data through an input layer, obtains joint feature data based on the temperature data, the acoustic wave data, and the image data through a fusion layer, processes the joint feature data through a detection layer to obtain gas leakage characteristic data, wherein the gas leakage characteristic data includes the location coordinates of the gas leakage and the category of the gas leakage, and outputs the gas leakage characteristic data of the target area through an output layer; wherein , obtaining joint feature data based on the temperature data, the acoustic wave data, and the image data through a fusion layer, including: determining, based on a feature extraction model, the temperature feature data corresponding to the temperature anomaly data, the acoustic wave feature data corresponding to the acoustic wave anomaly data, and the image feature data corresponding to the leakage port image data; determining, based on the temperature feature data, the acoustic wave feature data, and the image feature data, the joint feature data corresponding to the temperature feature data, the acoustic wave feature data, and the image feature data; the fusion layer includes a cross-modal attention mechanism and a weighted feature fusion process, wherein the mathematical expression relationship between the input layer and the fusion layer can be expressed as: , where is the joint feature data, is the temperature characteristic data, is the acoustic wave characteristic data, is the image feature data, 、 and is the weight coefficient.

2. The gas leakage detection method according to claim 1, characterized in that: Acquiring temperature data, acoustic wave data, and image data of the target area includes: Obtaining temperature data of the target area from an infrared sensor installed on the drone; Acquiring acoustic wave data of the target area from an acoustic sensor installed on a drone; The image data of the target area is collected by an image sensor installed on the drone.

3. The gas leakage detection method according to any one of claims 1 to 2, characterized in that: Also includes: Acquire a training data set; wherein the training data set includes a plurality of training sample data; each of the training sample data includes temperature data, acoustic wave data, image data, and gas leakage characteristic standard data; Based on the training data set, an iterative training operation is performed on the initial gas leak detection model until it is determined that an iterative training termination condition is satisfied, and the gas leak detection model is obtained based on the weights and thresholds of the initial gas leak detection model updated during the last execution of the iterative training operation; wherein the iterative training operation includes: Selecting target training sample data from the training data set; Inputting the temperature data, the acoustic wave data, and the image data in the target training sample data into the initial gas leakage detection model, so that the initial gas leakage detection model receives the temperature data, the acoustic wave data, and the image data through an input layer, obtains joint feature data based on the temperature data, the acoustic wave data, and the image data through a fusion layer, processes the joint feature data through a detection layer to obtain gas leakage feature data, wherein the gas leakage feature data includes the location coordinates of the gas leakage and the category of the gas leakage, and outputs the gas leakage feature data of the target area through an output layer; Based on the prediction error between the gas leakage feature data and the gas leakage feature standard data in the target training sample data, the weights and thresholds of the initial gas leakage detection model are updated.

4. The gas leakage detection method according to claim 3, characterized in that: Also includes: During the iterative training operation on the initial gas leakage detection model, the weights and thresholds of the initial gas leakage detection model are optimized.

5. The gas leakage detection method according to claim 3, wherein: Also includes: Based on the attention mechanism, the temperature feature data, sound wave feature data and image feature data are updated respectively.

6. A gas leak detection device, characterized in that: include: a data acquisition unit, configured to acquire temperature data, acoustic wave data, and image data of a target area and pre-process the temperature data to obtain temperature anomaly data; The acoustic wave data is preprocessed to obtain acoustic wave abnormality data; the image data is preprocessed to obtain leakage port image data; wherein the temperature abnormality data for Where, To set the temperature value of the abnormal temperature area, is the average temperature of the normal area; the abnormal sound wave data for: Where, is the amplitude value corresponding to the leakage acoustic wave data, is the average amplitude value of the background noise; the leakage port image data for: Where, is the LBP feature vector, P is the perimeter of the leakage port contour, and A is the area of the leakage port contour; A gas detection unit is configured to determine the gas leakage characteristic data of the target area using a gas leakage detection model based on the temperature data, the sound wave data, and the image data; wherein the gas leakage detection model is a bidirectional feature pyramid network model that receives the temperature data, the sound wave data, and the image data through an input layer, obtains joint feature data based on the temperature data, the sound wave data, and the image data through a fusion layer, and processes the joint feature data through a detection layer to obtain gas leakage characteristic data, wherein the gas leakage characteristic data includes the location coordinates of the gas leakage and the category of the gas leakage, and outputs the gas leakage characteristic data of the target area through an output layer. Type; wherein, the joint feature data is obtained based on the temperature data, the acoustic wave data and the image data through the fusion layer, including: determining the temperature feature data corresponding to the temperature abnormality data, the acoustic wave feature data corresponding to the acoustic wave abnormality data and the image feature data corresponding to the leakage port image data based on the feature extraction model; based on the temperature feature data, the acoustic wave feature data and the image feature data, determining the joint feature data corresponding to the temperature feature data, the acoustic wave feature data and the image feature data; the fusion layer includes a cross-modal attention mechanism and a weighted feature fusion process, wherein the mathematical expression relationship between the input layer and the fusion layer can be expressed as: , where is the joint feature data, is the temperature characteristic data, is the acoustic wave characteristic data, is the image feature data, 、 and is the weight coefficient.

7. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the gas leakage detection method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the gas leakage detection method according to any one of claims 1 to 5 is implemented.

Images