Colored Gas Detection Method, Device, Electronic Device, and Readable Storage Medium

Abstract

The present application provides a method, apparatus, electronic device and readable storage medium for detecting a colored gas. Among them, the method includes: obtaining target image data of a region to be detected; extracting features from the target image data to determine target image features of the target image data; predicting whether the target region contains a target gas according to the image features to obtain a prediction result. It can improve the applicable scenarios of gas detection and also improve the accuracy of gas detection.

Description

Technical Field

[0001] This application relates to the technical field of air detection. Specifically, it relates to a method, device, electronic device, and readable storage medium for detecting colored gases. Background Art

[0002] Currently, for the detection of harmful gas leakage, gas leakage is generally detected through sensors. Then, through wireless transmission, the gas-related signals obtained by the sensors are converted into electrical signals. If the gas leakage concentration exceeds the limit, the sensor can send an alarm signal to achieve early warning of abnormal leakage.

[0003] However, the defect of the above gas leakage detection method is that the alarm signal must be triggered when the gas reaches a certain concentration. For outdoor scenarios or some open places, the gas may disperse quickly, and the gas concentration is not easy to detect, resulting in the gas leakage detection method through sensors not being applicable to outdoor scenarios or some open places. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, electronic device, and readable storage medium for detecting colored gases to improve the problem of poor accuracy of existing outdoor gas detection.

[0005] In a first aspect, the present invention provides a method for detecting colored gases, including: obtaining target image data of a region to be detected; performing feature extraction on the target image data to determine target image features of the target image data; predicting whether the target region contains a target gas based on the image features to obtain a prediction result.

[0006] In an optional embodiment, the performing feature extraction on the target image data to determine target image features of the target image data includes: performing compression processing on the target image data to obtain first image features of the target image data; performing decompression processing on the first image features to obtain second image features of the target image data; obtaining the target image features of the target image data based on the first image features and the second image features.

[0007] In an optional embodiment, the performing compression processing on the target image data to obtain first image features of the target image data includes: compressing the target image data through convolution and downsampling to extract the first image features of the target image data.

[0008] In an optional embodiment, the performing decompression processing on the compressed image to obtain second image features of the target image data includes: performing feature decoding on the first image features through convolution and upsampling to obtain second image features.

[0009] In an alternative embodiment, the compression processing of the target image data to obtain the first image features of the target image data includes: performing N-level compression processing on the target image data through convolution and downsampling to extract N first image features extracted at each level of the target image data, where N is a positive integer greater than two; the decompression processing of the first image features to obtain the second image features of the target image data includes: performing N-level decompression processing on the first image features obtained by compression at each level to obtain N second image features of the target image data; the obtaining of the target image features of the target image data according to the first image features and the second image features includes: determining the target image features of the target image data according to the first image features obtained by the first-level compression and the second image features obtained by the N-level decompression.

[0010] In an alternative embodiment, the performing of N-level decompression processing on the first image features obtained by compression at each level to obtain N second image features of the target image data includes: performing decompression processing on the first image features obtained by the N-level compression to obtain the second image features of the first level; processing the first image features obtained by the (N - i)-level compression and the second image features of the (i + 1)-level to obtain the second image features of the (i + 2)-level, where i is a positive integer greater than or equal to 1 and less than or equal to N - 2.

[0011] In an alternative embodiment, the processing of the first image features obtained by the (N - i)-level compression and the second image features of the (i + 1)-level to obtain the second image features of the (i + 2)-level includes: performing size processing on the first image features obtained by the (N - i)-level compression and the second image features of the (i + 1)-level to obtain a first target feature and a second target feature; fusing the first target feature and the second target feature to obtain the second image features of the (i + 2)-level of the target image data.

[0012] In an alternative embodiment, the fusing of the first target feature and the second target feature to obtain the second image features of the (i + 2)-level of the target image data includes: performing convolution processing on the first target feature and the second target feature to obtain the second image features of the (i + 2)-level of the target image data.

[0013] In a second aspect, the present invention provides a gas detection device, including:

[0014] An acquisition module, configured to acquire target image data of a region to be detected;

[0015] An extraction module for extracting features from the target image data to determine the target image features of the target image data;

[0016] A prediction module for predicting whether the target region contains a target gas based on the image features to obtain a prediction result.

[0017] In a third aspect, the present invention provides an electronic device, including: a processor and a memory, where the memory stores machine-readable instructions executable by the processor. When the electronic device runs, when the machine-readable instructions are executed by the processor, the steps of any of the foregoing embodiments are executed.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of any of the foregoing embodiments are executed.

[0019] The embodiments of the present application at least include the following beneficial effects: By detecting the gas that may exist in the area to be detected through image processing, it is possible to detect the gas without using a sensor and without paying attention to the gas concentration in the area to be detected. It can improve the detection of colored gases more accurately in some situations where gas concentration cannot be aggregated, such as outdoors. Description of the Drawings

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative efforts.

[0021] Figure 1 It is a schematic diagram of the detection principle of detecting gas by a detector in the prior art;

[0022] Figure 2 It is a block diagram of the electronic device provided by the embodiment of the present application;

[0023] Figure 3 It is a flowchart of the colored gas detection method provided by the embodiment of the present application;

[0024] Figure 4 It is a network diagram used in the colored gas detection method provided by the embodiment of the present application;

[0025] Figure 5 It is a schematic flowchart of the multi-scale adaptive detail feature fusion module used in the network of the colored gas detection method provided by the embodiment of the present application;

[0026] Figure 6 It is a schematic diagram of the functional modules of the colored gas detection device provided by the embodiment of the present application. Specific implementation manners

[0027] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application.

[0028] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, the terms "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0029] In recent years, there have been many major safety accidents caused by chemical gas leaks in chemical industrial parks. Such gas leakage incidents can lead to huge losses in property safety and pose a great threat to people's lives. In the future, with the development of the economic society, the monitoring and early warning and emergency response of chemical industrial parks will also become the focus of attention of chemical industrial parks everywhere. And the key to controlling dangerous chemicals and preventing problems before they occur. If dangerous chemicals can be detected and disposed of in a timely manner at the initial stage of leakage, dangerous accidents can be avoided.

[0030] Currently, for the suggestions on chemical gas leakage in chemical industrial parks, generally, sensors are used to detect gas leakage. For example, by introducing wireless transmission, the on-site gas concentration signal obtained by the sensor is converted into an electrical signal. The corresponding signal is subjected to analog-to-digital conversion through the single-chip microcomputer structure inside the sensor. After the value is quantified and encoded, a gas leakage signal is obtained. The leakage concentration of toxic and harmful gases in the chemical industrial park can be monitored in real time. If the leakage concentration of toxic and harmful gases exceeds the limit, the sensor generates an alarm signal to realize the early warning of abnormal leakage. The defect of this technology is that the alarm signal must be triggered when the toxic and harmful gas reaches a certain concentration, and it is not suitable for outdoor scenarios. Based on this situation, for gas leakage detection in general open-air areas, a pan-tilt scanning laser combustible gas detector can be used. In a strong wind environment, a combination of a pan-tilt scanning laser combustible gas detector and / or an ultrasonic gas leakage detector can be used.

[0031] Among them, the pan-tilt scanning laser combustible detector is an instrument and equipment for real-time monitoring of gas concentration based on the principle of spectral absorption. The detection principle schematic diagram of the pan-tilt scanning laser combustible gas detector can be referred to Figure 1As shown in the figure, the single-chip microcomputer control circuit performs current modulation on the pan-tilt scanning laser combustible gas detector 1101 to control the pan-tilt scanning laser combustible gas detector 110 to emit laser of the required wavelength. After the laser passes through the gas area 120, it reaches the reflecting surface 130 (the reflecting surface can be a gas pipeline, ceiling, wall, floor, ground, etc.) and is reflected back to the pan-tilt scanning laser combustible gas detector 110. If there are toxic and harmful gases to be detected in the gas area 120 through which the laser of the pan-tilt scanning laser combustible gas detector 110 passes, the laser will interact with the toxic and harmful gases and be absorbed. Moreover, the higher the concentration of the toxic and harmful gases, the greater the absorption amount. The pan-tilt scanning laser combustible gas detector 110 will monitor the change in laser intensity and feedback it to the single-chip microcomputer control circuit for processing. Finally, the concentration result will be transmitted by the signal output circuit.

[0032] In order to overcome the problems that the above method for detecting gas leakage through detectors lacks universality and has a low recognition accuracy, there is currently a method for identifying ultrasonic signals of gas leakage based on a convolutional neural network. The convolutional neural network can extract the characteristics of gas leakage sound signals, but it is necessary to further simulate the actual pipeline leakage form and establish a multi-classification model.

[0033] The inventor has learned that the above gas leakage detection method based on sensors requires the gas to reach a certain concentration to trigger an alarm signal, which is not applicable to outdoor scenarios and has weak universality. And the method for identifying ultrasonic signals of gas leakage based on a convolutional neural network relies on sound signals, has poor stability, and low detection accuracy.

[0034] Based on the research of the above current situation, the present application provides a method for detecting colored gases. This method can be based on a visual image detection method, and can collect image data of the area to be detected through an image acquisition device, and realize pollutant emission monitoring and air quality monitoring through the image data.

[0035] To facilitate the understanding of this embodiment, first, the electronic device for executing the method for detecting colored gases disclosed in the embodiments of the present application will be introduced in detail.

[0036] As Figure 2 shown, it is a block diagram of the electronic device. The electronic device 200 may include a memory 211 and a processor 213. Those of ordinary skill in the art can understand that Figure 2 the structure shown is only schematic and does not limit the structure of the electronic device 200. For example, the electronic device 200 may further include more or fewer components than Figure 2 shown, or have a different configuration from Figure 2 shown.

[0037] The above-mentioned memory 211 and processor 213 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The above-mentioned processor 213 is used to execute the executable module stored in the memory.

[0038] Among them, the memory 211 can be, but is not limited to, a random access memory (Random Access Memory, referred to as RAM), a read-only memory (Read Only Memory, referred to as ROM), a programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), an erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), an electrically erasable programmable read-only memory (Electric Erasable Programmable Read-Only Memory, referred to as EEPROM), etc. Among them, the memory 211 is used to store a program. After receiving an execution instruction, the processor 213 executes the program. The method executed by the electronic device 200 defined by the process disclosed in any embodiment of the embodiments of the present application can be applied to the processor 213 or implemented by the processor 213.

[0039] The above-mentioned processor 213 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 213 can be a general-purpose processor, including a central processing unit (Central Processing Unit, referred to as CPU), a network processor (Network Processor, referred to as NP), etc.; it can also be a digital signal processor (digital signal processor, referred to as DSP), an application-specific integrated circuit

[0040] (Application Specific Integrated Circuit, referred to as ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0041] The electronic device 200 can be connected to one or more image acquisition devices, and the image acquisition devices can be installed in the area to be detected where it is necessary to detect whether the colored gas exceeds the standard. The area to be detected can be an indoor or outdoor environment in a chemical industrial park.

[0042] The electronic device 200 in this embodiment can be used to execute each step in the various methods provided by the embodiments of the present application. The implementation process of the colored gas detection method will be described in detail through several embodiments below.

[0043] Please refer to Figure 3 , which is a flowchart of the colored gas detection method provided by the embodiments of the present application. The method in this embodiment can be executed by the Figure 3 electronic device shown. The following will elaborate on the Figure 3 specific process shown in detail.

[0044] Step 310, obtain target image data of the area to be detected.

[0045] Optionally, the electronic device that executes the colored gas detection method of the embodiments of the present application can be communicatively connected to an image acquisition device that acquires image data of the area to be detected, and the electronic device can obtain the image data acquired by the image acquisition device.

[0046] Optionally, an acquisition unit can be integrated in the electronic device that executes the colored gas detection method of the embodiments of the present application, and the acquisition unit can acquire image data in the area to be detected.

[0047] The target image data can be a picture or a video.

[0048] Step 320, perform feature extraction on the target image data to determine the target image features of the target image data.

[0049] In this embodiment, a colored gas detection model obtained through neural network training can be used to perform feature extraction on the target image data. For example, the following algorithms can be used to perform feature extraction on the target image data: HOG (histogram of Oriented Gradient); SIFT (Scale-invariant features transform);

[0050] SURF (Speeded Up Robust Features); DOG (Difference of Gaussian); LBP (Local Binary Pattern), etc.

[0051] The colored gas detection model can also be a model determined based on the U-Net network. Exemplarily, a certain number of colored gas images can be used as training data to train the initial U-Net network to obtain the colored gas detection model. Among them, the colored gas images used as training data can include images of different color gases, images with different gas color depths, and images of colored gases in different backgrounds.

[0052] Step 330, predict whether the target region contains the target gas according to the image feature to obtain a prediction result.

[0053] Among them, the target gas is some gases with colors, such as yellow gas, gray gas, etc.

[0054] Exemplarily, the prediction result can be presented directly in the form of a prompt message. For example, the electronic device that executes the colored gas detection method includes a display unit, and the prediction result can be displayed through the display unit.

[0055] The prediction result can also be presented in the target image data. For example, the area identified as the target gas is framed in the target image data in the form of a prompt box.

[0056] In the above steps, the gas that may exist in the area to be detected can be detected by image processing. Without using a sensor and without paying attention to the gas concentration in the area to be detected, the detection of the gas can be achieved. It can improve the detection of colored gases more accurately in some situations where gas concentration cannot be aggregated, such as outdoors.

[0057] In order to make the extracted features better reflect the features of the target gas contained in the target image data, multi-class features can be extracted through downsampling and upsampling, and the multi-class features are fused to determine the target image feature of the target image data. Step 320 can include: Step 321 to Step 323.

[0058] Step 321, perform compression processing on the target image data to obtain the first image feature of the target image data.

[0059] Optionally, the target image data is compressed through convolution and downsampling to extract the first image feature of the target image data.

[0060] By means of downsampling, the size of the image feature can be reduced, and multiple first image features obtained based on downsampling can be obtained during the process of reducing the image feature.

[0061] Exemplarily, the target image data can be compressed at the Nth level through convolution and downsampling to extract N first image features extracted at each level of the target image data.

[0062] Where N is a positive integer greater than two. For example, the value of N can be 3, 4, 5, etc.

[0063] In an optional embodiment, the image features can be processed sequentially through convolution and activation functions, and then the image features can be downsampled through max pooling to obtain the first image features obtained by the first-level compression process.

[0064] Exemplarily, during the process of compressing the image features, a 3×3 convolutional kernel can be used to convolve the picture, and then the activation function ReLU outputs the feature channels. Finally, the image features are downsampled through max pooling with a pooling kernel size of 2×2.

[0065] Step 322, decompress the first image features to obtain the second image features of the target image data.

[0066] Optionally, the first image features are decoded through convolution and upsampling to obtain the second image features.

[0067] Through upsampling, the size of the image features can be increased, and multiple second image features obtained based on upsampling can be obtained during the process of reducing the size of the image features.

[0068] Exemplarily, the first image features obtained by compression at each level are decompressed at the Nth level to obtain N second image features of the target image data.

[0069] In an optional embodiment, the image features can be processed sequentially through convolution and activation functions, and then the image features can be upsampled through transposed convolution to obtain the second image features obtained by the first-level decompression process.

[0070] Exemplarily, during the process of compressing the image features, a 3×3 convolutional kernel can be used to convolve the picture, and then the activation function ReLU outputs the feature channels. Finally, the image features are upsampled through transposed convolution with a convolutional kernel size of 2×2.

[0071] The above-mentioned decompressing the first image features obtained by compression at each level at the Nth level to obtain N second image features of the target image data may include: decompressing the first image features obtained by the Nth-level compression to obtain the second image features of the first level; processing according to the first image features obtained by the (N-i)th-level compression and the second image features of the (i+1)th level to obtain the second image features of the (i+2)th level.

[0072] Wherein, i is a positive integer greater than or equal to 1 and less than or equal to N - 2.

[0073] Optionally, it is possible to process based on the first image feature obtained by the N - i - th level compression and the second image feature of the (i + 1) - th level to obtain the second image feature of the (i + 2) - th level.

[0074] When the size of the first image feature obtained by the N - i - th level compression is different from the size of the second image feature obtained by decompressing the (i + 1) - th level, it is possible to first adjust the image features of the two sizes and then perform fusion to obtain the second image feature. The above processing based on the first image feature obtained by the N - i - th level compression and the second image feature obtained by decompressing the (i + 1) - th level to obtain the second image feature of the (i + 2) - th level includes: performing size processing on the first image feature obtained by the N - i - th level compression and the second image feature obtained by decompressing the (i + 1) - th level to obtain a first target feature and a second target feature; fusing the first target feature and the second target feature to obtain the second image feature of the (i + 2) - th level of the target image data.

[0075] Wherein, the first target feature and the second target feature have the same size.

[0076] Optionally, the way of size processing can be to enlarge the image feature with a smaller size or reduce the image feature with a smaller size to adjust the sizes of the two image features to image features with the same size.

[0077] Optionally, it is also possible to select an intermediate size and adjust the two image features to image features of the intermediate size to obtain the above - mentioned first target feature and second target feature.

[0078] Step 323, obtain the target image feature of the target image data according to the first image feature and the second image feature.

[0079] Wherein, if the first image feature and the second image feature are image features of the same size, the two image features can be fused through a convolutional layer.

[0080] Wherein, if the first image feature and the second image feature are image features of different sizes, the two image features can be first adjusted to image features of the same size and then the two image features can be fused through a convolutional layer.

[0081] In this embodiment, if multi - level compression and multi - level decompression are performed, the target image feature of the target image data can be determined according to the first image feature obtained by the first - level compression and the second image feature obtained by the N - th level decompression.

[0082] When the sizes of the first image features obtained by the first-level compression and the second image features obtained by the N-level decompression are different, the image features of the two sizes can be adjusted first, and then fused to obtain the second image features. The target image features of the target image data obtained by processing the first image features obtained by the first-level compression and the second image features obtained by the N-level decompression can include: performing size processing on the first image features obtained by the first-level compression and the second image features obtained by the N-level decompression to obtain first adjustment features and second adjustment features; fusing the first adjustment features and the second adjustment features to obtain the target image features of the target image data.

[0083] Among them, the sizes of the first adjustment feature and the second adjustment feature are the same.

[0084] Optionally, the method of adjusting to obtain the first adjustment feature and the second adjustment feature can be the same as the method of adjusting to obtain the first target feature and the second target feature described above.

[0085] In this embodiment, U-Net can be used to extract features and perform classification and recognition on the target image data.

[0086] Optionally, when the sizes of any two different-sized image features need to be adjusted, the image feature with a relatively smaller size can be filled with zeros to the size of the image feature with a relatively larger size.

[0087] Optionally, when the sizes of any two different-sized image features need to be adjusted, the image feature with a relatively smaller size can be filled with zeros to the size of the image feature with a relatively larger size.

[0088] The following combines Figure 4 to introduce the process of feature extraction and prediction of the target image data in steps 320 and 330.

[0089] Steps 320 and 330 in the colored gas detection method provided by the embodiments of the present application can be implemented through a U-Net network. Inputting the target image data into the colored gas detection model trained based on the U-Net network, the output is the prediction result for the target image data, and in the prediction result, the area of the colored gas in the target image data can be framed by a specified box.

[0090] In Figure 4In the illustrated example, the networks used at each arrow can be as follows: The network processing flow used by arrow ① includes: After convolving the image features using a 3×3 convolutional kernel, the feature channels are output through the activation function ReLU; The network processing flow used by arrow ② includes: Cropping and replicating the image features during the left downsampling process; The network processing flow used by arrow ③ includes: Downsampling the image features through max pooling, with a pooling kernel size of 2×2; The network processing flow used by arrow ④ includes: Upsampling the image features through transposed convolution, and the convolutional kernel size can be 2×2; The network processing flow used by arrow ⑤ includes: Convolving the image features using a 1×1 convolutional kernel to output the prediction result.

[0091] In Figure 4 In the illustrated example, the colored gas detection model obtained by training the U-Net network has a total of four layers, and the picture is downsampled and upsampled four times respectively.

[0092] Starting from the leftmost side, the process of compressing the image features can be described as the following process:

[0093] Input the target image data, the size of which is 572×572×1. Then, it is convolved through 64 3×3 convolutional kernels, and after being processed by the activation function ReLU, 64 feature channels of 570×570×1 are obtained. Then, the 570×570×64 image features are convolved through 64 3×3 convolutional kernels again, and after passing through the activation function ReLU, 64 feature extraction results of 568×568×1 are obtained. Then, the first image feature of 568×568×64 of the first layer's processing result can be obtained, which can also be used as the first image feature obtained by the first-level compression.

[0094] Perform pooling with a 2×2 pooling kernel on the first layer's processing result to downsample the image features to half of the original size, 284×284×64; Then, through 128 3×3 convolutional kernels, and after being processed by the activation function ReLU, 128 feature channels of 284×284×1 can be obtained. Then, the 284×284×128 image features are passed through 128 3×3 convolutional kernels and processed by the activation function ReLU again, and 128 feature channels of 280×280×1 can be obtained. Then, the first image feature of 280×280×128 of the second layer's processing result can be obtained, which can also be used as the first image feature obtained by the second-level compression.

[0095] And so on, the processing result of the third layer is the first image feature of 136×136×256, which can also be used as the first image feature obtained by the third-level compression; The processing result of the fourth layer is the first image feature of 64×64×512, which can also be used as the first image feature obtained by the fourth-level compression.

[0096] Based on the image features of 64×64×512 from the processing result of the fourth layer, the network processing flow used by arrow ① can obtain image features of 28×28×1024, and the image features of 28×28×1024 can be used as the second image features of the first level. Starting from Figure 4 the bottom right corner of the instance shown, the compression processing stage of the image features can be described as the following process:

[0097] Starting from Figure 4 the bottom right corner of the instance shown, the image features of 28×28×1024 are deconvolved with 512 2×2 convolutional kernels to expand the image features to 56×56×512; then the processing result obtained from the fourth layer compression process, that is, the first image features with a size of 64×64×512, are fused to obtain image features of 56×56×1024, and the image features of 56×56×1024 can be used as the second image features of the second level;

[0098] Then, the image features of 56×56×1024 are convolved with 512 convolutional kernels, and after two convolutions, image features of 52×52×512 are obtained, and then upsampling is performed again, repeating the above process. Two convolutions are performed for each layer to extract features. Each time upsampling is performed, the image features are doubled, and the number of convolutional kernels is halved. Finally, the processing result after four upsamplings is image features of 388×388×64, that is, there are a total of 64 feature layers, and the feature size of each layer is an image feature of 388×388. In the last step, 2 1×1 convolutional kernels can be used to change 64 feature channels into 2, that is, the final image features of 388×388×2. Actually, this is a binary classification operation to divide the target image data into two categories: background and target gas.

[0099] Among them, in the compression process, the size of the image features is reduced through convolution and downsampling to extract some superficial first image features. Among them, this compression process can be called Encoder.

[0100] The right part is the upsampling process of decompression, which obtains some deep second image features through convolution and upsampling. This decompression process can be called Decoder.

[0101] In Figure 4 the instance shown, a colored gas detection model is formed after four layers of downsampling and four layers of upsampling respectively. According to different actual scenarios, this colored gas detection model can also include Figure 4 more or fewer layer structures than the instance shown.

[0102] Among them, the number of convolution kernels in each convolution layer in the above-mentioned colored gas detection model is only exemplary. In actual situations, the size of the convolution kernels used in each layer of the network in the model can be selected according to requirements.

[0103] Among them, Figure 4 The network processing flow used by the arrow ③ shown can be implemented through a multi-scale adaptive detail feature fusion module. The first image feature obtained by compression can be fused with the second image feature, and the deep and shallow features can be refined to the information contained in the image features.

[0104] Next, in combination with Figure 5 , the multi-scale adaptive detail feature fusion module will be introduced.

[0105] In the embodiment of the present application, during the decompression process, the first image feature obtained during the compression process needs to be fused with the second image feature obtained by decompression, and the second image feature of the next-level decompression can be obtained. Through the multi-scale adaptive detail feature fusion module, the network bottom layer detail features can be better captured, helping the network improve the colored gas segmentation accuracy. Specifically, reference can be made to Figure 4 The schematic diagram of the image feature processing module shown. Let any image feature X ∈ RH×W, and X1 and X2 be image features of different sizes, where H and W respectively represent the height and width of the image feature, and R represents the infinite set of real numbers.

[0106] First, the first image feature X1 and the second image feature X2 are transformed to the same size through a unified size operation, and an image feature C can be obtained;

[0107] Then, convolution processing can be performed ( Figure 5 In the example shown, convolution processing is performed through a 1×1 convolution kernel), and a convolution result can be obtained;

[0108] The above convolution result is passed through two different deformable convolutions (Deform Conv) ( Figure 5 In the example shown, they are deformable convolutions of 3×3 convolution kernels and 5×5 convolution kernels respectively), and a first deformed feature and a second deformed feature can be obtained;

[0109] The first deformed feature and the second deformed feature are spliced to obtain a spliced feature;

[0110] The spliced feature is subjected to deformable convolution (Deform Conv) and normalization (Softmax) processing to obtain a normalized feature;

[0111] The normalized feature is segmented to obtain two image features, namely image feature S1 and image feature S2;

[0112] Concatenate the image feature S1 with the first image feature X1 to obtain a first concatenated feature;

[0113] Concatenate the image feature S2 with the second image feature X2 to obtain a second concatenated feature;

[0114] Then, obtain the second image feature Y of the next level from the first concatenated feature and the second concatenated feature.

[0115] Taking as an example the process of processing the first image feature obtained by compression at the N - i level and the second image feature obtained by decompression at the i + 1 level to obtain the second image feature at the i + 2 level, the input first image feature X1 can be the first image feature obtained by compression at the N - i level, the input second image feature X2 can be the second image feature obtained by decompression at the i + 1 level, and the output second image feature Y can be the second image feature at the i + 2 level.

[0116] The above-mentioned unified size operation can include: upsampling, downsampling, copy operation, etc. For example, if the size of an image feature is too large, the size of the image feature can be compressed by downsampling; if the size of an image feature is too small, the size of the image feature can be enlarged by upsampling; if the size of an image feature is the same as the required size, the image feature can be directly copied to retain its size.

[0117] Among them, Figure 5 The shown deformable convolution (Deform Conv) can be implemented by the following formula:

[0118]

[0119] Among them, R represents the infinite set of real numbers; w(p n ) represents the weight corresponding to the image feature; x(p0) represents the image feature of the first channel of the input image feature; x(p n ) represents the image feature of the n + 1 channel of the input image feature.

[0120] The colored gas detection method provided by the embodiments of the present application can be independent of gas concentration and sound signals and has good stability. Further, a U - Net network that can alleviate the loss of useful information is proposed. During the image processing process, a multi - size adaptive detail feature fusion module is designed to adaptively fuse multi - size features before the skip connection of the U - Net network, fully capture detail features of different sizes, reduce the loss of detail information in image data, and improve the accuracy of colored gas recognition.

[0121] Based on the same inventive concept, an embodiment of the present application further provides a colored gas detection device corresponding to the colored gas detection method. Since the principle of the device in the embodiment of the present application for solving problems is similar to that of the foregoing colored gas detection method embodiment, the implementation of the device in this embodiment can refer to the description in the embodiment of the above method, and the repeated parts will not be elaborated.

[0122] Please refer to Figure 6 , which is a schematic diagram of the functional modules of the colored gas detection device provided by the embodiment of the present application. Each module in the colored gas detection device in this embodiment is used to execute each step in the above method embodiment. The colored gas detection device includes: an acquisition module 410, an extraction module 420, and a prediction module 430; the content of each module is as follows:

[0123] The acquisition module 410 is used to acquire target image data of the area to be detected;

[0124] The extraction module 420 is used to perform feature extraction on the target image data to determine the target image features of the target image data;

[0125] The prediction module 430 is used to predict whether the target area contains the target gas according to the image features to obtain a prediction result.

[0126] In a possible implementation manner, the extraction module 420 includes: a first extraction unit, a second extraction unit, and a fusion unit:

[0127] The first extraction unit is used to perform compression processing on the target image data to obtain the first image features of the target image data;

[0128] The second extraction unit is used to perform decompression processing on the first image features to obtain the second image features of the target image data;

[0129] The fusion unit is used to obtain the target image features of the target image data according to the first image features and the second image features.

[0130] In a possible implementation manner, the above first extraction unit is used to compress the target image data through convolution and downsampling to extract the first image features of the target image data.

[0131] In a possible implementation manner, the above second extraction unit is used to perform feature decoding on the first image features through convolution and upsampling to obtain the second image features.

[0132] In a possible implementation manner, the above-mentioned first extraction unit is configured to perform N-level compression processing on the target image data through convolution and downsampling, so as to extract N first image features extracted at each level of the target image data, where N is a positive integer greater than two;

[0133] The above-mentioned second extraction unit is configured to perform N-level decompression processing on the first image features obtained by compression at each level to obtain N second image features of the target image data;

[0134] The fusion unit is configured to determine the target image feature of the target image data according to the first image feature obtained by the first-level compression and the second image feature obtained by the N-level decompression.

[0135] In a possible implementation manner, the above-mentioned second extraction unit is configured to perform decompression processing on the first image feature obtained by the N-level compression to obtain the second image feature of the first level; perform processing according to the first image feature obtained by the N-i level compression and the second image feature of the i+1 level to obtain the second image feature of the i+2 level, where i is a positive integer greater than or equal to 1 and less than or equal to N-2.

[0136] In a possible implementation manner, the above-mentioned second extraction unit is configured to perform size processing on the first image feature obtained by the N-i level compression and the second image feature of the i+1 level to obtain a first target feature and a second target feature; fuse the first target feature and the second target feature to obtain the second image feature of the i+2 level of the target image data.

[0137] In a possible implementation manner, the above-mentioned second extraction unit is further configured to perform convolution processing on the first target feature and the second target feature to obtain the second image feature of the i+2 level of the target image data.

[0138] In addition, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, it executes the steps of the colored gas detection method described in the above method embodiment.

[0139] The computer program product of the colored gas detection method provided by the embodiment of the present application includes a computer-readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the steps of the colored gas detection method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0140] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0141] In addition, the functional modules in each embodiment of the present application may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

[0142] When the above-mentioned functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs. It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, the elements defined by the statement "including..." do not exclude the existence of additional identical elements in the process, method, article or device comprising the said elements.

[0143] The foregoing are only the preferred embodiments of this application and are not used to limit this application. For those skilled in the art, this application can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included within the protection scope of this application. It should be noted that similar reference numerals and letters indicate similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0144] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by this application and should be covered by the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

1. A method for detecting a colored gas, characterized in that, Including: Obtain target image data of the area to be detected; Extract features from the target image data to determine the target image features of the target image data, including: performing N-level compression processing on the target image data to extract N first image features of the target image data; performing N-level decompression processing on the first image features obtained by each level of compression to obtain N second image features of the target image data; obtaining the target image features of the target image data according to the first-level first image features and the N-level second image features; Predict whether the area to be detected contains the target gas according to the target image features to obtain a prediction result; The performing N-level decompression processing on the first image features obtained by each level of compression to obtain N second image features of the target image data includes: performing decompression processing on the first image features obtained by the N-level compression to obtain the first-level second image features; processing according to the first image features obtained by the (N - i)-level compression and the second image features of the (i + 1)-level to obtain the second image features of the (i + 2)-level, where i is a positive integer greater than or equal to 1 and less than or equal to N - 2; The processing according to the first image features obtained by the (N - i)-level compression and the second image features of the (i + 1)-level to obtain the second image features of the (i + 2)-level includes: performing a unified size operation on the first image features obtained by the (N - i)-level compression and the second image features of the (i + 1)-level to transform the two image features to the same size to obtain an image feature; then performing convolution processing to obtain a convolution result; obtaining a first deformed feature and a second deformed feature by passing the convolution result through two different deformable convolutions; splicing the first deformed feature and the second deformed feature to obtain a spliced feature; performing deformable convolution and normalization processing on the spliced feature to obtain a normalized feature; segmenting the normalized feature to obtain two image features, namely a first segmented image feature and a second segmented image feature; splicing the first segmented image feature and the first image features obtained by the (N - i)-level compression to obtain a first spliced feature; splicing the second segmented image feature and the second image features of the (i + 1)-level to obtain a second spliced feature; obtaining the second image features of the (i + 2)-level of the target image data from the first spliced feature and the second spliced feature.

2. The method according to claim 1, wherein The performing N-level compression processing on the target image data to extract N first image features of the target image data includes: Performing N-level compression processing on the target image data through convolution and downsampling to extract N first image features of the target image data.

3. A colored gas detection device, characterized in that, Including: An acquisition module for acquiring target image data of the area to be detected; An extraction module for performing N-level compression processing on the target image data to extract N first image features of the target image data; Perform N-level decompression processing on the first image features obtained by each level of compression to obtain N second image features of the target image data; according to the first-level first image features and the N-level second image features, obtain the target image features of the target image data; A prediction module, configured to predict whether the region to be detected contains a target gas according to the target image features to obtain a prediction result; Wherein, the extraction module is configured to perform decompression processing on the first image features obtained by N-level compression to obtain the second image features of the first level; process the first image features obtained by N-i level compression and the second image features of the i+1 level to obtain the second image features of the i+2 level, where i is a positive integer greater than or equal to 1 and less than or equal to N-2; The extraction module is further configured to perform a unified size operation on the first image features obtained by N-i level compression and the second image features of the i+1 level to transform the two image features into the same size to obtain an image feature; then perform convolution processing to obtain a convolution result; pass the convolution result through two different deformable convolutions to obtain a first deformable feature and a second deformable feature; splice the first deformable feature and the second deformable feature to obtain a spliced feature; perform deformable convolution and normalization processing on the spliced feature to obtain a normalized feature; segment the normalized feature to obtain two image features, namely a first segmented image feature and a second segmented image feature; splice the first segmented image feature and the first image features obtained by N-i level compression to obtain a first spliced feature; splice the second segmented image feature and the second image features of the i+1 level to obtain a second spliced feature; obtain the second image features of the i+2 level of the target image data from the first spliced feature and the second spliced feature.

4. An electronic device, characterized in that, Including: A processor and a memory, the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the machine-readable instructions are executed by the processor to perform the steps of the method according to any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is run by the processor, it performs the steps of the method according to any one of claims 1 to 2.

Images