A gas spill detection method based on contrastive learning

Abstract

A gas overflow detection method based on contrast learning, comprising the following steps: S1, a contrast learning training stage, a model weight of picture feature extraction is trained, the model comprises three branches respectively for feature extraction, then three features are fused to obtain a final picture feature I o ; S2, the last model in step S1 is frozen, and two fully connected layers are connected thereafter, a nonlinear feature is obtained through a sigmoid activation function between the fully connected layers, finally a feature map of a specific dimension size of the feature is output, an output of positive and negative classification is obtained, representing the classification of whether there is gas overflow; S3, a preset gas leakage confidence threshold value, a picture and a thermal map acquisition device are fixed at a certain angle to fixed-point shoot the device to be acquired, an RGB three-channel picture corresponding to the device and a corresponding single-channel thermal map are acquired, the two fused pictures are input into the model obtained in step S2 for gas overflow classification prediction. Through the present application, gas leakage detection can be better recognized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of gas overflow detection, and particularly relates to a gas overflow detection method based on contrast learning. BACKGROUND

[0002] Safe use of gas is an important consideration for gas storage, in order to realize monitoring of gas overflow, not only sensors are used, but also the following technologies are used.

[0003] 1. (CN202011618527.2) Liquid and gas running, leaking and dripping identification method and system based on image recognition

[0004] The application provides a liquid and gas running, leaking and dripping identification method and system based on image recognition, which aims to improve the accuracy of identification of the running, leaking and dripping phenomenon. The method is as follows: video materials are collected by a camera device in a monitoring site; the video materials are decomposed to obtain a plurality of picture materials; the picture materials are labeled by an online labeling tool, that is, a liquid and gas running, leaking and dripping area in the picture is selected; an AI model is trained based on the above and is arranged in a monitoring device; when the liquid and gas running, leaking and dripping area exists, the picture materials obtained by decomposition are labeled and an alarm is given. The system comprises a plurality of monitoring devices arranged in a monitoring site, and the monitoring device comprises a camera module, a decomposition processing module, a detection module, a classification module and an alarm module. The patent only identifies according to target detection, a large number of training samples are needed in the identification process for feature extraction, and especially the collection of negative samples has great limitations, thereby causing difficulty in model training.

[0005] 2. (CN202111526834.2) Gas leakage monitoring method based on images

[0006] The application relates to gas leakage monitoring, in particular to an image-based gas leakage monitoring method, a gas concentration lookup table is generated based on a leakage gas infrared radiation model and gas temperature, infrared panoramic images and real-time gas temperature of each to-be-detected region are acquired, and gas concentration values corresponding to each pixel point are obtained in combination with the gas concentration lookup table, the infrared panoramic images of each to-be-detected region and the gas concentration values corresponding to each pixel point are sent to a control terminal, the control terminal obtains gas concentration spatial distribution data of the entire to-be-detected region, and a leakage source position is determined in the entire to-be-detected region; the technical scheme provided by the application can effectively overcome the defects that the detection result cannot be quantitatively processed in real time and the leakage source position cannot be quickly and accurately determined in the prior art. The application detects gas leakage based on only infrared pictures and temperature information, is often affected by environmental temperature, and can only detect only gases with a relatively large temperature difference from surrounding gases, so the application has great limitations.

[0007] 3. (CN112131993A) A gas leakage visual detection system and method based on image recognition technology

[0008] The application discloses a gas leakage visual detection system and method based on image recognition technology, which comprises a detection module provided with a gas-sensitive color-changing material, a to-be-detected gas storage device, an image acquisition device and a processor, a plurality of detection modules provided with the gas-sensitive color-changing material are installed on the surface of the to-be-detected gas storage device, the image acquisition device is used for acquiring images of the detection modules provided with the gas-sensitive color-changing material, a plurality of groups of the image acquisition device are in communication connection with the processor, and the processor is used for identifying color changes of the gas-sensitive color-changing material. The gas leakage visual detection system and method based on image recognition technology provided by the application utilize the discoloration characteristics of the gas-sensitive color-changing material, and through acquisition and identification of the detection modules provided with the gas-sensitive color-changing material, gas leakage detection can be quickly and accurately realized, and the application has the characteristics of high automation level, simple implementation and low cost. The application needs to detect based on the gas-sensitive material, and the type of the detected gas has certain limitations, and the application cannot accurately detect gas leakage. SUMMARY

[0009] To solve the problems in the background art, for this purpose, the application provides a gas overflow detection method based on contrast learning, and the specific scheme is as follows:

[0010] A gas overflow detection method based on contrast learning, comprising the following steps:

[0011] S1, a contrast learning training stage, a model weight of picture feature extraction is trained, the model comprises three branches for feature extraction respectively, and then three features are fused to obtain a final picture feature I o ;

[0012] S2, freeze the last model in step S1, and then connect two fully connected layers, and obtain nonlinear features through a sigmoid activation function between the fully connected layers, and finally output a feature map with a specific feature dimension size, to obtain an output of positive and negative classification, representing the classification of whether there is gas overflow;

[0013] S3, preset a confidence threshold of gas leakage, fix the angle of the picture and the thermal map acquisition device to a certain angle to capture the device to be captured, obtain the RGB three-channel picture of the corresponding device and the corresponding single-channel thermal map, input the fusion picture into the model obtained in step S2 for gas overflow classification prediction, and predict whether the confidence of the gas leakage is greater than the set value.

[0014] Specifically, S11, the steps of the three branches in step S1 are as follows:

[0015] The first branch is:

[0016] SA11, the input picture and the thermal map are merged in the concate merging module to obtain a picture I i ;

[0017] SA12, input the picture I i into the first branch to obtain a feature F DBR ; the specific steps are:

[0018] SA121, the picture I i is input into a first ordinary convolution unit to obtain an input feature, and the input feature is input into an SEBlock module to obtain a feature F SE , and the specific steps are as follows:

[0019] SA122, the feature F SE is input into two SEBlock modules and a ConvBlock module in series to obtain a feature F CC , and then the obtained F CC is input into a depth separable convolution block to obtain a feature F DBR ;

[0020] The second branch is:

[0021] SB11, input the picture into a spatial separable convolution block; the spatial separable convolution block is composed of a spatial separable convolution, a LayerNorm and a SiLu activation function, and the input picture is input into the four spatial separable convolution blocks in series to obtain a feature F SC ;

[0022] The third branch is:

[0023] SC11, input the picture into an ordinary convolution module to obtain a feature F CBThe common convolution module is composed of common convolution and ReLu activation function.

[0024] SC12, the feature F CB is input into the CBAM attention module to obtain the feature F CBAM , and the feature F CBAM is input into the PPM module to obtain the feature F PPM ; the PPM module includes 3x3, 5x5, 7x7, and 9x9 convolution kernels of different sizes arranged side by side, and the feature F CBAM is convolved to obtain features of different sizes, and then deconvolution is performed to obtain three features of a unified size.

[0025] SC13, the three features are input into the concate module and are up-sampled by UnSampling to obtain the feature F CBAM , which is added to the feature F PPM .

[0026] Specifically, step SA121 is specifically:

[0027] SA1211, the input feature is sequentially input into the first 1x1 convolution unit, the first LeakyRelu unit, and the first BatchNorm unit in series, and finally the feature F C1 is obtained.

[0028] SA1212, the feature F C1 is input into the 3*3 convolution unit, the first GeLu activation function unit, and the BatchNorm unit in series to obtain the feature F C2 .

[0029] SA1213, the feature F C2 is input into the SELayout unit to obtain the feature F SEL , which is then input into the second 1x1 convolution unit, the second LeakyRelu unit, and the second BatchNorm unit in series to obtain the feature F C3 , and the feature F C3 is obtained. i The input feature obtained by common convolution is input into the first Relu activation function unit after being merged by the concate unit to obtain the feature F SE .

[0030] Specifically, S12, the three branches are fused by the PCE fusion module to obtain the final picture I o ; the specific steps are as follows:

[0031] S121, the feature FDBR Feature F SC The features F are obtained through convolution operations respectively. DBRC and feature F SCC ;

[0032] S122, Feature F DBRC and the features respectively compared with the corresponding input features F DBR Feature F SC Concatenate to obtain feature F DF With feature F CSF ;

[0033] S123, Feature F DF With feature F CSF Feature F is obtained by concatenation. F ;

[0034] S124, F F The input is fed into the convolutional block and BatchNorm, and the feature F is obtained through upsampling. cbu ;

[0035] S125, Feature F PPM By using dilated convolution and the ReLU activation function with feature F cbu Multiplying them together yields the characteristic F. DFM ;

[0036] S125, Regarding feature F DFM Figure I is obtained by performing deconvolution and unpooling operations. o .

[0037] Specifically, step S2 ultimately outputs a feature map with a feature dimension of 2x1.

[0038] Specifically, Figure I is calculated in step S1. o with I i The loss function used is the MSE loss function.

[0039] Specifically, the model trained in steps S21-S22 uses the cross-entropy loss function.

[0040] The beneficial effects of this invention are as follows:

[0041] (1) This patent uses a multimodal information fusion method based on gas leak images and gas thermogram images to identify gas leaks. That is, features can be extracted through optical and thermodynamic information, which can better identify gas leaks.

[0042] (2) The patent greatly solves the problem of poor feature extraction caused by the lack of a large number of negative samples based on the feature extraction of the contrast learning mode, can make the model more fully understand the gas leakage characteristics and thus improve the accuracy of the model.

[0043] (3) The patent based on the IHNet network invented by the multiple attention mechanisms detects gas leakage, extracts features based on images and heat maps, has good compatible features for multiple gases, and improves the accuracy of the model for gas leakage detection. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 A structure diagram of a gas overflow detection method based on contrast learning is proposed. DETAILED DESCRIPTION

[0045] As shown in Figure 1 , a gas overflow detection method based on contrast learning includes the following steps:

[0046] S1, a contrast learning training stage, the model weight of picture feature extraction is trained, the model includes three branches for feature extraction respectively, and then the three features are fused to obtain the final picture feature;

[0047] S11, the steps of extracting by the three branches are as follows:

[0048] The first branch is:

[0049] SA11, the input picture and the heat map are merged in the concate merging module to obtain graph I i ;

[0050] SA12, input graph I i into the first branch to obtain feature F DBR ; the specific steps are as follows:

[0051] SA121, input graph I i through the first ordinary convolution unit to obtain the input feature, and input the input feature into the SEBlock module, and the specific steps are as follows:

[0052] SA1211, the input feature sequentially passes through the input of the first 1x1 convolution unit, the first LeakyRelu unit and the first BatchNorm unit in series module, and finally obtains feature F C1 ;

[0053] SA1212, take feature F C1 as the input of the 3*3 convolution unit, the first GeLu activation function unit and the BatchNorm unit in series module to obtain feature F C2 ;

[0054] SA1213, the feature F C2 Input into the SELayout unit, set the reduction factor to 8, get the feature F SEL , then input it as the input of the second 1x1 convolution unit, the second LeakyRelu unit, the second BatchNorm unit in series, get the feature F C3 , the feature F C2 and figure I i The input feature obtained by the ordinary convolution is input into the first Relu activation function unit after merging by the concate unit to obtain the feature F SE .

[0055] SA122, the feature F SE After passing through two SEBlock modules and ConvBlock modules in series, the feature F CC is obtained, and then the obtained F CC is input into the depth separable convolution block to obtain the feature F DBR .

[0056] The ConvBlock module is composed of a second ordinary convolution unit, a GeLU activation function and a BatchNorm function in series; and the depth separable convolution block is composed of a depth separable convolution, a BatchNorm and a Relu in series.

[0057] The second branch is:

[0058] SB11, input the picture into the spatial separable convolution block; the spatial separable convolution block is composed of a spatial separable convolution, a LayerNorm and a SiLu activation function, and the input picture passes through the four spatial separable convolution blocks in series to obtain the feature F SC ;

[0059] The third branch is:

[0060] SC11, input the picture into the ordinary convolution module to obtain the feature F CB ; the ordinary convolution module is composed of an ordinary convolution and a ReLu activation function;

[0061] SC12, input the feature F CB into the CBAM attention module to obtain the feature F CBAM , and input the feature F CBAM into the PPM module to obtain the feature F PPM ;

[0062] The PPM module includes 3x3, 5x5, 7x7 and 9x9 convolution kernels of different sizes arranged side by side, and the feature F CBAMThe convolution is performed to obtain features of different sizes, and then inverse convolution is performed to obtain three features of a unified size;

[0063] SC13, the three features are concatenated through a concate module and are up-sampled through UnSampling and combined with the feature F CBAM The addition is performed and the feature F is obtained through the dilated convolution with a dilated rate of 3. PPM .

[0064] S12, the three branches are fused through a PCE fusion module to obtain the final picture I o The specific steps are as follows:

[0065] S121, the features F DBR , F SC are respectively obtained through convolution operation. DBRC SCC ;

[0066] S122, the features F DBRC and F DBR are respectively concatenated with the corresponding input features F SC , F DF to obtain the features F CSF ;

[0067] S123, the features F DF and F CSF are concatenated to obtain the feature F F ;

[0068] S124, the F F is input into a convolution block and a BatchNorm, and is up-sampled to obtain the feature F cbu ;

[0069] S125, the feature F PPM is multiplied by the feature F cbu through the dilated convolution and the Relu activation function to obtain the feature F DFM ;

[0070] S125, the feature F DFM is inverse-convoluted and inverse-pooled to obtain the picture I o ;

[0071] The loss function between the picture I o and I i is the MSE loss function, and finally the network for image feature extraction is obtained.

[0072] ​S2, freeze the last model in step S1, and then connect two fully connected layers, obtain nonlinear features through a sigmoid activation function between the fully connected layers, finally output a feature map with a specific dimension of features, and obtain the output of positive and negative classification, which represents the classification of gas overflow or not, and the specific steps are as follows:

[0073] S21, freeze the weight of the obtained FDFM, and then connect two fully connected layers, and obtain nonlinear features through a sigmoid activation function between the fully connected layers;

[0074] S22, finally output a feature map with a dimension of 2x1, obtain the output of positive and negative classification, which represents the classification of gas overflow or not, and the classification is two categories of gas leakage and no gas leakage.

[0075] The loss function used by the model trained in steps S21-S22 is a cross-entropy loss function.

[0076] S3, test phase; fix the picture and thermal map acquisition device at a certain angle to fixedly shoot the device to be acquired, obtain the RGB three-channel picture of the corresponding device and the corresponding single-channel thermal map, fuse the two pictures into a picture, input the picture into the model obtained in step S2 for gas overflow classification prediction, set the prediction confidence threshold to 0.75, if the confidence of the predicted gas leakage is greater than 0.75, it is considered that the device at the position has gas leakage, and an alarm needs to be sent to remind the staff to check.

[0077] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can make equivalent replacement or change according to the technical scheme and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A contrastive learning based gas spill detection method, characterized in that, Comprise the following steps: S1, a contrast learning training phase, a model weight of picture feature extraction is trained, the model includes three branches respectively performing feature extraction, and then three features are fused to obtain a final picture feature ; S2, freeze the last model in step S1, and then connect two fully connected layers, get nonlinear features through sigmoid activation function between the fully connected layers, finally output feature map with specific dimension size of features, get the output of positive and negative classification, representing the classification of whether there is gas overflow; S3, preset the confidence threshold of gas leakage, fix the angle of the picture and thermal map acquisition device to fixed-point shooting of the device to be collected, obtain the RGB three-channel picture of the corresponding device and the corresponding single-channel thermal map, input the fusion picture into the model obtained in step S2 for gas overflow classification prediction, and predict whether the confidence of gas leakage is greater than the set value; S11, the steps of the three branches in step S1 are as follows: The first branch is: SA11, the input picture and the heat map are merged in the concate merging module to obtain a picture ; SA12, the figure input into the first branch, resulting in the feature ; the specific steps are: SA121, to the figure The input feature is obtained through the first general convolution unit, and the input feature is input into the SEBlock module to obtain the feature The specific steps are as follows: SA122, the feature After two SEBlock modules in series, ConvBlock modules get the feature , then the obtained Input to the depth separable convolution block to get the feature ; The second branch is: SB11, input the input picture into a spatial separable convolution block; the spatial separable convolution block is composed of a spatial separable convolution, a LayerNorm and a SiLu activation function, and the input picture obtains features through four spatial separable convolution blocks in series ; The third branch is: SC11, input the picture into a normal convolution module to obtain features ; the normal convolution module is composed of a normal convolution and a ReLu activation function SC12, the feature input into the CBAM attention module to obtain the feature , and the feature input into the PPM module to obtain the feature ; the PPM module includes 3x3, 5x5, 7x7, 9x9 different size convolution kernels arranged side by side, and the feature is convolved to obtain features of different sizes respectively, and then three features of a unified size are obtained by deconvolving the three features respectively. SC13, concatenate the three features and upsample them by UnSampling and add them to the feature with a dilation of 3 to get the feature .

2. The contrastive learning based gas spill detection method of claim 1, wherein, Step SA121 is specifically: The input feature of SA1211 sequentially passes through the input of a first 1x1 convolution unit, a first Leaky Relu unit, and a first BatchNorm unit in series module, and finally obtains the feature ; SA1212, the feature As the input of the 3*3 convolution unit, the first GeLu activation function unit, and the BatchNorm unit in series, the feature ; SA1213, the feature input into the SELayout unit, and the feature , then input into the second 1x1 convolution unit, the second LeakyRelu unit, and the second BatchNorm unit in series, and the feature , the feature and the figure The input feature obtained by the ordinary convolution is input into the first Relu activation function unit after being merged by the concate unit to obtain the feature .

3. The contrastive learning based gas spill detection method of claim 1, wherein, S1 further comprises a step S12 of fusing the three branches by a PCE fusion module to obtain a final picture The specific steps are as follows: S121、obtain the feature , the feature , respectively, through a convolution operation and the feature ; S122, feature and feature respectively with the corresponding input feature , feature is obtained by concate and feature ; S123, get feature with feature get feature by concate ; S124, will The input is fed into the convolutional block and BatchNorm, and the features are obtained through upsampling. ; S125、feature by the hole convolution and the Relu activation function and the feature S125, feature ; S125、to feature deconvolution and unpooling operations to the graph .

4. The contrastive learning based gas spill detection method of claim 1, wherein, The final output feature dimension of step S2 is 2x1 size feature map.

5. The contrastive learning based gas spill detection method of claim 1, wherein, The loss function between the computed obtained map in step S1 and the ground truth map is computed using a MSE loss function. With The loss function between the computed obtained map in step S1 and the ground truth map is computed using a MSE loss function.

6. The contrastive learning based gas spill detection method of claim 1, wherein, The loss function used in the model trained in step S2 is cross entropy loss function.

Images