An infrared gas detection method, device and system based on improved YOLOv5

Abstract

The present application relates to an infrared gas detection method, device and system based on improved YOLOv5.The infrared gas detection method based on improved YOLOv5 includes: acquiring infrared data, performing feature extraction through deformable convolution, and respectively outputting shallow feature maps, middle feature maps and deep feature maps; the shallow feature maps, the middle feature maps and the deep feature maps are processed through the SE attention mechanism, feature aggregation and fusion, and finally the shallow feature maps, the middle feature maps and the deep feature maps are output; the final shallow feature maps, the final middle feature maps and the final deep feature maps are classified and predicted.The infrared gas detection method based on improved YOLOv5 solves the situation that the irregular shape of the gas in the continuous frame movement leads to missed detection, and the situation that the signal quantity of the gas in the infrared video is low, and the noise is more, which interferes with the feature extraction and detection of the device.

Description

Technical Field

[0001] This invention relates to the field of target detection, and in particular to an infrared gas detection method, apparatus and system based on an improved YOLOv5. Background Technology

[0002] With the rapid advancement of industrialization and urbanization, people face significant risks to life and property from the emission or leakage of common toxic gases such as ammonia, chlorine, and sulfur hexafluoride. Therefore, the efficient and accurate detection and identification of hazardous gases is crucial for protecting people's lives and property and has significant research value. However, traditional detection methods have limitations in terms of real-time performance, detection efficiency, detection range, and accuracy. To address these issues, infrared imaging technology has been widely applied in the power and chemical industries in recent years for visual gas leak detection. Nevertheless, infrared imaging technology still faces challenges, such as requiring substantial time and effort and the inability to achieve remote real-time detection.

[0003] In recent years, with the rapid development of computer vision, deep learning-based object detection technology has gradually replaced traditional sensor detection. Object detection is one of the core applications in computer vision. It is based on image processing, convolutional neural networks, and other technologies to determine whether an object exists in an input image or video, outline its location, and classify it. Traditional object detection methods generally require manually designed rules to extract object features. This not only requires extensive comparative experiments but also involves a degree of subjectivity in feature selection, making the process cumbersome and resulting in insufficient generalization ability and low accuracy. With the increasing development of deep learning, convolutional neural networks have been found to have excellent feature extraction capabilities and are increasingly being widely applied in the field of object detection. Deep learning-based object detection generally requires a large number of training samples. By constructing deep network devices and training them on a large amount of image data, powerful feature extraction capabilities are obtained. Compared with traditional object detection methods, deep learning methods have significant advantages. On the one hand, they save time on manual feature extraction; on the other hand, through autonomous training on a large amount of data, the network device has strong feature extraction capabilities, greater robustness, and higher accuracy.

[0004] Although deep learning-based target detection algorithms have achieved high recognition rates on various public datasets and cover a wide range of target types, there are currently no publicly available target detection algorithms specifically for infrared gas detection. Summary of the Invention

[0005] Therefore, the purpose of this invention is to provide an infrared gas detection method based on an improved YOLOv5.

[0006] An infrared gas detection method based on an improved YOLOv5 includes the following steps:

[0007] S1: Acquire infrared data, extract features through deformable convolution, and output shallow feature maps, middle feature maps and deep feature maps respectively;

[0008] S2 uses the SE attention mechanism to aggregate and fuse shallow feature maps, mid-level feature maps, and deep feature maps, and outputs the final shallow feature map, final mid-level feature map, and final deep feature map respectively.

[0009] S3 predicts and classifies the final shallow feature map, the final middle feature map, and the final deep feature map.

[0010] The infrared gas detection method based on the improved YOLOv5 described in this invention solves the problems of missed detection caused by the irregular shape of gas in continuous frame motion, as well as the problems of low gas signal and high noise in infrared video, which interfere with the feature extraction and detection of the device.

[0011] Furthermore, step S1 includes the following sub-steps:

[0012] S11 performs a convolutional feature extraction on its data once, and outputs a shallow feature map;

[0013] S12 further performs convolutional feature extraction on the shallow feature map and outputs the middle-layer feature map;

[0014] S13 uses deformable convolution to further extract features from the mid-level feature map and outputs a deep feature map.

[0015] S14 performs SPPF pooling on the deep feature map, removes related duplicate feature extraction, and outputs the pooled deep feature map.

[0016] By replacing the general convolution before SPPF pooling with deformable convolution, the problem of the irregular shape of the gas is solved, which prevents the general convolution from accurately extracting feature information beyond the target annotation range, resulting in feature information loss and affecting the detection results.

[0017] Furthermore, step S2 includes the following sub-steps:

[0018] S210 weights shallow, medium, and deep feature maps using the SE attention mechanism and performs feature aggregation and fusion through upsampling.

[0019] S220: Upsample the shallow, medium, and deep feature maps after aggregation and fusion, respectively, and then perform feature aggregation and fusion by downsampling.

[0020] Furthermore, the SE attention mechanism also includes the following steps:

[0021] S214A aggregates two feature maps and then performs global average pooling (dimensionality reduction) to obtain a feature vector;

[0022] S214B maps the feature vector to a new feature vector through a fully connected layer and performs weighted summation to represent the importance of different channels, resulting in a feature vector with weighted information.

[0023] By multiplying the weighted feature vector with the aggregated feature map, the channels of the first aggregated feature map are weighted. During the upsampling, aggregation, and fusion of the shallow, medium, and deep feature maps, weight information is added with minimal computational cost. This enables the YOLOv5 device to quickly locate the region of interest and notice information that is more correlated with the gas, thereby improving detection accuracy with the lowest computational cost.

[0024] Furthermore, step S3 includes the following sub-steps:

[0025] S31 performs convolutional fusion on the weighted information aggregated feature map to output the final shallow feature map;

[0026] S32 determines whether gas features exist in the final shallow feature map;

[0027] S33 performs convolutional fusion on the final mid-layer feature map after aggregation to obtain the final mid-layer feature map;

[0028] S34 determines whether gas features exist in the final mid-layer feature map after polymerization;

[0029] S35 performs convolutional fusion on the aggregated deep feature map to obtain the final deep feature map;

[0030] S36 determines whether gas features exist in the final deep feature map;

[0031] S37 integrates all test results to perform a unified test and outputs the final test result.

[0032] Furthermore, the trainable parameters of the device are trained through the following steps:

[0033] Predictions are made on the infrared image dataset to obtain gas detection results;

[0034] Calculate the IOU loss of gas detection results;

[0035] Calculate the distance loss of gas detection results;

[0036] Calculate the directional loss of gas detection results;

[0037] The total EIOU loss of the gas detection results is obtained by summing up the IOU loss, distance loss, and orientation loss.

[0038] Backpropagation is performed based on the total EIOU loss value to update the trainable parameters of the device until the total EIOU loss converges.

[0039] By using the directional loss of the EIOU loss function, the disproportionate changes in the height and width of the detection box caused by gas deformation can be calculated more efficiently, thereby improving training efficiency.

[0040] An infrared gas detection device based on an improved YOLOv5 includes a Backbone unit, a Neck unit, and a Head unit;

[0041] The Backbone unit is used to acquire the infrared data, extract features through deformable convolution, and output shallow feature maps, mid-level feature maps and deep feature maps respectively.

[0042] The Neck unit is used to aggregate and fuse shallow feature maps, mid-level feature maps, and deep feature maps through the SE attention mechanism, and output the final shallow feature map, final mid-level feature map, and final deep feature map respectively.

[0043] The Head unit is used to predict, classify, and integrate the final shallow feature map, the final middle feature map, and the final deep feature map, based on the integrated prediction results.

[0044] An infrared gas detection system based on an improved YOLOv5 includes an infrared camera, an infrared gas detection device based on an improved YOLOv5, and a display device;

[0045] The infrared camera captures infrared data and sends it to the aforementioned infrared gas detection device based on the improved YOLOv5, generating a detection frame corresponding to the gas position and outputting it to the display device.

[0046] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the infrared gas detection system based on the improved YOLOv5 of the present invention;

[0048] Figure 2 for Figure 1 The diagram shows the structure of the infrared gas detection device.

[0049] Figure 3 for Figure 2 A flowchart of infrared gas detection using the infrared gas detection device shown.

[0050] Figure 4 for Figure 2 A schematic diagram of the overall structure of the infrared gas detection device shown.

[0051] Figure 5 for Figure 3 The following is a flowchart of the overall process for infrared gas detection;

[0052] Figure 6 This is a flowchart of the loss feedback process for an infrared gas detection device. Detailed Implementation

[0053] The inventors of this application address the problem of infrared gas detection by using the YOLOv5 (You Only Look Once, YOLO version 5) target detection device as a foundation. The YOLOv5 includes a backbone unit, a neck unit, and a head unit. The backbone unit is primarily responsible for extracting features from the input image, obtaining feature maps at different scales. The neck unit is responsible for multi-scale feature fusion of the feature maps at different scales and then transmitting the fused feature map to the head unit. The head unit is responsible for predicting the fused feature map and outputting the prediction result.

[0054] During the research process, the inventors discovered that infrared gas detection using YOLOv5 suffered from low accuracy and inaccurate detection. After a series of investigations, the inventors improved the Backbone unit of YOLOv5. During multiple improvement experiments, the inventors found that reducing computational costs while improving the Backbone unit was extremely difficult. Therefore, to address this issue, the inventors further improved the Neck unit in YOLOv5, reducing computational costs while maintaining detection accuracy. Simultaneously, the inventors collected relevant infrared gas data for use in publicly available datasets that were lacking, and performed data augmentation. However, they found that the low quality of this infrared gas data led to slow convergence and low accuracy, impacting time costs. Based on this, the inventors further improved the loss function of YOLOv5. Finally, the inventors proposed an infrared gas detection method based on the improved YOLOv5.

[0055] Please see Figure 1 , Figure 1 This is a schematic diagram of the infrared gas detection system based on the improved YOLOv5 of the present invention.

[0056] The infrared gas detection system based on improved YOLOv5 of the present invention includes an infrared camera, an infrared gas detection device based on improved YOLOv5, and a display device. The infrared camera is placed in an environment where gas leakage may occur. The infrared gas detection device based on improved YOLOv5 is connected to the infrared camera, processes the infrared data transmitted by the infrared camera, and if a leaking gas is detected, generates a prediction frame to enclose the gas and outputs it to the display device to alert the observer to the presence of a gas leak in the current environment.

[0057] Please also refer to Figure 2 and Figure 3 , Figure 2 for Figure 1 The diagram shown is a structural diagram of an infrared gas detection device. Figure 3 for Figure 2 The flowchart of infrared gas detection using the infrared gas detection device shown is as follows.

[0058] The infrared gas detection device includes a backbone unit 1, a neck unit 2, and a head module 3.

[0059] The Backbone unit 1 is used to perform step S1: acquire infrared data, extract features through deformable convolution, and output shallow feature maps, mid-level feature maps and deep feature maps respectively.

[0060] The Neck unit 2 is used to perform step S2: to aggregate and fuse the shallow feature map, the middle feature map and the deep feature map through the SE attention mechanism, and output the final shallow feature map, the final middle feature map and the final deep feature map respectively.

[0061] The Head module 3 is used to perform step S3: predict, classify and integrate the final shallow feature map, the final middle feature map and the final deep feature map, and generate a detection box corresponding to the gas position and output it to the display device based on the integrated prediction results.

[0062] Please also refer to Figure 4 and Figure 5 , Figure 4 for Figure 2 The diagram shown is a schematic representation of the overall structure of the infrared gas detection device. Figure 5 Figure 3 The diagram shows the overall process flow for infrared gas detection.

[0063] Specifically, the Backbone unit 1 includes a first feature extraction module 11, a second feature extraction module 12, a deformable convolutional feature extraction module 13, and an SPPF pooling module 14.

[0064] The first feature extraction module 11 is used to perform step S11: acquire the infrared data to be detected, perform one round of convolution feature extraction on the data, and output a shallow feature map.

[0065] The second feature extraction module 12 is used to perform step S12: further perform convolutional feature extraction on the shallow feature map and output the middle feature map.

[0066] The deformable convolution feature extraction module 13 is used to perform step S13: through deformable convolution, further deformable convolution feature extraction is performed on the intermediate feature map to output the deep feature map.

[0067] The SPPF pooling module 14 is used to perform step S14: perform SPPF pooling operation on the deep feature map, remove related duplicate feature extraction, and output the pooled deep feature map.

[0068] Specifically, the convolutional feature extraction described in this invention is actually the C3 feature extractor used in YOLOv5 in the prior art. The C3 uses a lightweight convolutional neural network structure, which reduces the size of the feature map through multiple convolution and upsampling operations, while increasing the number of channels to ensure feature fusion and feature compression of the feature map. This improves the detection accuracy and speed of YOLOv5, reduces the number of parameters in YOLOv5, and improves detection efficiency.

[0069] The deformable convolution can adjust the position of the convolution kernel by introducing a learnable offset, thus enabling the kernel to better adapt to the deformation of the gas. Furthermore, using deformable convolution in deep feature extraction can improve the semantic information about the gas carried in deep feature maps.

[0070] The learnable offset of the deformable convolution increases the computational cost. Therefore, after investigation and adjustment, the inventors set it in the part of feature extraction of deep feature maps, so that the deformable convolution can use the minimum computational cost while maximizing the extraction of semantic information of gas features, thereby maximizing the improvement of accurate gas feature extraction and generalization ability.

[0071] This invention addresses the issue of irregular gas shapes leading to inaccurate feature extraction beyond the target annotation range, resulting in feature loss and impacting detection results. It replaces the general convolution before SPPF pooling with deformable convolution. Simultaneously, it solves the problem of missed detections caused by the irregular shape of gas during continuous frame motion and its constantly changing shape.

[0072] Furthermore, the Neck unit 2 includes a feature pyramid module 210 and a path aggregation module 220.

[0073] The feature pyramid module 210 is used to perform step S210: weighting the shallow feature map, the middle feature map and the deep feature map through the SE attention mechanism, and performing feature aggregation and fusion through upsampling.

[0074] The path aggregation module 220 is used to perform step S220: upsample the aggregated and fused shallow feature map, middle feature map and deep feature map respectively, and perform feature aggregation and fusion by downsampling.

[0075] Specifically, the feature pyramid module 210 further includes a channel matching submodule 211, a first upsampling submodule 212, a first splicing submodule 213, an SE attention submodule 214, a weight aggregation submodule 215, a weight fusion submodule 216, a second upsampling submodule 217, and a second splicing submodule 218.

[0076] The channel matching submodule 211 is used to perform step S211: convolving the pooled deep feature map to increase the number of channels, so that the number of channels in the deep feature map matches the number of channels in the middle feature map, and outputting a channel-matched deep feature map.

[0077] The first upsampling submodule 212 is used to perform step S212: perform upsampling operation on the channel matching deep feature map and output an enlarged deep feature map.

[0078] The first splicing submodule 213 is used to perform step S213: splicing the expanded deep feature map with the middle feature map to obtain the first aggregated feature map.

[0079] The SE attention submodule 214 is used to perform step S214: reduce the dimensionality and weight the first aggregated feature map to obtain a weighted feature vector.

[0080] The weight aggregation submodule 215 is used to perform step S215: multiply the weight feature vector with the first aggregated feature map, and weight the channels of the first aggregated feature map to obtain a weight information feature map.

[0081] The weight fusion submodule 216 is used to perform step S216: perform convolution fusion on the weight information feature map to obtain a fused weight information feature map.

[0082] The second upsampling submodule 217 is used to perform step S217: perform upsampling operation on the fused weight information feature map to obtain an expanded weight information feature map.

[0083] The second splicing submodule 218 is used to perform step S218: splicing the expanded weight information feature map with the shallow feature map to obtain a weight information aggregated feature map.

[0084] Specifically, the SE attention submodule 214 further includes a pooling dimensionality reduction submodule 214A and a mapping weighting submodule 214B.

[0085] The pooling dimensionality reduction submodule 214A is used to perform step S214A: global average pooling (dimensionality reduction) on the first aggregated feature map to obtain the feature vector.

[0086] The mapping weighting submodule 214B is used to perform step S214B: mapping the feature vector to a new feature vector through a fully connected layer, and weighting to represent the importance of different channels, so as to obtain a feature vector with weight information.

[0087] The inventors introduced the SE attention mechanism to add weight information to shallow, medium and deep feature maps during feature aggregation and fusion with minimal computational cost. This enables the YOLOv5 device to quickly locate the region of interest and notice information that is more relevant to the gas, thereby improving detection accuracy with minimal computational cost.

[0088] Furthermore, the deep feature maps possess rich semantic information. Upsampling allows the scale of deep feature maps with high semantic information to match that of shallow feature maps, enabling the concatenation of feature maps at different scales (deep, mid-level, and shallow). Accordingly, after the upsampling operation, feature maps of different scales are concatenated and then fused via convolution, imbuing the shallow feature maps with the high semantic information of the deep feature maps.

[0089] Specifically, the path aggregation module 220 further includes a first downsampling submodule 221, a third splicing submodule 222, a second downsampling submodule 223, and a fourth splicing submodule 224.

[0090] The first downsampling submodule 221 is used to perform step S221: perform convolutional fusion and downsampling on the weight information aggregated feature map, and output the final shallow feature map after downsampling.

[0091] The third splicing submodule 222 is used to perform step S222: splicing the final shallow feature map after downsampling with the fused weight information feature map and performing a horizontal connection to obtain the final middle-layer feature map after aggregation.

[0092] The second downsampling submodule 223 is used to perform step S223: perform convolutional fusion and downsampling on the aggregated final mid-layer feature map, and output the reduced final mid-layer feature map.

[0093] The fourth splicing submodule 224 is used to perform step S224: splicing the reduced final middle layer feature map with the channel matching deep feature map and performing horizontal connection to obtain the aggregated final deep feature map.

[0094] Deep feature maps possess rich semantic information but lack localization information. Upsampling allows shallow feature maps to incorporate the semantic information of deep feature maps. Conversely, downsampling allows deep feature maps to incorporate the localization and weight information of shallow feature maps, ensuring that the localization and weight information of shallow feature maps are not lost. This process fully integrates feature information from different scales, assigning the high localization information of shallow feature maps to deep feature maps.

[0095] Accordingly, this invention introduces the SE attention mechanism to fully integrate weight information between feature maps of different scales, thereby solving the problem that the gas signal quantity in infrared video is low and there is a lot of noise, which easily interferes with the feature extraction and detection of the device.

[0096] Preferably, during the concatenation and aggregation of deep and mid-level feature maps, an SE attention mechanism is introduced, which allows weight information to be assigned as quickly as possible. During subsequent feature fusion, the feature maps will carry this weight information. Accordingly, the SE attention module 214 adds sufficient weight information to the feature maps during feature aggregation and fusion with minimal computational cost, thereby improving the accuracy of infrared gas detection.

[0097] Meanwhile, the deep feature map contains semantic information extracted by deformable convolution. Therefore, by setting the SE attention module 214 after the deep feature map and the middle feature map are spliced ​​and aggregated, the semantic information extracted by deformable convolution can be further utilized, making the most effective use of computational cost and improving the detection accuracy of infrared gas by the YOLOv5 device.

[0098] Furthermore, the Head unit 3 includes a first fusion module 31, a first detection module 32, a second fusion module 33, a second detection module 34, a third fusion module 35, a third detection module 36, and a final detection module 37.

[0099] The first fusion module 31 is used to perform step S31: after performing convolutional fusion on the weight information aggregated feature map, output the final shallow feature map.

[0100] The first detection module 32 is used to perform step S32: determine whether gas features exist in the final shallow feature map.

[0101] The second fusion module 33 is used to perform step S33: perform convolutional fusion on the aggregated final mid-layer feature map to obtain the final mid-layer feature map.

[0102] The second detection module 34 is used to perform step S34: determine whether there are gas features in the final mid-layer feature map after polymerization.

[0103] The third fusion module 35 is used to perform step S35: convolutionally fusing the aggregated final deep feature map to obtain the final deep feature map.

[0104] The third detection module 36 is used to perform step S36: determine whether gas features exist in the final deep feature map.

[0105] The final detection module 37 is used to perform step S37: by combining all detection results, perform integrated detection, and output the final detection result.

[0106] Specifically, steps S32, S34, and S36 detect gases of different sizes using shallow, medium, and deep feature maps, respectively. The integrated detection outputs the detection results of gases of different sizes to the same screen, thereby achieving multi-target detection.

[0107] This invention introduces deformable convolution characteristics and SE attention mechanism into the existing YOLOv5 architecture. Accordingly, it achieves considerable improvements in the size, location, and recognition of infrared gases. Furthermore, given the lightweight nature of YOLOv5, the improvements in this invention do not significantly increase computational costs. Therefore, the infrared gas detection device based on the improved YOLOv5 can achieve considerable accuracy in long-range infrared gas detection.

[0108] To improve the accuracy of the infrared gas detection device, the inventors trained the device, including infrared gas data collection and loss feedback of the infrared gas detection device.

[0109] The infrared gas data collection includes the following steps:

[0110] 1. Use an infrared thermal imager to take infrared gas simulation photos of the gas cylinder.

[0111] 2. Compile the captured video frames into a dataset.

[0112] 3. Augment and expand the dataset by performing data augmentation.

[0113] Specifically, the data augmentation involves performing operations such as rotation, translation, scaling, and mirroring on each image in the dataset.

[0114] 4. Perform Gaussian filtering on each image in the expanded dataset to remove noise.

[0115] 5. Perform histogram equalization on the denoised dataset to enhance the signal-to-noise ratio and contrast of the dataset.

[0116] 6. Label the dataset.

[0117] Specifically, the labeling involves outlining the location of the gas in each image and labeling it with a gas tag, generating a corresponding label file for each image.

[0118] 7. Divide the dataset into training, validation, and test sets in an 8:1:1 ratio.

[0119] Please see Figure 6 , Figure 6 This is a flowchart of the loss feedback process for an infrared gas detection device.

[0120] The loss feedback of the infrared gas detection device includes the following steps:

[0121] A1: Predict the gas from the infrared images in the dataset.

[0122] A2: Calculate the IOU loss of the gas prediction results.

[0123] Specifically, the IOU is the ratio of the intersection to the union of the predicted bounding boxes generated by the infrared gas detection device and the ground truth bounding boxes labeled in the dataset. Therefore, the IOU loss is:

[0124] IOU Loss = 1 - IOU,

[0125] When the two boxes overlap, the IOU is 1, so the IOU loss is 1-IOU. There is a problem that when the predicted box does not intersect with the ground truth box, the IOU is 0, and there is no gradient backpropagation, so further learning and training are impossible.

[0126] A3: Calculate the distance loss in the gas prediction results.

[0127] The distance loss can be expressed as:

[0128]

[0129] Where, ρ 2 The Euclidean distance between the diagonals of the minimum bounding box is represented by b,b gt c represents the center coordinates of the predicted bounding box and the ground truth bounding box. 2 This represents the diagonal length of the minimum bounding matrix that simultaneously contains the predicted and ground truth boxes. Based on this, a distance loss (Loss) is added. disIt can directly minimize the distance between the center points of two boxes, and solve the problem of still providing a direction for movement of the predicted box when the predicted box does not intersect with the ground truth box. However, there is still a problem that the aspect ratio of the predicted box is not detected.

[0130] A4: Calculate the directional loss of gas prediction results.

[0131] A5: Summarize the IOU loss, distance loss, and orientation loss to obtain the total EIOU loss.

[0132] The directional loss can be expressed as:

[0133]

[0134] Among them, w,w gt h,h represents the width of the predicted bounding box and the ground truth bounding box. gt The height of the predicted bounding box and the ground truth bounding box. and These represent the minimum box width and height covering the ground truth box and the predicted box, respectively. The changes in the width and height of the predicted box can be calculated independently based on the directional loss. This directional loss solves the problem of aspect ratio-based detection.

[0135] A6: Perform backpropagation based on the total EIOU loss value to update the trainable parameters of the device until the total EIOU loss converges.

[0136] Therefore, the total EIOU loss is expressed as:

[0137]

[0138] The original YOLOv5 device used the CIOU loss function, which includes IOU loss, distance loss, and aspect ratio loss. The inventors discovered that the aspect ratio loss used by CIOU, during training, stops changing when the predicted bounding box's width and height are linearly proportional to the ground truth bounding box. Therefore, due to the irregular deformation characteristics of gases, the width and height of the predicted bounding box are easily affected. Meanwhile, since this invention uses deformable convolution as a feature extraction module, its learnable offset affects the convolution kernel position, thus influencing the aspect ratio of the generated predicted bounding box. Therefore, when the erroneous predicted bounding box generated under the influence of deformable convolution is linearly proportional to the ground truth bounding box, it will cause the predicted bounding box to learn in the wrong direction. Therefore, the inventors selected the EIOU loss function, whose directional loss perfectly matches the requirements for gas detection.

[0139] Compared to existing technologies, this invention, through an improved YOLOv5-based infrared gas detection device, solves the problems of missed detections caused by the irregular shapes of gases in continuous frame motion, as well as the low signal strength and high noise levels of gases in infrared video, which interfere with feature extraction and detection. Furthermore, the device training process is improved to address the characteristics of the improved features and the properties of gases, effectively increasing training efficiency. Simultaneously, this device improves the accuracy of infrared gas detection while maintaining low computational costs, enabling remote real-time detection of infrared gases.

[0140] Based on the same inventive concept, this application also provides an electronic device, which can be a server, desktop computing device, or mobile computing device (e.g., laptop computing device, handheld computing device, tablet computer, netbook, etc.) or other terminal device. The device includes one or more processors and a memory, wherein the processor is used to execute a program to implement the fall location joint detection method of the embodiments of the present invention; the memory is used to store a computer program executable by the processor.

[0141] Based on the same inventive concept, this application also provides a computer-readable storage medium corresponding to the aforementioned embodiments of the infrared gas detection method based on improved YOLOv5. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the infrared gas detection method based on improved YOLOv5 described in any of the above embodiments.

[0142] This application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0143] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and the present invention also intends to include these modifications and variations.

Claims

1. An infrared gas detection method based on improved YOLOv5, characterized in that, Includes the following steps: S1: Acquire infrared data, extract features through deformable convolution, and output shallow feature maps, medium feature maps and deep feature maps respectively; S2 uses the SE attention mechanism to aggregate and fuse shallow feature maps, mid-level feature maps, and deep feature maps, and outputs the final shallow feature map, final mid-level feature map, and final deep feature map respectively. S3 predicts and classifies the final shallow feature map, the final middle feature map, and the final deep feature map; Step S1 includes the following sub-steps: S11, perform one convolutional feature extraction on its data, and output a shallow feature map; S12, further perform convolutional feature extraction on the shallow feature map to output the middle layer feature map; S13, through deformable convolution, further deformable convolution feature extraction is performed on the middle layer feature map to output the deep layer feature map; S14, Perform SPPF pooling on the deep feature map, remove related duplicate feature extraction, and output the pooled deep feature map; Step S2 includes the following sub-steps: S211, convolve the pooled deep feature map to increase the number of channels, so that the number of channels in the deep feature map matches the number of channels in the middle feature map, and the output channels match the deep feature map; S212, performs an upsampling operation on the channel matching deep feature map and outputs an enlarged deep feature map; S213, the expanded deep feature map is concatenated with the intermediate feature map to obtain the first aggregated feature map; S214, reduce the dimensionality and weight the first aggregated feature map to obtain the weighted feature vector; S215, multiply the weighted feature vector with the first aggregated feature map, and weight the channels of the first aggregated feature map to obtain the weighted information feature map; S216, perform convolution on the weight information feature map to fuse the features, and obtain the fused weight information feature map; S217, perform upsampling operation on the fused weight information feature map to obtain an expanded weight information feature map; S218, the weight information feature map is expanded and concatenated with the shallow feature map to obtain the weight information aggregated feature map; Step S214 includes the following sub-steps: S214A, perform global average pooling on the first aggregated feature map to obtain the feature vector; S214B maps the feature vector to a new feature vector through a fully connected layer, and performs weighted summation to represent the importance of different channels, thus obtaining a feature vector with weight information. Step S2 further includes the following steps: S221, perform convolutional fusion and downsampling on the weight information aggregated feature map, and output the final shallow feature map after downsampling; S222, after downsampling, the final shallow feature map is concatenated with the fused weight information feature map and then horizontally connected to obtain the final mid-layer feature map after aggregation. S223 performs convolutional fusion and downsampling on the final mid-layer feature map after aggregation, and outputs the reduced final mid-layer feature map; S224: The final mid-layer feature map after reduction is concatenated with the channel-matched deep feature map and then horizontally connected to obtain the final aggregated deep feature map.

2. The method according to claim 1, characterized in that, Step S3 includes the following sub-steps: S31, perform convolutional fusion on the weight information aggregation feature map to output the final shallow feature map; S32, determine whether gas features exist in the final shallow feature map; S33, perform convolutional fusion on the final mid-layer feature map after aggregation to obtain the final mid-layer feature map; S34, determine whether there are gas features in the final mid-layer feature map after polymerization; S35, perform convolution to fuse the features of the aggregated final deep feature map to obtain the final deep feature map; S36, Determine whether gas features exist in the final deep feature map; S37 integrates all test results to perform a unified test and outputs the final test result.

3. An infrared gas detection device based on an improved YOLOv5, characterized in that, Includes Backbone units, Neck units, and Head units; The Backbone unit is used to acquire infrared data, extract features through deformable convolution, and output shallow feature maps, mid-level feature maps and deep feature maps respectively. The Neck unit is used to aggregate and fuse shallow feature maps, mid-level feature maps, and deep feature maps through the SE attention mechanism, and output the final shallow feature map, final mid-level feature map, and final deep feature map respectively. The Head unit is used to perform prediction, classification and integration of the final shallow feature map, the final middle feature map and the final deep feature map, and generate a detection box corresponding to the gas location based on the integrated prediction results. The Backbone unit includes a first feature extraction module, a second feature extraction module, a deformable convolutional feature extraction module, and an SPPF pooling module; The first feature extraction module is used to acquire the infrared data to be detected, perform one convolutional feature extraction on the data, and output a shallow feature map. The second feature extraction module is used to further extract convolutional features from the shallow feature map and output the middle feature map; The deformable convolution feature extraction module further extracts deformable convolution features from the mid-level feature map through deformable convolution, and outputs a deep feature map. The SPPF pooling module is used to perform SPPF pooling operation on the deep feature map, remove related duplicate feature extraction, and output the pooled deep feature map. The Neck unit includes a channel matching submodule, a first upsampling submodule, a first stitching submodule, an SE attention submodule, a weight aggregation submodule, a weight fusion submodule, a second upsampling submodule, a second stitching submodule, a first downsampling submodule, a third stitching submodule, a second downsampling submodule, and a fourth stitching submodule. The channel matching submodule is used to convolve the pooled deep feature map to increase the number of channels, so that the number of channels in the deep feature map matches the number of channels in the middle feature map, and outputs a channel-matched deep feature map. The first upsampling submodule is used to perform an upsampling operation on the channel matching deep feature map and output an enlarged deep feature map; The first splicing submodule is used to splice the expanded deep feature map with the mid-level feature map to obtain the first aggregated feature map; The SE attention submodule is used to perform global average pooling on the first aggregated feature map to obtain a feature vector. After obtaining the feature vector, the feature vector is mapped to a new feature vector through a fully connected layer, and weighted to represent the importance of different channels, so as to obtain a feature vector with weight information. The weight aggregation submodule is used to perform the multiplication of the weight feature vector with the first aggregated feature map, and to weight the channels of the first aggregated feature map to obtain the weight information feature map. The weight fusion submodule is used to perform convolutional fusion of features on the weight information feature map to obtain a fused weight information feature map; The second upsampling submodule is used to perform an upsampling operation on the fused weight information feature map to obtain an expanded weight information feature map; The second splicing submodule is used to splice the expanded weight information feature map with the shallow feature map to obtain a weight information aggregated feature map; The first downsampling submodule is used to perform convolutional fusion and downsampling on the weight information aggregated feature map, and output the final shallow feature map after downsampling; The third splicing submodule is used to splice the downsampled final shallow feature map with the fused weight information feature map and perform a horizontal connection to obtain the aggregated final middle-layer feature map. The second downsampling submodule is used to perform convolutional fusion and downsampling on the final mid-layer feature map after aggregation, and output the reduced final mid-layer feature map; The fourth stitching submodule is used to stitch the reduced final middle layer feature map with the channel matching deep feature map, and perform horizontal connection to obtain the aggregated final deep feature map. The Head unit includes a first fusion module, a first detection module, a second fusion module, a second detection module, a third fusion module, a third detection module, and a final detection module; The first fusion module performs convolutional fusion on the weight information aggregated feature map and outputs the final shallow feature map. The first detection module is used to determine whether gas features exist in the final shallow feature map; The second fusion module is used to perform convolutional fusion on the aggregated final mid-layer feature map to obtain the final mid-layer feature map; The second detection module is used to determine whether gas features are present in the final mid-layer feature map after polymerization; The third fusion module is used to perform convolutional fusion of features on the aggregated final deep feature map to obtain the final deep feature map; The third detection module is used to determine whether gas features exist in the final deep feature map; The final detection module is used to perform integrated detection by combining all detection results and output the final detection result.

4. The apparatus according to claim 3, characterized in that, The trainable parameters of the device are trained through the following steps: Predictions are made on the infrared image dataset to obtain gas detection results; Calculate the IOU loss of gas detection results; Calculate the distance loss of gas detection results; Calculate the directional loss of gas detection results; The total EIOU loss of the gas detection results is obtained by summing up the IOU loss, distance loss, and orientation loss. Backpropagation is performed based on the total EIOU loss value to update the trainable parameters of the device until the total EIOU loss converges.

5. An infrared gas detection system based on an improved YOLOv5, characterized in that: Includes infrared camera equipment, an infrared gas detection device based on the improved YOLOv5 as described in claim 3, and a display device; The infrared camera captures infrared data and sends it to the infrared gas detection device based on the improved YOLOv5. The infrared gas detection device based on the improved YOLOv5 includes a Backbone unit, a Neck unit, and a Head unit; The Backbone unit is used to acquire the infrared data, extract features through deformable convolution, and output shallow feature maps, mid-level feature maps and deep feature maps respectively. The Neck unit is used to aggregate and fuse shallow feature maps, mid-level feature maps, and deep feature maps through the SE attention mechanism, and output the final shallow feature map, final mid-level feature map, and final deep feature map respectively. The Head unit is used to predict, classify, and integrate the final shallow feature map, the final middle feature map, and the final deep feature map. Based on the integrated prediction results, it generates a detection box corresponding to the gas location and outputs it to the display device.

Images