A carbon monoxide gas infrared image segmentation and detection method based on improved U-Net
By improving the U-Net model, introducing ResNet50 and ECA attention mechanisms, and adopting a hybrid loss function of Focal Loss and Dice Loss, the problem of low segmentation accuracy in infrared image segmentation of carbon monoxide gas was solved, achieving higher accuracy in gas leak area identification and segmentation.
Patent Information
- Application Number
- CN202611318318.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-28
- Publication Date
- 2026-11-17
AI Technical Summary
Existing methods struggle to accurately segment leakage areas in infrared images of carbon monoxide gas, especially when background contrast is low and target boundaries are blurred, resulting in poor segmentation accuracy.
The U-Net model is improved by introducing the ResNet50 module to enhance feature extraction. The model training is optimized by combining the ECA attention mechanism and the hybrid loss function (Focal Loss and Dice Loss) to improve the accuracy of gas region segmentation.
It significantly improves the segmentation accuracy of infrared images of carbon monoxide gas, enabling more accurate identification and segmentation of gas leak areas, and is suitable for real-time monitoring in industrial scenarios.
Smart Images

Figure CN123118528A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of infrared image processing and target detection technology, specifically to a carbon monoxide gas infrared image segmentation and detection method based on an improved U-Net, which is suitable for real-time monitoring of invisible carbon monoxide gas leaks in industrial scenarios. Background Technology
[0002] In industrial production, gases are widely used across various industries as important raw materials and auxiliary media. However, some industrial gases possess dangerous characteristics such as flammability, explosiveness, toxicity, or asphyxiation, posing significant potential safety risks. These gases are typically colorless and invisible, making them highly concealed and dangerous. Leaks can easily lead to poisoning, fires, or even explosions, seriously threatening lives and property. Carbon monoxide, in particular, is a colorless, odorless, and non-irritating toxic gas that is not only highly toxic but also poses a high explosion hazard. When the concentration of carbon monoxide in the air exceeds 12%, it is highly likely to explode under certain conditions. Therefore, timely and effective detection of CO gas leaks and accurate location of the leak and its diffusion area are of significant research importance.
[0003] Infrared imaging technology, as an important non-contact detection method, is widely used in gas leak detection due to its advantages such as wide detection range, intuitive visualization, and long-distance dynamic detection. However, CO gas infrared images generally suffer from low contrast between the gas area and the background, as well as blurred gas morphology details, which affects their effectiveness in practical applications such as accurate CO gas leak detection and environmental monitoring. Therefore, appropriate algorithms are needed to enhance the acquired CO gas infrared images to meet the requirements of various fields.
[0004] In recent years, with the rapid development of deep learning technology, intelligent infrared gas leak detection methods based on convolutional neural networks and image segmentation models have emerged. Compared with traditional manual feature extraction and threshold discrimination methods, deep learning models can automatically learn the gas texture, morphology, and spatial distribution characteristics in infrared images, significantly improving the accuracy and automation of gas leak detection. However, existing research mainly focuses on gases with obvious infrared absorption characteristics, such as methane, volatile organic compounds (VOCs), and sulfur hexafluoride (SF6), while research on the detection of gases with weak infrared absorption and low imaging contrast, such as carbon monoxide, is relatively limited. Furthermore, due to the problems of blurred target boundaries, irregular regional morphology, and small gray-level difference between carbon monoxide and the background in infrared images, existing methods struggle to accurately extract its leak area. Therefore, a method to improve the segmentation accuracy of carbon monoxide infrared images is urgently needed. Summary of the Invention
[0005] The technical problem to be solved by the present invention is: The present invention provides a carbon monoxide gas infrared image segmentation and detection method based on improved U-Net, which aims to solve the technical problems of low contrast between gas region and background and low segmentation accuracy in the existing infrared image segmentation and detection of invisible carbon monoxide gas.
[0006] To achieve the above objectives, this invention provides a carbon monoxide gas infrared image segmentation and detection method based on an improved U-Net, the specific technical solution of which is as follows: The method includes: The encoder network of the U-Net model is improved by introducing a ResNet50 module to replace the original U-Net encoder. The residual structure is used to enhance the feature extraction capability, and transfer learning is combined to accelerate model convergence and improve generalization performance. The skip connection mechanism of the model is optimized by adding an ECA attention mechanism, which enables adaptive adjustment of the importance of channel features while maintaining low parameter and computational cost. A hybrid loss function combining Focal Loss and Dice Loss is used.
[0007] The specific steps of the method include: S1. Dataset Construction and Processing: Considering the lack of infrared images of carbon monoxide gas with human image interference in existing public datasets, this invention uses a laboratory infrared camera to collect 16-bit infrared images of carbon monoxide gas at a certain concentration with human image interference in the background. Since 16-bit CO gas infrared images are not visible to the human eye, they are converted into 8-bit images for visualization and enhanced to highlight and strengthen the morphology and distribution characteristics of the gas. Subsequently, the Labelme tool is used to annotate the images at the pixel level, resulting in a dataset containing 2500 images, which are divided into a training set of 2000 images, a validation set of 250 images, and a test set of 250 images in an 8:1:1 ratio, providing data support for model training and performance verification.
[0008] S2. Improved U-Net Model Construction: Based on the traditional U-Net model, to address the problems of insufficient feature extraction capability and low segmentation accuracy in carbon monoxide gas infrared image segmentation, the encoder network and skip connection mechanism are improved respectively to obtain the improved U-Net model; S21. Encoder Network Improvement: The continuous convolutional feature extraction and pooling downsampling structure in the traditional U-Net model encoder are replaced with a ResNet50 backbone network. The ResNet50 backbone network consists of an initial feature extraction layer and four residual feature extraction stages. Each stage is composed of multiple Bottleneck residual blocks. The input information is preserved and deep feature extraction is performed through residual connections. At the same time, the spatial resolution of the feature map is reduced and the number of feature channels is increased step by step, thereby obtaining multi-level encoded features with different scales and semantic levels.
[0009] S22. Improved Skip Connection Mechanism: ECA channel attention modules are introduced into the four skip connections between the ResNet50 encoder and the U-Net decoder. The skip features at each level output by the encoder are first enhanced by ECA channel attention, and then concatenated with the decoded features upsampled in the corresponding decoding stage in the channel dimension. The ECA channel attention module establishes local cross-channel interaction relationships and adaptively adjusts the importance of each channel. It enhances the effective channel response and suppresses redundant channel information with a lower number of parameters and computational cost, thereby improving the fusion effect of encoded and decoded features.
[0010] S3. Loss Function Improvement: A composite loss function consisting of Focal Loss and Dice Loss is constructed. The Focal Loss dynamically adjusts the loss weights based on the pixel classification difficulty, reducing the loss contribution of easily classified background pixels and strengthening the constraint on difficult-to-classify gas pixels. The Dice Loss constrains the overall segmentation result of the gas region based on the degree of overlap between the predicted segmentation region and the real labeled region. The two are added together as the total loss for model training, which guides the model parameter update from both pixel classification and region overlap perspectives, thereby alleviating the class imbalance between the gas region and the background region and improving the model's ability to discriminate pixels and segment regions in gas leakage areas.
[0011] S4. Model Training and Testing: Set the training parameters of the improved U-Net model; train the improved U-Net model using the training set obtained in step S1; adjust the model parameters using the validation set; test the performance of the trained model using the test set; and output the segmentation and detection results of the carbon monoxide gas infrared image.
[0012] As a further aspect of the present invention, in S21, the specific working process of the Bottleneck residual block includes: S211. Feature extraction: The input features are sequentially processed through 1×1 convolution, 3×3 convolution and 1×1 convolution by convolution branches to perform channel compression, spatial feature extraction and channel expansion, and batch normalization and activation processing are performed after the corresponding convolution. S212. Dimension Matching: Input features are preserved through shortcut branches. When the spatial dimensions and number of channels of the input features are consistent with those of the output features of the convolution branch, the input features are directly passed. When the spatial dimensions or number of channels are inconsistent, 1×1 convolution and batch normalization are used to perform dimension matching on the input features. S213. Residual Fusion: The output features of the convolutional branch and the shortcut branch are added element by element and then activated by ReLU to obtain residual features, which enhances the ability of feature reuse and deep information transmission.
[0013] As a further aspect of the present invention, in step S21, the ResNet50 backbone network uses 7×7 convolution, batch normalization, and ReLU activation to perform initial feature extraction on the input image, and completes spatial downsampling through max pooling; the downsampled features are sequentially input into four residual feature extraction stages, each stage containing 3, 4, 6, and 3 Bottleneck residual blocks respectively, and deep feature extraction is performed through residual connections, and the spatial resolution of the feature map is gradually reduced and the number of feature channels is increased between adjacent stages; the outputs of the initial convolutional layer and the four residual feature extraction stages are obtained respectively to form five-level encoded features with different spatial resolutions and channel dimensions.
[0014] As a further embodiment of the present invention, in step S22, the ECA channel attention module includes a global average pooling layer, a one-dimensional convolutional layer, a sigmoid activation layer, and a channel weighting layer; the input coding features first obtain channel description vectors through global average pooling, then establish local interaction relationships between adjacent channels using one-dimensional convolution, and generate channel weights through sigmoid activation, and finally multiply the channel weights with the input coding features channel by channel to obtain the channel-enhanced coding features.
[0015] As a further aspect of the present invention, in step S3, the Focal Loss calculates the classification loss based on the model's prediction probability of the true category of each pixel, and introduces a difficult-to-classify sample adjustment factor to dynamically adjust the loss weight of different pixels; when the model's prediction probability of the true category of a pixel is high, the loss weight corresponding to that pixel is reduced; when the prediction probability is low, the loss weight corresponding to that pixel is increased, so as to reduce the contribution of easily classified pixels to the total loss and increase the proportion of difficult-to-classify pixels in the model parameter update.
[0016] As a further aspect of the present invention, in step S3, the Dice Loss calculates the region overlap loss based on the intersection between the model's predicted probability and the actual segmentation label, as well as the sum of their regions. By increasing the degree of overlap between the predicted segmentation region and the actual labeled region, the overall segmentation result of the carbon monoxide gas region is constrained, thereby alleviating the class imbalance problem between the gas region and the background region and improving the integrity and overlap consistency of the gas region segmentation.
[0017] As a further aspect of the present invention, in step S1, the final dataset contains 2500 infrared images of carbon monoxide gas, of which 2000 are training images, 250 are validation images, and 250 are test images.
[0018] As a further aspect of the present invention, in step S4, the average intersection-union ratio (mIoU), average pixel accuracy (mPA), precision, and recall are used as performance evaluation metrics for the model.
[0019] As a further aspect of the present invention, the hardware environment for model training and testing is as follows: CPU model is Intel(R) Xeon(R) Platinum 8474C, GPU model is NVIDIA RTX 4090D, CUDA version is 12.1, and operating system is Windows 11; the software environment is: PyTorch 2.1.0 and Python 3.10.0.
[0020] The practical benefits of this invention are as follows: Based on a self-built carbon monoxide gas infrared image dataset, this invention improves the U-Net model and its training method, effectively enhancing the model's ability to extract and segment features from low-contrast, blurred-boundary, and irregularly shaped carbon monoxide gas regions, thereby improving the accuracy of gas leak area identification and segmentation. This method can provide technical support for the monitoring and analysis of carbon monoxide gas leaks in industrial scenarios, and also has certain reference value and application value for other low-contrast gas infrared image segmentation and detection tasks. Attached Figure Description
[0021] Figure 1 This is a diagram showing the overall structure of the improved U-Net model of this invention. Figure 2 is a schematic diagram of the ResNet50 module of the present invention; Figure 3 is a schematic diagram of the structure of the ECA module of the present invention; Figure 4 is a visual comparison of the results of segmenting infrared images of carbon monoxide gas using the original U-Net and the improved U-Net of this invention. Detailed Implementation
[0022] The technical solution of the present invention will be described in detail below with reference to specific embodiments: Example 1: As Figures 1-4 As shown, a carbon monoxide gas infrared image segmentation and detection method based on an improved U-Net is disclosed, the method comprising the following steps: S1. Dataset Construction and Expansion: Considering the lack of infrared images of carbon monoxide gas with human image interference in existing public datasets, this invention uses a laboratory infrared camera to collect 16-bit infrared images of carbon monoxide gas at a certain concentration with human image interference in the background. Since 16-bit CO gas infrared images are not visible to the human eye, they are converted into 8-bit images for visualization and enhanced to highlight and strengthen the morphology and distribution characteristics of the gas. Subsequently, the Labelme tool is used to annotate the images at the pixel level, resulting in a dataset containing 2500 images, which are divided into a training set of 2000 images, a validation set of 250 images, and a test set of 250 images in an 8:1:1 ratio, providing data support for model training and performance verification.
[0023] S2. Improved U-Net Model Construction: Based on the traditional U-Net model, improvements were made to the encoder network and skip connection mechanism to address the issues of insufficient feature extraction capability and low segmentation accuracy in carbon monoxide gas infrared image segmentation. S21. Encoder Network Improvement: The continuous convolutional feature extraction and pooling downsampling structure in the traditional U-Net model encoder are replaced with a ResNet50 backbone network. The ResNet50 backbone network consists of an initial feature extraction layer and four residual feature extraction stages. Each stage is composed of multiple Bottleneck residual blocks. The input information is preserved and deep feature extraction is performed through residual connections. At the same time, the spatial resolution of the feature map is reduced and the number of feature channels is increased step by step, thereby obtaining multi-level encoded features with different scales and semantic levels.
[0024] Furthermore, in S21, the specific working process of the Bottleneck residual block includes: S211. Feature extraction: The input features are sequentially processed through 1×1 convolution, 3×3 convolution and 1×1 convolution by convolution branches to perform channel compression, spatial feature extraction and channel expansion, and batch normalization and activation processing are performed after the corresponding convolution. S212. Dimension Matching: Input features are preserved through shortcut branches. When the spatial dimensions and number of channels of the input features are consistent with those of the output features of the convolution branch, the input features are directly passed. When the spatial dimensions or number of channels are inconsistent, 1×1 convolution and batch normalization are used to perform dimension matching on the input features. S213. Residual Fusion: The output features of the convolutional branch and the shortcut branch are added element by element and then activated by ReLU to obtain residual features, which enhances the ability of feature reuse and deep information transmission.
[0025] Furthermore, traditional U-Net encoders primarily extract features through continuous convolution and max pooling, resulting in relatively limited network depth. This invention replaces the traditional U-Net encoder with a ResNet50 backbone network, utilizing residual structures to perform hierarchical feature extraction on carbon monoxide gas infrared images. The ResNet50 backbone network consists of an initial feature extraction layer and four residual feature extraction stages. The input image first undergoes 7×7 convolution, batch normalization, and ReLU activation to obtain a first-layer encoded feature with 64 channels. Then, spatial downsampling is performed through max pooling. Subsequently, the downsampled features are sequentially input into four residual feature extraction stages containing 3, 4, 6, and 3 Bottleneck residual blocks, respectively. Each Bottleneck residual block's convolutional branches sequentially employ 1×1, 3×3, and 1×1 convolutions for channel compression, spatial feature extraction, and channel expansion, and pass input features through shortcut branches. When the feature size or number of channels of the convolutional branch and the shortcut branch are inconsistent, a 1×1 convolution is used for dimension matching, and then the features of the two branches are added element-wise. Through the above processing, five levels of encoded features with 64, 256, 512, 1024, and 2048 channels are output sequentially, with spatial resolutions of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image, respectively. These multi-scale encoded features are then passed to the corresponding attention modules and decoding stages.
[0026] S22. Improved Skip Connection Mechanism: ECA channel attention modules are introduced into the skip connections between the ResNet50 encoder and the U-Net decoder. The skip features at each level output by the encoder are first enhanced by ECA channel attention, and then concatenated with the decoded features upsampled in the corresponding decoding stage in the channel dimension. The ECA channel attention module establishes local cross-channel interaction relationships and adaptively adjusts the importance of each channel. It enhances the effective channel response and suppresses redundant channel information with a lower parameter and computational cost, thereby improving the fusion effect of encoded and decoded features.
[0027] Furthermore, in S22, the ECA channel attention module includes a global average pooling layer, a one-dimensional convolutional layer, a sigmoid activation layer, and a channel weighting layer. The input encoded features first obtain channel description vectors through global average pooling, then establish local interaction relationships between adjacent channels using one-dimensional convolution, and generate channel weights through sigmoid activation. Finally, the channel weights are multiplied by the input encoded features channel by channel to obtain the channel-enhanced encoded features.
[0028] Furthermore, the multi-scale features at each level of the encoder output are enhanced using an ECA channel attention mechanism, and then concatenated with the upsampled decoded features in the channel dimension. The ECA attention mechanism obtains global descriptive information for each channel through global average pooling, then establishes local cross-channel interaction relationships using one-dimensional convolution, and generates corresponding channel weights through a Sigmoid activation function. The channel weights are multiplied with the input encoded features channel by channel to adaptively enhance the feature response related to the carbon monoxide gas region and suppress channel information corresponding to the background, human image, and other irrelevant regions.
[0029] S3. Improved Loss Function: To address the issues of high background pixel ratio, small gas region, and difficulty in accurately classifying edge pixels in carbon monoxide gas infrared images, a composite loss function consisting of Focal Loss and Dice Loss is constructed. Focal Loss dynamically adjusts the loss weight based on pixel classification difficulty, reducing the loss contribution of easily classified background pixels and strengthening the constraint on difficult-to-classify gas pixels. Dice Loss constrains the overall segmentation result of the gas region based on the overlap between the predicted segmentation region and the ground truth labeled region. The sum of the two is used as the total loss for model training, guiding model parameter updates from both pixel classification and region overlap perspectives. This alleviates the class imbalance between gas and background regions and improves the model's ability to discriminate pixels and segment regions in gas leak areas.
[0030] After the model outputs pixel-level category prediction results, Focal Loss is used to calculate the classification loss of each pixel, and Dice Loss is used to calculate the overlap loss between the predicted segmented region and the ground truth labeled region. The two losses are added together to obtain the total loss of the model training, and the network parameters are updated through backpropagation.
[0031] The Focal Loss algorithm calculates the classification error based on the predicted probability of the true class for each pixel and dynamically adjusts the loss weight of each pixel using a difficulty-sample adjustment factor. When the predicted probability of the true class is high, the loss contribution of the corresponding pixel is reduced; when the predicted probability of the true class is low, the loss contribution of the corresponding pixel is increased. Specifically, the class balance parameter α is set to 0.5, and the difficulty-sample adjustment parameter γ is set to 2, to calculate the classification loss for background pixels and carbon monoxide gas pixels respectively.
[0032] In step S3, the Dice Loss calculates the region overlap loss based on the intersection between the model's predicted probability and the actual segmentation label, as well as the sum of their regions. By increasing the degree of overlap between the predicted segmentation region and the actual labeled region, the overall segmentation result of the carbon monoxide gas region is constrained, thereby alleviating the class imbalance problem between the gas region and the background region and improving the integrity and overlap consistency of the gas region segmentation.
[0033] Furthermore, the Dice Loss first normalizes the model output to obtain the predicted probability of each pixel belonging to different categories, and compares it with the true segmentation label. It then calculates the Dice coefficient based on the intersection of the predicted and true regions and the sum of their regions, and uses the difference between 1 and the Dice coefficient as the region overlap loss. Finally, FocalLoss and Dice Loss are directly added to form a composite loss, i.e. The composite loss is used as the basis for optimizing the model parameters.
[0034] S4. Model Training and Testing: To ensure the stability of the model training process and the comparability of experimental results, the preprocessed carbon monoxide gas infrared images were uniformly adjusted to 320×256 pixels and trained using a batch size of 2 as input to the improved U-Net model. The model was trained for 100 epochs, with the Adam optimizer used to update network parameters. The initial learning rate was set to 0.0001, and the learning rate was dynamically adjusted using a cosine annealing strategy. During the initial training phase, the ResNet50 backbone network parameters were frozen, and only the ECA channel attention module, decoder, and classification output layer were trained. After the 20th epoch, the backbone network parameters were unfrozen, and the entire network was jointly trained. After each training epoch, the validation loss and various segmentation evaluation metrics were calculated using the validation set to monitor the model's training status and generalization performance, and the model weights corresponding to the lowest validation loss were saved. After training, the carbon monoxide gas infrared images to be detected were input into the trained model to obtain pixel-level segmentation results of the gas leakage area. The hardware environment for model training and testing was as follows: Intel(R) Xeon(R) Platinum 8474C CPU, NVIDIA RTX 4090D GPU, CUDA version 12.1, and Windows 11 operating system; the software environment was PyTorch 2.1.0 and Python 3.10.0. During training, the improved U-Net model was iteratively trained using the training set, and model parameters (such as learning rate and regularization coefficient) were adjusted in real time using the validation set to avoid overfitting. After training, the model performance was evaluated using the test set, with mean intersection-over-union ratio (mIoU), mean pixel accuracy (mPA), precision, and recall as evaluation metrics, outputting the segmentation and detection results of carbon monoxide gas infrared images.
[0035] The technical effects of this invention were verified through ablation experiments and comparative experiments: The ablation experiment was designed on a self-built carbon monoxide gas infrared image dataset, with 4 control experiments set up: Group 1: Original U-Net model (baseline), mIoU is 75.26, mPA is 86.81, Precision is 65, Recall is 75; Group 2: U-Net+ResNet50 structure, mIoU is 78.73, mPA is 92.3, Precision is 66, Recall is 87, proving that the ResNet50 structure can effectively improve feature extraction capability; Group 3: U-Net+ECA module, mIoU is 79.01, mPA is 88.12, Precision is 73, Recall is 78, indicating that the ECA module can effectively enhance the model's ability to focus on key feature regions while maintaining a low number of parameters and computational cost, thereby improving segmentation accuracy; Group 4: U-Net+ResNet50+ECA (improved model of this invention), mIoU is 81.97, mPA is 92.78, Precision is 73, Recall is 87, proving that the improved method adopted in this invention can effectively improve the segmentation performance of the U-Net model, and the modules have good complementarity. The improved model performs better in terms of segmentation accuracy and stability.
[0036] Furthermore, to verify the effectiveness of the hybrid loss function, the improved model of this invention selected the commonly used cross-entropy loss function CE Loss, the improved Focal Loss, and the Focal Loss+Dice Loss hybrid loss function for comparative experiments, setting up a total of 3 sets of control experiments: Group 1: CE Loss, mIoU is 78.61, mPA is 88.48, Precision is 71, Recall is 78; Group 2: Focal Loss, mIoU 80.48, mPA 91.48, Precision 71, Recall 84; Group 3: Focal Loss + Dice Loss, mIoU is 81.97, mPA is 92.78, Precision is 73, Recall is 87. Compared with the CE Loss function alone, the improved Focal Loss + Dice Loss hybrid function slightly improves the segmentation effect of this model, proving that the use of hybrid loss function is effective in the dataset of this paper.
[0037] The comparative experiments compare the improved model of this invention with traditional mainstream image segmentation algorithms (R2U-Net, U-Net++, and the original U-Net) on the same dataset, setting up 4 sets of control experiments: Group 1: Original U-Net model (baseline), mIoU is 75.26, mPA is 86.81, Precision is 65, Recall is 75; Group 2: R2U-Net, mIoU is 76.55, mPA is 90.49, Precision is 63, Recall is 83; Group 3: U-Net++, mIoU is 80.76, mPA is 89.88, Precision is 75, Recall is 81; Group 4: U-Net+ResNet50+ECA (improved model of this invention), mIoU is 81.97, mPA is 92.78, Precision is 73, and Recall is 87.
[0038] In summary, the results of the infrared image segmentation and detection of carbon monoxide gas verify the effectiveness of the improved model of this invention in feature extraction, attention modeling and loss function optimization, and it is suitable for semantic segmentation tasks with complex backgrounds and small target proportions. Figure 4 The image presents a visual comparison of the original U-Net model and the improved U-Net model of this invention in the segmentation and detection results of carbon monoxide gas in infrared images. As can be seen from the figure, the improved model can segment the carbon monoxide gas region more accurately, with clearer edges and a significant reduction in missed detections and false detections, intuitively reflecting the advantages of the method of this invention in improving segmentation accuracy and robustness.
[0039] Example 2: Figures 1-4 As shown, a carbon monoxide gas infrared image segmentation and detection method based on an improved U-Net is disclosed, the method comprising: The encoder network of the U-Net model is improved by introducing a ResNet50 module to replace the original U-Net encoder. The residual structure is used to enhance the feature extraction capability, and transfer learning is combined to accelerate model convergence and improve generalization performance. The skip connection mechanism of the model is optimized by adding an ECA attention mechanism, which enables adaptive adjustment of the importance of channel features while maintaining low parameter and computational cost. A hybrid loss function combining Focal Loss and Dice Loss is used.
[0040] The method includes the following steps: Step 1: Dataset Construction and Processing; Data acquisition equipment: Laboratory infrared camera was selected, and the shooting parameters were set as follows: resolution of 320×256, frame rate of 30fps, and focal length of 25mm. Collection environment: In an open outdoor setting, a certain concentration of carbon monoxide gas is released, and 1-2 experimental personnel are arranged to move around in the background to simulate human interference in an industrial setting; Image visualization processing: Converting 2500 initial images into 8-bit image visualizations: Image enhancement: Using traditional background subtraction methods, the image grayscale range is stretched to 0-255; Dataset partitioning: After augmentation, 2500 images were obtained and divided into a training set of 2000 images, a validation set of 250 images, and a test set of 250 images in an 8:1:1 ratio. The LabelMe tool was used to annotate the carbon monoxide gas region in all images and generate a mask file.
[0041] Improved U-Net model implementation: Step 2: Improved U-Net model construction: Based on the PyTorch 2.1.0 framework, the encoder network and skip connection mechanism of the traditional U-Net model are improved to obtain the improved U-Net model; Step 21. Encoder Network Improvement: Replace the continuous convolutional feature extraction and pooling downsampling structure in the traditional U-Net model encoder with a ResNet50 backbone network. The ResNet50 backbone network consists of an initial feature extraction layer and four residual feature extraction stages. Each stage is composed of multiple Bottleneck residual blocks. The input information is preserved and deep feature extraction is performed through residual connections. At the same time, the spatial resolution of the feature map is reduced step by step and the number of feature channels is increased, thereby obtaining multi-level encoded features with different scales and semantic levels.
[0042] Step 22. Improved Skip Connection Mechanism: In the skip connections between the ResNet50 encoder and the U-Net decoder, ECA channel attention modules are introduced. The skip features at each level output by the encoder are first enhanced by ECA channel attention, and then concatenated with the decoded features upsampled in the corresponding decoding stage in the channel dimension. The ECA channel attention module establishes local cross-channel interaction relationships and adaptively adjusts the importance of each channel. It enhances the effective channel response and suppresses redundant channel information with a lower number of parameters and computational cost, thereby improving the fusion effect of encoded and decoded features.
[0043] Furthermore, the ResNet50 implementation involves replacing the continuous convolutional feature extraction and pooling downsampling structures in the traditional U-Net model encoder with the ResNet50 backbone network, and modifying the module call code of the encoder network. A specific code snippet is shown below: Python run class ResNet(nn.Module): def __init__(self, block, layers, num_classes=1000): self.inplanes = 64 super(ResNet, self).__init__() self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2,padding=3, bias=False) self.bn1 = nn.BatchNorm2d(64) self.relu = nn.ReLU(inplace=True) self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=0, ceil_mode=True) # change self.layer1 = self._make_layer(block, 64, layers[0]) self.layer2 = self._make_layer(block, 128, layers[1], stride=2) self.layer3 = self._make_layer(block, 256, layers[2], stride=2) self.layer4 = self._make_layer(block, 512, layers[3], stride=2) self.avgpool = nn.AvgPool2d(7) self.fc = nn.Linear(512 * block.expansion, num_classes) for m in self.modules(): if isinstance(m, nn.Conv2d): n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels m.weight.data.normal_(0, math.sqrt(2. / n)) elif isinstance(m, nn.BatchNorm2d): m.weight.data.fill_(1) m.bias.data.zero_() def _make_layer(self, block, planes, blocks, stride=1): downsample = None if stride != 1 or self.inplanes != planes * block.expansion: downsample = nn.Sequential( nn.Conv2d(self.inplanes, planes * block.expansion, kernel_size=1, stride=stride, bias=False), nn.BatchNorm2d(planes * block.expansion), ) layers = [] layers.append(block(self.inplanes, planes, stride,downsample)) self.inplanes = planes * block.expansion for i in range(1, blocks): layers.append(block(self.inplanes, planes)) return nn.Sequential(*layers) def forward(self, x): x = self.conv1(x) x = self.bn1(x) feat1 = self.relu(x) x = self.maxpool(feat1) feat2 = self.layer1(x) feat3 = self.layer2(feat2) feat4 = self.layer3(feat3) feat5 = self.layer4(feat4) return [feat1, feat2, feat3, feat4, feat5] def resnet50(pretrained=False, **kwargs): model = ResNet(Bottleneck, [3, 4, 6, 3], **kwargs) if pretrained: model.load_state_dict(model_zoo.load_url('https: / / s3.amazonaws.com / pytorch / models / resnet50-19c8e357.pth', model_dir='model_data'), strict=False) del model.avgpool del model.fc return model ECA module implementation: An ECA attention mechanism module is added to the skip connection mechanism. The specific code snippet is as follows: Python run class EfficientChannelAttention(nn.Module): def __init__(self, c, b=1, gamma=2): super(EfficientChannelAttention, self).__init__() t = int(abs((math.log(c, 2) + b) / gamma)) k = t if t % 2 else t + 1 self.avg_pool = nn.AdaptiveAvgPool2d(1) self.conv1 = nn.Conv1d(1, 1, kernel_size=k, padding=int(k / 2),bias=False) self.sigmoid = nn.Sigmoid() def forward(self, x): out = self.avg_pool(x) out = self.conv1(out.squeeze(-1).transpose(-1, -2)).transpose(-1, -2).unsqueeze(-1) out = self.sigmoid(out) return out * x Step 3, Loss Function Improvement: Construct a composite loss function consisting of Focal Loss and Dice Loss. The Focal Loss dynamically adjusts the loss weights based on the pixel classification difficulty, reducing the loss contribution of easily classified background pixels and strengthening the constraint on difficult-to-classify gas pixels. The Dice Loss constrains the overall segmentation result of the gas region based on the degree of overlap between the predicted segmentation region and the real labeled region. The two are added together as the total loss for model training, guiding the model parameter update from both pixel classification and region overlap perspectives, thereby alleviating the class imbalance between the gas region and the background region and improving the model's ability to discriminate pixels and segment regions in gas leakage areas.
[0044] Step 4, Model Training and Testing: Set the training parameters of the improved U-Net model; train the improved U-Net model using the training set obtained in step S1; adjust the model parameters using the validation set; test the performance of the trained model using the test set; and output the segmentation and detection results of the carbon monoxide gas infrared image.
[0045] Furthermore, including: Model training: Training parameter settings: Input image resolution adjusted to 320×256, Batch Size=2, initial learning rate=0.0001, training epochs=100, optimizer selected as Adam; Training process: Every 5 rounds, performance is evaluated on the validation set, and the weight file with the highest mIoU on the validation set is saved as the weight file of the optimal model; Model testing: Test metrics: Calculate the mean intersection-union ratio (mIoU), mean pixel precision (mPA), precision, and recall for the test set; Test results: The improved model has an mIoU of 81.97, an mPA of 92.78, a Precision of 73, and a Recall of 87, all of which are better than the original U-Net model.
[0046] This invention improves the U-Net model and its training method based on a self-built carbon monoxide gas infrared image dataset. This effectively enhances the model's ability to extract and segment features from low-contrast, blurred-boundary, and irregularly shaped carbon monoxide gas regions, thereby increasing the accuracy of gas leak area identification and segmentation. This method can provide technical support for the monitoring and analysis of carbon monoxide gas leaks in industrial scenarios, and also has certain reference value and application value for other low-contrast gas infrared image segmentation and detection tasks.
[0047] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.
Claims
1. A carbon monoxide gas infrared image segmentation and detection method based on an improved U-Net, characterized in that, The method includes: The encoder network of the U-Net model is improved by introducing a ResNet50 module to replace the original U-Net encoder. The residual structure is used to enhance the feature extraction capability, and transfer learning is combined to accelerate model convergence and improve generalization performance. The skip connection mechanism of the model is optimized by adding an ECA attention mechanism, which enables adaptive adjustment of the importance of channel features while maintaining low parameter and computational cost. A hybrid loss function combining Focal Loss and Dice Loss is used.
2. The carbon monoxide gas infrared image segmentation and detection method based on improved U-Net according to claim 1, characterized in that, The method includes the following specific steps: S1. Dataset Construction and Processing: Infrared images of carbon monoxide gas leaks with a certain concentration and background interference from human figures were collected, converted into 8-bit images for visualization, and enhanced to highlight and strengthen the morphology and distribution characteristics of the gas. Then, the Labelme tool was used to annotate the images at the pixel level to form the final dataset, and the final dataset was divided into training set, validation set and test set according to a certain ratio. S2. Improved U-Net Model Construction: Based on the traditional U-Net model, its encoder network and skip connection mechanism are improved to obtain the improved U-Net model; S21. Encoder Network Improvement: The continuous convolutional feature extraction and pooling downsampling structure in the traditional U-Net model encoder are replaced with a ResNet50 backbone network. The ResNet50 backbone network consists of an initial feature extraction layer and four residual feature extraction stages. Each stage is composed of multiple Bottleneck residual blocks. The input information is preserved and deep feature extraction is performed through residual connections. At the same time, the spatial resolution of the feature map is reduced and the number of feature channels is increased step by step, thereby obtaining multi-level encoded features with different scales and semantic levels. S22. Improved Skip Connection Mechanism: ECA channel attention modules are introduced into the skip connections between the ResNet50 encoder and the U-Net decoder. The skip features at each level output by the encoder are first enhanced by ECA channel attention, and then concatenated with the decoded features upsampled in the corresponding decoding stage in the channel dimension. The ECA channel attention module establishes local cross-channel interaction relationships and adaptively adjusts the importance of each channel, enhancing the effective channel response and suppressing redundant channel information with low parameter and computational cost, thereby improving the fusion effect of encoded and decoded features. S3. Loss Function Improvement: Construct a composite loss function consisting of Focal Loss and Dice Loss; Focal Loss dynamically adjusts the loss weights according to the difficulty of pixel classification, reducing the loss contribution of easily classified background pixels and strengthening the constraint on difficult-to-classify gas pixels; Dice Loss constrains the overall segmentation result of the gas region based on the degree of overlap between the predicted segmentation region and the real labeled region; the two are added together as the total loss for model training. S4. Model Training and Testing: Set the training parameters of the improved U-Net model; train the improved U-Net model using the training set obtained in step S1; adjust the model parameters using the validation set; test the performance of the trained model using the test set; and output the segmentation and detection results of the carbon monoxide gas infrared image.
3. The carbon monoxide gas infrared image segmentation and detection method based on improved U-Net according to claim 2, characterized in that, In step S21, the specific working process of the Bottleneck residual block includes: S211. Feature extraction: The input features are sequentially processed through 1×1 convolution, 3×3 convolution and 1×1 convolution by convolution branches to perform channel compression, spatial feature extraction and channel expansion, and batch normalization and activation processing are performed after the corresponding convolution. S212. Dimension Matching: Input features are preserved through shortcut branches. When the spatial dimensions and number of channels of the input features are consistent with those of the output features of the convolution branch, the input features are directly passed. When the spatial dimensions or number of channels are inconsistent, 1×1 convolution and batch normalization are used to perform dimension matching on the input features. S213. Residual Fusion: The output features of the convolutional branch and the shortcut branch are added element by element and then activated by ReLU to obtain residual features, which enhances the ability of feature reuse and deep information transmission.
4. The carbon monoxide gas infrared image segmentation and detection method based on improved U-Net according to claim 2, characterized in that, In step S21, the ResNet50 backbone network uses 7×7 convolution, batch normalization, and ReLU activation to extract initial features from the input image, and performs spatial downsampling through max pooling. The downsampled features are then sequentially input into four residual feature extraction stages, each containing 3, 4, 6, and 3 Bottleneck residual blocks, respectively. Deep feature extraction is performed through residual connections, and the spatial resolution of the feature map is gradually reduced and the number of feature channels is increased between adjacent stages. The outputs of the initial convolutional layer and the four residual feature extraction stages are obtained respectively, forming five-level encoded features with different spatial resolutions and channel dimensions.
5. The carbon monoxide gas infrared image segmentation and detection method based on improved U-Net according to claim 2, characterized in that, In step S22, the ECA channel attention module includes a global average pooling layer, a one-dimensional convolutional layer, a sigmoid activation layer, and a channel weighting layer. The input encoded features first obtain channel description vectors through global average pooling, then establish local interaction relationships between adjacent channels using one-dimensional convolution, and generate channel weights through sigmoid activation. Finally, the channel weights are multiplied by the input encoded features channel by channel to obtain the channel-enhanced encoded features.
6. The carbon monoxide gas infrared image segmentation and detection method based on improved U-Net according to claim 2, characterized in that, In step S3, the Focal Loss calculates the classification loss based on the model's prediction probability of the true class of each pixel, and introduces a difficult-to-classify sample adjustment factor to dynamically adjust the loss weight of different pixels; when the model's prediction probability of the true class of a pixel is high, the loss weight corresponding to that pixel is reduced; when the prediction probability is low, the loss weight corresponding to that pixel is increased, so as to reduce the contribution of easily classified pixels to the total loss and increase the proportion of difficult-to-classify pixels in the model parameter update.
7. The carbon monoxide gas infrared image segmentation and detection method based on improved U-Net according to claim 2, characterized in that, In step S3, the Dice Loss calculates the region overlap loss based on the intersection between the model's predicted probability and the actual segmentation label, as well as the sum of their regions. By increasing the degree of overlap between the predicted segmentation region and the actual labeled region, the overall segmentation result of the carbon monoxide gas region is constrained, thereby alleviating the class imbalance problem between the gas region and the background region and improving the integrity and overlap consistency of the gas region segmentation.
8. The carbon monoxide gas infrared image segmentation and detection method based on improved U-Net according to claim 2, characterized in that, In S1, the final dataset contains 2,500 infrared images of carbon monoxide gas, including 2,000 images for training, 250 images for validation, and 250 images for testing.
9. The carbon monoxide gas infrared image segmentation and detection method based on improved U-Net according to claim 2, characterized in that, In S4, the average intersection-union ratio (mIoU), average pixel accuracy (mPA), precision, and recall are used as performance evaluation metrics for the model.
10. The carbon monoxide gas infrared image segmentation and detection method based on improved U-Net according to claim 2, characterized in that, The hardware environment for model training and testing is as follows: CPU model is Intel(R) Xeon(R) Platinum 8474C, GPU model is NVIDIA RTX 4090D, CUDA version is 12.1, and operating system is Windows 11; the software environment is PyTorch 2.1.0 and Python 3.10.0.
