Temperature measuring rod segmentation and intelligent filling method and device for blast furnace burden surface infrared image

Abstract

The invention discloses a temperature measuring rod segmentation and intelligent filling method and device for a blast furnace burden surface infrared image. The method comprises the following steps: acquiring an infrared image data set of a blast furnace burden surface through an infrared imager, and marking a temperature measuring rod area in the data set; training a semantic segmentation model constructed based on DeepLabv3 + by using the annotation data set, so that the model can accurately segment shielding areas such as a temperature measurement rod and the like and generate corresponding masks; and in combination with a LaMa generation type repair model, filling a temperature measuring rod area according to pixel characteristics around the mask, and recovering material surface texture information to obtain a complete blast furnace material surface infrared image. The problems that a traditional method is low in segmentation precision and poor in repairing effect are solved, the quality and the visual effect of the blast furnace burden surface infrared image are remarkably improved, and applicability and reliability are achieved.

Description

Technical Field

[0001] This invention relates to the field of semantic segmentation and intelligent filling technology for blast furnace burden surface, and in particular to a method and apparatus for segmenting and intelligently filling infrared images of blast furnace burden surface using temperature measuring rods. Background Technology

[0002] Blast furnace ironmaking, as a core link in the modern steel industry, generates molten iron through the layering of coke, ore, and flux, and a high-temperature reduction reaction. Real-time monitoring and control of the charge surface shape has a decisive impact on the distribution of gas flow, heat transfer efficiency, and the uniformity of chemical reactions within the furnace, directly affecting the blast furnace's energy consumption, production efficiency, and molten iron quality. Traditional monitoring methods, such as mechanical probes and radar probes, are limited by the harsh environment of high temperature, high pressure, and high dust levels inside the blast furnace, making it difficult to obtain continuous, high-quality three-dimensional information on the charge surface. Infrared imaging technology, with its advantages of non-contact operation, strong anti-interference capabilities, and ability to penetrate dust, has become an important technical means for monitoring the blast furnace charge surface. By acquiring infrared images of the blast furnace charge surface using infrared cameras and analyzing temperature distribution, combined with three-dimensional reconstruction algorithms, the morphology of the charge surface is reconstructed, providing crucial information for optimizing the charge distribution system.

[0003] However, in practical applications, the temperature measuring rods deployed on the furnace top appear as slender metal structures in infrared images of the blast furnace burden surface. Their high reflectivity severely obscures the infrared radiation information of the burden surface in the area they cover. The loss of depth information in the obscured area leads to distortions in 3D reconstruction, such as depressions or faults, and can also mask local overheating or cooling anomalies, resulting in incorrect temperature field reconstruction and severely limiting the practicality of the monitoring system. Although image segmentation and restoration technologies have made progress in many fields, they still face significant bottlenecks in blast furnace burden surface infrared image processing.

[0004] In the process of realizing this invention, the inventors discovered at least the following problems in the prior art: Traditional threshold segmentation and edge detection methods struggle to distinguish the low-contrast boundaries between temperature measuring rods and coke blocks, especially at the junction of high-temperature and low-temperature zones, where missegmentation is common. Morphological operations are poorly adapted to slender, closely spaced temperature measuring rods. General-purpose deep learning models such as U-Net and Mask R-CNN have poor segmentation capabilities for small or slender objects and insufficient global contextual information capture, making them unsuitable for the complex scenario of blast furnace material surfaces. Existing image inpainting methods such as GAN and PatchMatch often generate pseudo-textures or temperature jump regions, disrupting the consistency of the material surface morphology. Furthermore, simple interpolation results in harsh inpainting boundaries and step-like distortions in 3D reconstruction.

[0005] Deeplabv3+ is designed specifically for semantic segmentation tasks, combining dilated convolution with depthwise separable convolution to provide powerful global context modeling capabilities. It can effectively extract features at different scales, exhibiting strong segmentation capabilities for small objects and complex scenes, and is suitable for infrared images of blast furnace burden surfaces. Furthermore, it is computationally efficient and resource-saving, and its improved encoder structure enhances segmentation accuracy. The LaMa model, through its latent space representation and self-supervised learning, maintains higher image quality during image inpainting, particularly in detail restoration, resulting in sharper edges, more natural textures, and stronger generalization ability. Summary of the Invention

[0006] In view of this, this application provides a method for segmenting and intelligently filling infrared images of blast furnace burden surfaces using temperature measuring rods, in order to solve the technical problems of insufficient segmentation accuracy and poor image filling and repair effects in related technologies.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: According to a first aspect of the present invention, a method for segmenting and intelligently filling temperature measuring rods in infrared images of blast furnace burden surfaces is provided, comprising: Collect infrared image datasets of blast furnace burden surface, expand the dataset with additional samples, and mark the locations of the temperature measuring rod areas; An infrared image semantic segmentation model was built based on Deeplabv3+. The model was trained using an annotated dataset to achieve accurate segmentation of the area obscured by the temperature measuring rod in the infrared image of the blast furnace burden surface and output a binary mask image. Define a mask generation method that conforms to the characteristics of the temperature measuring rod, fine-tune the LaMa pre-trained model, and perform generative filling and repair on the mask area in the binary mask image based on the fine-tuned LaMa model to restore the material surface texture and output a complete infrared image of the blast furnace material surface. The complete infrared image of the blast furnace burden surface is smoothed to obtain a high-quality infrared image of the blast furnace burden surface without interference from the temperature measuring rod.

[0008] Optionally, the dataset is augmented with additional samples, and the locations of the temperature measuring rod areas are labeled, including: Data augmentation techniques, including rotation, translation, and noise addition, were used to augment the dataset. The temperature measuring rod area is finely annotated pixel by pixel using a polygon annotation tool to obtain the annotated dataset.

[0009] Optionally, an infrared image semantic segmentation model can be built based on Deeplabv3+, including: A lightweight encoder-decoder architecture is constructed, integrating a dilated spatial pyramid pooling module to enhance multi-scale feature extraction capabilities. The encoder part replaces the original Xception backbone network with a lightweight MobileNetV2, introducing multi-scale dilated convolutions to expand the receptive field and extract low-level image features. These low-level features are divided into two parts: one part is directly fed into the decoder, and the other part is fed into the dilated spatial pyramid pooling module. The dilated spatial pyramid pooling module includes four dilated convolutional blocks with different dilation rates and a global average pooling block. The dilated spatial pyramid pooling module... The pyramid pooling module extracts contextual information from the image at different scales, concatenates five sets of feature maps by channel dimension, and performs dimensionality compression and fusion processing through convolution to form a high semantic feature map, which is then fed into the decoder. The decoder receives the low-level features of the backbone network and the high-level semantic feature map of the hollow space pyramid pooling module as input. It performs dimensionality reduction on the low-level features through convolution, then upsamples the high-level features to the same spatial size and concatenates them with the low-level features by channel. The fused features are extracted through two layers of convolution and then upsampled again to the original image resolution. Finally, the temperature measuring rod mask is output through a softmax layer.

[0010] Optionally, the semantic segmentation model is trained using the labeled dataset to accurately segment the area obscured by the temperature measuring rod in the infrared image of the blast furnace burden surface, and output a binary mask image, including: The semantic segmentation model is trained using the labeled dataset, with weighted cross-entropy loss as the optimization objective, and higher weights are assigned to the edge pixels of the temperature measuring rod. An early stopping strategy is used to obtain a semantic segmentation model with convergent loss function. The infrared image of the blast furnace surface is input into the trained model to obtain the mask corresponding to the temperature measuring rod occlusion area. The mask of the temperature measuring rod occlusion area output by the semantic segmentation model is subjected to morphological dilation to expand the mask coverage to a predetermined number of pixels outside the occlusion area.

[0011] Optionally, the weighted cross-entropy loss function is as follows: in, This represents the total number of pixels in the image. Based on the number of categories, the task scene is divided into two categories: background and temperature measuring rod. Category weights; Indicates the first Does the pixel belong to the ? kind, The model predicts the first... The pixel is the first The probability of a class; This indicates that a higher weight is assigned to certain pixels.

[0012] Optionally, define a mask generation method that matches the characteristics of the temperature measuring rod, and fine-tune the LaMa pre-trained model, including: A mask generation strategy conforming to the shape characteristics of the temperature measuring rod is defined. A narrow quadrilateral region mask with a set width of pixels is randomly generated. The generated mask is then fused with the infrared dataset of the blast furnace burden surface to construct a training sample set containing both occlusion and target texture. Let the original image be... The mask is inverted and multiplied with the infrared image of the blast furnace burden surface to obtain an infrared image of the blast furnace burden surface with a mask; the infrared image of the blast furnace burden surface with a mask and the mask are spliced ​​together according to the channel dimension to form a four-channel input tensor. Using the aforementioned training sample set, the LaMa pre-trained model was fine-tuned. During fine-tuning, a self-supervised training method was employed, with loss functions including perceptual loss and adversarial loss to enhance the realism and continuity of the recovered texture. The perceptual loss function is as follows: in, It is an element-wise operation. It is a sequential two-stage averaging operation, namely, the inter-level mean of the intra-level mean. This refers to a base model with a high receptive field, specifically employing Fourier or dilated convolution operations. The adversarial loss function is as follows:

[0013] Optionally, generative filling repair can be performed on the masked region based on the fine-tuned LaMa model. The filling steps are as follows: After the image is input, it is first compressed in spatial dimension by the downsampling module of the LaMa model; After downsampling, the input feature tensor is processed by a Fast Fourier Convolution module. This module divides the input feature tensor into two branches based on the channels. One branch uses regular convolution operations to extract the local spatial features of the image. Another branch extracts global contextual features of the image based on Fourier transform operations. This enables the modeling and understanding of the entire image; The results from the two branches are cross-fused to enhance the performance of the restored image in terms of edge continuity and global consistency. The fused features are stitched along the channel dimension and gradually restored to the original image resolution through the upsampling module.

[0014] Optionally, the smoothing process includes: The complete infrared image of the blast furnace burden surface is subjected to bilateral filtering to preserve edge sharpness while eliminating high-frequency noise. The filtering parameters are dynamically adjusted according to the temperature. The formula for bilateral filtering is as follows: ; ; ; ; in, pixels in the original image pixel values, For pixels in the filtered image pixel values, For pixels The neighboring region and These are the range weight function and the spatial domain weight function, respectively. As the normalization factor, and The influence range of pixel differences and spatial variation is controlled separately, defined according to temperature as follows: ; ; in, This is the temperature scaling factor. For pixels The corresponding temperature This represents the highest temperature in the infrared image of the blast furnace charge surface.

[0015] According to a second aspect of the present invention, a temperature measuring rod segmentation and intelligent filling device for infrared images of blast furnace burden surface is provided, comprising: The data acquisition and processing module is used to acquire infrared image datasets of blast furnace burden surface, expand the datasets, and mark the locations of the temperature measuring rod areas; The semantic segmentation module is used to build an infrared image semantic segmentation model based on Deeplabv3+. The semantic segmentation model is trained using an annotated dataset to achieve accurate segmentation of the area obscured by the temperature measuring rod in the infrared image of the blast furnace burden surface and output a binary mask image. The fill and repair module is used to define a mask generation method that conforms to the characteristics of the temperature measuring rod, fine-tune the LaMa pre-trained model, and perform generative fill and repair on the mask area in the binary mask image based on the fine-tuned LaMa model to restore the material surface texture and output a complete infrared image of the blast furnace material surface. The post-processing module smooths the complete infrared image of the blast furnace burden surface to obtain a high-quality infrared image of the blast furnace burden surface without interference from the temperature measuring rod.

[0016] According to a third aspect of the present invention, an electronic device is provided, comprising: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in the first aspect.

[0017] The technical solutions provided by the embodiments of this application may include the following beneficial effects: The segmentation, removal, and restoration of occluded objects such as temperature measuring rods in infrared images of blast furnace burden surfaces are crucial for the accurate acquisition of 3D information about the burden surface. This invention proposes a method for segmenting and intelligently restoring temperature measuring rods in infrared images of blast furnace burden surfaces. By integrating deep learning and morphological operations, it overcomes the challenge of occlusion interference in complex industrial scenarios. A semantic segmentation network is built based on DeepLabv3+, achieving pixel-level accurate recognition of the temperature measuring rod region. This overcomes the issues of missed detections and misjudgments in the segmentation of low-contrast, elongated structures by traditional algorithms, improving mask generation accuracy and laying a reliable foundation for subsequent image restoration. By customizing the mask generation mode and fine-tuning the LaMa pre-trained model, the restoration results significantly outperform other restoration methods in terms of texture continuity and visual consistency. The coke block morphology and texture of the restored area seamlessly blend with the surrounding burden surface, avoiding interference from artifacts and pixel abrupt changes in 3D reconstruction. Furthermore, combined with dynamically optimized bilateral filtering, the edges of the restored image are smooth and natural. Despite the obstruction and influence of the temperature measuring rod inside the furnace, the temperature measuring rod segmentation and intelligent filling method provided by this invention can obtain high-quality infrared images of the blast furnace material surface, ensuring the accuracy of subsequent three-dimensional reconstruction and improving production efficiency and product quality. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a method for segmenting and intelligently filling infrared images of blast furnace burden surfaces using temperature measuring rods, according to an exemplary embodiment.

[0019] Figure 2 This is a clearly imaged infrared schematic diagram of the blast furnace charge surface according to an exemplary embodiment.

[0020] Figure 3 It is a mask image of the temperature measuring rod region output according to a semantic segmentation model shown in an example embodiment.

[0021] Figure 4 This is a diagram showing the complete blast furnace charge surface after filling and repair, according to an example embodiment.

[0022] Figure 5 This is a block diagram illustrating a temperature measuring rod segmentation and intelligent filling device for infrared images of blast furnace burden surface according to an exemplary embodiment.

[0023] Figure 6 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0025] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0026] Figure 1 This is a flowchart illustrating a method for segmenting and intelligently filling infrared images of blast furnace burden surfaces using temperature measuring rods, according to an exemplary embodiment. Figure 1 As shown, this method, when applied to a terminal, may include the following steps: In step S1, an infrared image dataset of the blast furnace charge surface is acquired, the dataset is expanded, and the location of the temperature measuring rod area is marked. Specifically, an infrared imager is deployed at the top of the blast furnace. By changing the way the furnace charge is piled up, images are acquired to obtain infrared images of the blast furnace charge surface with different temperature distributions. Changing the position of the infrared imager also allows for the acquisition of infrared images of the blast furnace charge surface from different viewpoints, increasing the diversity of the dataset. (Reference) Figure 2 The image shown is an example of an infrared image of the blast furnace burden surface captured by an infrared imager.

[0027] Expanding the dataset involves a series of data processing operations, which may include the following sub-steps: (1) Data augmentation techniques such as rotation, translation, and noise addition were used to expand the sample; Specifically, since the number and diversity of images acquired by changing the furnace charge stacking method and the position of the infrared imager are limited, data augmentation is performed to expand the sample. Random rotation operation was performed on the infrared image dataset of blast furnace burden surface, with an angle range of... Step length Perform translation operations, randomly translating the image width or height by ±10% in both horizontal and vertical directions; add Gaussian noise to the blast furnace burden infrared image dataset, and randomly generate black spots with a diameter of 3-5 pixels to simulate blast furnace dust, with a random dotting density of 5%; (2) The temperature measuring rod area is finely annotated pixel by pixel using a polygon annotation tool to obtain the annotated dataset; Specifically, the LabelMe annotation tool is used to continuously annotate vertices along the edge of the temperature measuring rod, forming a closed polygonal mask area.

[0028] In step S2, an infrared image semantic segmentation model is constructed based on Deeplabv3+. The semantic segmentation model is trained using the labeled dataset to achieve accurate segmentation of the temperature measuring rod occlusion area in the infrared image of the blast furnace material surface and output a binary mask image. Specifically, the infrared image semantic segmentation model built based on Deeplabv3+ includes: A lightweight encoder-decoder architecture is constructed, integrating a dilated spatial pyramid pooling module to enhance multi-scale feature extraction capabilities. The encoder uses the lightweight MobileNetV2 as its backbone network, replacing the original Xception backbone to reduce model parameters and improve computational efficiency. For infrared images, standard convolutions are replaced with dilated convolutions to expand the receptive field. The input image first passes through the backbone network to extract low-level features, which are then divided into two parts: one part is directly fed into the decoder to preserve spatial location information, and the other part is input into the dilated spatial pyramid pooling module to enhance semantic information extraction. The dilated spatial pyramid pooling module includes four dilated convolutional blocks with different dilation rates and a global average pooling block. The module extracts contextual information of the image at different scales, concatenates the five feature maps along the channel dimension, and uses a 1×1... Convolutional processing performs dimensionality compression and fusion to form a high semantic feature map, which is then fed into the decoder. The decoder takes low-level features extracted from the backbone network and high-level semantic feature maps from the hollow spatial pyramid pooling module as input. The low-level features are compressed using 1×1 convolution (reduced from 256 to 48 channels) to reduce channel redundancy when fused with high-level features. Then, the high-level semantic feature map is upsampled to the same spatial size as the low-level features through bilinear interpolation. The two are then concatenated through channels to form a fused feature map. The joint features are further extracted through two consecutive 3×3 convolutions, and then upsampled again to restore the same resolution as the original image. Finally, the segmentation prediction map, i.e., the mask of the temperature measuring rod area, is output through a softmax layer.

[0029] The semantic segmentation model is trained using the labeled dataset to accurately segment the area obscured by the temperature measuring rod in the infrared image of the blast furnace burden surface and output a binary mask image. Specifically, when training the semantic segmentation model, a weighted cross-entropy loss function is used to assign different loss weights to pixels of different categories in the samples. The weight of the temperature measuring rod category is increased, while the weight of the background category is decreased to alleviate the category imbalance problem and improve the accuracy of the segmentation boundaries. Furthermore, considering that edge pixels of the temperature measuring rod often have blurred and transitional features, an edge enhancement strategy is introduced to further improve boundary recognition accuracy. By adding edge weights to edge pixels within a 2-pixel range of the mask's outer edge (specifically set to 1.5 to 2 times that of ordinary pixels), the model is guided to learn more accurate edge localization capabilities. The weighted cross-entropy loss function is as follows: in, This represents the total number of pixels in the image. To determine the number of categories, the task scenario is divided into two categories: background and temperature measuring rod. Therefore, [the following is a list of categories]. Set to 2; To emphasize the importance of the temperature measuring rod, the weight of the temperature measuring rod class is determined by category weight. Set to 2, background class weight Set to 1; Indicates the first Does the pixel belong to the ? kind, The model predicts the first... The pixel is the first The probability of a class; This indicates that a higher weight is assigned to certain pixels (pixels at the edge of the temperature measuring rod).

[0030] Using an early stopping strategy, a semantic segmentation model with convergent loss function is obtained. The infrared image of the blast furnace charge surface is input into the trained model to obtain the mask corresponding to the area blocked by the temperature measuring rod. Specifically, to prevent overfitting during model training and improve the model's generalization ability, this invention employs an early stopping strategy to automatically terminate training when the performance on the validation set stops improving. During model training, after each iteration (epoch), i.e., after completing one full training cycle using the training set, the validation loss is calculated on the validation set. Record the minimum value of the validation loss. And the epoch number at which this value is reached. If the validation loss does not decrease again within P consecutive epochs, the training process is stopped, and the model parameters of the epoch corresponding to the minimum validation loss are saved.

[0031] After training is complete, load The corresponding model parameters are input into the trained model using an infrared image of the blast furnace charge surface. The model then performs forward inference to obtain a prediction of the temperature measuring rod area, which is output as a mask. This mask is a single-channel binary image. (Refer to...) Figure 3 The image shown is the mask image of the temperature measuring rod area output by the model.

[0032] The mask of the temperature measuring rod occlusion area output by the semantic segmentation model is subjected to morphological dilation operation to expand the mask coverage to a predetermined number of pixels outside the occlusion area.

[0033] Specifically, considering that the semantic segmentation model may have slight deviations in edge localization, resulting in some temperature measuring rod pixels not being fully labeled, in order to avoid edge residue during the repair process, a morphological dilation operation is performed on the mask image using a 7×7 rectangular kernel to expand the occluded area by 3~5 pixels, thereby ensuring that the repair mask covers the complete occluded area.

[0034] In step S3, a mask generation method that conforms to the characteristics of the temperature measuring rod is defined, the LaMa pre-trained model is fine-tuned, and the mask area in the binary mask image is generated and repaired based on the fine-tuned LaMa model to restore the material surface texture and output a complete infrared image of the blast furnace material surface. Specifically, to more accurately simulate the occlusion area formed by the temperature measuring rod in the infrared image of the blast furnace burden, a mask generation method conforming to its geometric characteristics is defined. The temperature measuring rod typically appears as a slender, regular, rectangular occlusion area. Therefore, in this embodiment, a set of slender quadrilateral masks is generated using a random sampling strategy, with a width of approximately 10 to 15 pixels and a length dynamically adjusted according to the image size and occlusion ratio. This mask shape conforms to the occlusion pattern of the temperature measuring rod, which is beneficial for the model to learn the corresponding texture structure and recover features during the fine-tuning stage.

[0035] The generated mask is denoted as The generated mask is fused with the blast furnace burden infrared dataset to construct a training sample set containing occlusion and target texture. The mask is then inverted, setting the masked areas to 0 and the other areas to 1, and multiplied pixel-by-pixel with the original blast furnace burden infrared image. Let the original image be... The infrared image of the blast furnace charge surface with a mask is obtained, and is represented as The occluded image and the mask The input tensor is constructed by concatenating the components along the channel dimension. The calculation formula is as follows: Here, stack(·) represents the operation of concatenating according to the channel dimension.

[0036] Using the aforementioned training sample set, the LaMa pre-trained model was fine-tuned through a transfer learning strategy. Self-supervised training was employed during the fine-tuning process, and the loss function included perceptual loss and adversarial loss to enhance the realism and continuity of the recovered texture. The perceptual loss function is as follows: in, It is an element-wise operation. It is a sequential two-stage averaging operation, namely, the inter-level mean of the intra-level mean. This refers to a base model with a high receptive field, specifically employing Fourier or dilated convolution operations. The adversarial loss function is as follows:

[0037] Generative filling and repair of the masked region is performed based on the fine-tuned LaMa model. The specific filling steps are as follows: (1) After the image is input, the spatial dimension of the input image is compressed first through the downsampling module of the LaMa model; (2) After downsampling, the input feature tensor is processed by a fast Fourier convolution module. This module divides the input feature tensor into two branches based on the channels. One branch uses conventional convolution operations to extract the local spatial features of the image. In particular, it extracts detailed information such as edges and textures; another branch extracts global contextual features of the image based on Fourier transform operations. This enables the modeling and understanding of the entire image;

[0038] refer to Figure 4 The image shows an infrared image of the complete blast furnace charge surface after filling and repair. The repaired area performs well in terms of visual continuity and texture naturalness, meeting the requirements of the subsequent 3D reconstruction module for the integrity and consistency of the input image.

[0039] In step S4, the complete infrared image of the blast furnace charge surface is smoothed to obtain a high-quality infrared image of the blast furnace charge surface without interference from the temperature measuring rod. Specifically, the filled and repaired area is smoothed using a bilateral filtering method. This method combines spatial similarity and pixel value similarity to avoid edge blurring during the smoothing process. The filtered pixel value is calculated as a weighted average of its neighboring pixel values, with the weights considering both spatial distance and pixel value differences. This preserves edge sharpness while eliminating high-frequency noise, improving image quality. The filtering parameters are dynamically adjusted according to temperature. The formula for bilateral filtering is as follows: in, pixels in the original image pixel values, For pixels in the filtered image pixel values, For pixels The neighboring region and These are the range weight function and the spatial domain weight function, respectively. and Control the influence range of pixel differences and spatial differences respectively. This is the normalization factor.

[0040] Usually smaller This causes the filter to only consider nearby pixels, larger ones. This will cause the filter to consider more pixels with significant grayscale differences. Combining the infrared image characteristics of the blast furnace burden surface, the higher the temperature, the less texture is present on the surface. Therefore, it is necessary to strictly limit pixel color differences and to ensure that a smaller range of pixels participates in smoothing to preserve detail and avoid blurring. and The formula varies dynamically with temperature, as shown below: in, This is the temperature scaling factor. The temperature corresponding to pixel p. This represents the highest temperature in the infrared image of the blast furnace charge surface.

[0041] As demonstrated by the above embodiments, accurate temperature measuring rod segmentation and intelligent filling repair, as a key preprocessing step in the 3D reconstruction of blast furnace burden surface, directly determines the authenticity and reliability of the burden surface morphology restoration, and has important guiding significance for blast furnace burden optimization and gas flow control. This invention employs a deep learning-based semantic segmentation method, replacing the original Xception backbone network with the lightweight MobileNetV2, and incorporating multi-scale dilated convolutions to overcome the problem of poor segmentation performance for slender targets by traditional segmentation methods, while reducing computational complexity. A mask generation mode tailored to the shape characteristics of the temperature measuring rod is designed, making the trained repair model more adaptable to downstream tasks and improving the repair quality of areas obscured by the temperature measuring rod. Temperature-adaptive bilateral filtering and Poisson fusion techniques eliminate artifacts caused by traditional repair methods while preserving texture details such as edges, further improving the quality of the repaired image. The temperature measuring rod segmentation and intelligent filling method provided by this invention can accurately identify and remove the temperature measuring rod, obtaining a complete infrared image of the blast furnace burden surface without obstruction, laying a solid foundation for subsequent 3D reconstruction of the burden surface.

[0042] Corresponding to the aforementioned embodiments of the method for segmenting and intelligently filling infrared images of blast furnace burden surfaces, this application also provides embodiments of a device for segmenting and intelligently filling infrared images of blast furnace burden surfaces.

[0043] Figure 5 This is a block diagram illustrating a temperature measuring rod segmentation and intelligent filling device for infrared images of blast furnace burden surface, according to an exemplary embodiment. (Refer to...) Figure 5 The device includes: Data acquisition and processing module 1 is used to acquire infrared image datasets of blast furnace material surface, expand the datasets, and mark the positions of the temperature measuring rod areas; Semantic segmentation module 2 is used to build an infrared image semantic segmentation model based on Deeplabv3+. The semantic segmentation model is trained using the labeled dataset to achieve accurate segmentation of the temperature measuring rod occlusion area in the infrared image of the blast furnace burden and output a binary mask image. The filling and repair module 3 is used to define a mask generation method that conforms to the characteristics of the temperature measuring rod, fine-tune the LaMa pre-trained model, and perform generative filling and repair on the mask area in the binary mask image based on the fine-tuned LaMa model to restore the material surface texture and output a complete infrared image of the blast furnace material surface. Post-processing module 4 smooths the complete infrared image of the blast furnace burden surface to obtain a high-quality infrared image of the blast furnace burden surface without interference from the temperature measuring rod.

[0044] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0045] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0046] Accordingly, this application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-described method for segmenting and intelligently filling temperature measuring rods in infrared images of blast furnace burden surfaces. Figure 6 The diagram shown is a hardware structure diagram of any device with data processing capabilities, including a temperature measuring rod segmentation and intelligent filling device for infrared images of blast furnace burden surface provided by an embodiment of the present invention. (Except for...) Figure 6 In addition to the processor and memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0047] Accordingly, this application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the above-described method for segmenting and intelligently filling temperature measuring rods in infrared images of blast furnace burden surfaces. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.

[0048] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0049] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for segmenting and intelligently filling infrared images of blast furnace burden surfaces using temperature measuring rods, characterized in that, include: Collect infrared image datasets of blast furnace burden surface, expand the dataset with additional samples, and mark the locations of the temperature measuring rod areas; An infrared image semantic segmentation model was built based on Deeplabv3+. The model was trained using an annotated dataset to achieve accurate segmentation of the area obscured by the temperature measuring rod in the infrared image of the blast furnace burden surface and output a binary mask image. Define a mask generation method that conforms to the characteristics of the temperature measuring rod, fine-tune the LaMa pre-trained model, and perform generative filling and repair on the mask area in the binary mask image based on the fine-tuned LaMa model to restore the material surface texture and output a complete infrared image of the blast furnace material surface. The complete infrared image of the blast furnace burden surface is smoothed to obtain a high-quality infrared image of the blast furnace burden surface without interference from the temperature measuring rod.

2. The method according to claim 1, characterized in that, The dataset is augmented with additional samples, and the locations of the temperature measuring rod areas are labeled, including: Data augmentation techniques, including rotation, translation, and noise addition, were used to augment the dataset. The temperature measuring rod area is finely annotated pixel by pixel using a polygon annotation tool to obtain the annotated dataset.

3. The method according to claim 1, characterized in that, An infrared image semantic segmentation model based on Deeplabv3+ is constructed, including: A lightweight encoder-decoder architecture is constructed, integrating a dilated spatial pyramid pooling module to enhance multi-scale feature extraction capabilities. The encoder part replaces the original Xception backbone network with a lightweight MobileNetV2, introducing multi-scale dilated convolutions to expand the receptive field and extract low-level image features. These low-level features are divided into two parts: one part is directly fed into the decoder, and the other part is fed into the dilated spatial pyramid pooling module. The dilated spatial pyramid pooling module includes four dilated convolutional blocks with different dilation rates and a global average pooling block. The dilated spatial pyramid pooling module... The pyramid pooling module extracts contextual information from the image at different scales, concatenates five sets of feature maps by channel dimension, and performs dimensionality compression and fusion processing through convolution to form a high semantic feature map, which is then fed into the decoder. The decoder receives the low-level features of the backbone network and the high-level semantic feature map of the hollow space pyramid pooling module as input. It performs dimensionality reduction on the low-level features through convolution, then upsamples the high-level features to the same spatial size and concatenates them with the low-level features by channel. The fused features are extracted through two layers of convolution and then upsampled again to the original image resolution. Finally, the temperature measuring rod mask is output through a softmax layer.

4. The method according to claim 1, characterized in that, The semantic segmentation model is trained using the labeled dataset to accurately segment the area obscured by the temperature measuring rod in the infrared image of the blast furnace burden surface, and outputs a binary mask image, including: The semantic segmentation model is trained using the labeled dataset, with weighted cross-entropy loss as the optimization objective, and higher weights are assigned to the edge pixels of the temperature measuring rod. An early stopping strategy is used to obtain a semantic segmentation model with convergent loss function. The infrared image of the blast furnace surface is input into the trained model to obtain the mask corresponding to the temperature measuring rod occlusion area. The mask of the temperature measuring rod occlusion area output by the semantic segmentation model is subjected to morphological dilation to expand the mask coverage to a predetermined number of pixels outside the occlusion area.

5. The method according to claim 4, characterized in that, The weighted cross-entropy loss function is as follows: ； in, This represents the total number of pixels in the image. Based on the number of categories, the task scene is divided into two categories: background and temperature measuring rod. Category weights; Indicates the first Does the pixel belong to the ? kind, The model predicts the first... The pixel is the first The probability of a class; This indicates that a higher weight is assigned to certain pixels.

6. The method according to claim 1, characterized in that, Define a mask generation method that matches the characteristics of the temperature measuring rod, and fine-tune the LaMa pre-trained model, including: A mask generation strategy conforming to the shape characteristics of the temperature measuring rod is defined. A narrow quadrilateral region mask with a set width of pixels is randomly generated. The generated mask is fused with the infrared dataset of blast furnace burden surface to construct a training sample set containing occlusion and target texture. The mask is inverted and multiplied with the infrared image of blast furnace burden surface to obtain the infrared image of blast furnace burden surface with mask. The infrared image of blast furnace burden surface with mask and the mask are concatenated according to the channel dimension to form a four-channel input tensor. The LaMa pre-trained model was fine-tuned using the training sample set. During the fine-tuning process, a self-supervised training method was adopted, and the loss function included perceptual loss and adversarial loss to enhance the realism and continuity of the recovered texture.

7. The method according to claim 1, characterized in that, Generative filling and repair of the mask region based on the fine-tuned LaMa model includes: After the image is input, it is first compressed in spatial dimension by the downsampling module of the LaMa model; After downsampling, the input feature tensor is processed by a Fast Fourier Convolution module. This module divides the input feature tensor into two branches based on the channels. One branch uses conventional convolution operations to extract local spatial features of the image; the other branch uses Fourier transform operations to extract global contextual features of the image, thereby achieving overall modeling and understanding of the image. The results from the two branches are cross-fused to enhance the performance of the restored image in terms of edge continuity and global consistency. The fused features are stitched along the channel dimension and gradually restored to the original image resolution through the upsampling module.

8. The method according to claim 1, characterized in that, The smoothing process includes: The complete infrared image of the blast furnace burden surface is subjected to bilateral filtering to preserve edge sharpness while eliminating high-frequency noise. The filtering parameters are dynamically adjusted according to the temperature. The formula for bilateral filtering is as follows: ； ； ； ； in, pixels in the original image pixel values, For pixels in the filtered image pixel values, For pixels The neighboring region and These are the range weight function and the spatial domain weight function, respectively. As the normalization factor, and The influence range of pixel differences and spatial variation is controlled separately, defined according to temperature as follows: ； ； in, This is the temperature scaling factor. For pixels The corresponding temperature This represents the highest temperature in the infrared image of the blast furnace charge surface.

9. A temperature measuring rod segmentation and intelligent filling device for infrared images of blast furnace burden surface, characterized in that, include: The data acquisition and processing module is used to acquire infrared image datasets of blast furnace material surface, expand the samples, and mark the location of the temperature measuring rod area; The semantic segmentation module is used to build an infrared image semantic segmentation model based on Deeplabv3+. The semantic segmentation model is trained using an annotated dataset to achieve accurate segmentation of the area obscured by the temperature measuring rod in the infrared image of the blast furnace burden surface and output a binary mask image. The fill and repair module is used to define a mask generation method that conforms to the characteristics of the temperature measuring rod, fine-tune the LaMa pre-trained model, and perform generative fill and repair on the mask area in the binary mask image based on the fine-tuned LaMa model to restore the material surface texture and output a complete infrared image of the blast furnace material surface. The post-processing module smooths the complete infrared image of the blast furnace burden surface to obtain a high-quality infrared image of the blast furnace burden surface without interference from the temperature measuring rod.

10. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-8.

Images