Fusion method and device of infrared leakage segmentation and small target detection, and storage medium
An infrared leak segmentation and small target detection method based on multi-scale feature fusion, attention mechanism, and rotating box inference, combined with dynamic CLAHE parameter adjustment, solves the problems of weak small target features and poor adaptation of rotating regions in infrared images of industrial boilers, and achieves high-precision leak location detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-04
- Publication Date
- 2026-04-10
AI Technical Summary
In existing technologies, infrared images of industrial boilers have high noise and weak micro-leakage features of small targets. The segmentation network is weak in extracting small target features, and the detection network has poor adaptation to the rotation region. The result fusion has no effective constraints. The CLAHE algorithm uses fixed parameters without dynamic adjustment based on image entropy values, resulting in low matching degree between the enhancement effect and leakage features.
By acquiring and preprocessing infrared images, combining a segmentation network with multi-scale feature fusion and attention mechanism, and a detection network with rotating bounding box inference, triple consistency constraints are applied. A weighted voting mechanism is used to fuse mask features and leaked candidate boxes, and the equilibrium parameters of the CLAHE algorithm are dynamically adjusted to eliminate block artifacts and improve segmentation and detection accuracy.
It significantly improves the recall and localization accuracy in scenarios with minute leaks and weak signals, solves the problems of blurred features of small targets and adaptation of rotating regions, and achieves high-precision leak location detection.
Smart Images
Figure CN121837844A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer vision, in particular to an infrared leakage segmentation and small target detection fusion method, device and storage medium. BACKGROUND
[0002] The four tubes of the boiler are the core pressure-bearing components of the industrial boiler, and leakage failure will cause shutdown loss and even safety accidents. Traditional manual inspection and contact detection have the problems of poor real-time performance and high missed detection rate. The infrared thermal imaging technology has become the mainstream means due to its non-contact and real-time visualization. However, industrial infrared images have the problems of large noise, weak micro-leakage characteristics of small targets, and easy rotation and inclination of the leakage area, which need more accurate segmentation and detection fusion technology to solve.
[0003] However, the prior art has obvious deficiencies in solving this problem: At the fusion level, the small target feature extraction of the segmentation network is weak, the detection network is poor in adapting to the rotated area, and the result fusion has no effective constraint. At the preprocessing level, the CLAHE algorithm uses fixed parameters and does not dynamically adjust the image entropy value, so the enhancement effect and leakage feature matching degree are low, and there is also a sub-block stitching block effect. SUMMARY
[0004] The present application provides an infrared leakage segmentation and small target detection fusion method, device and storage medium, which uses an infrared thermal imaging semantic segmentation and efficient detection model multi-scale fusion leakage recognition method to improve the recall and positioning accuracy of micro-leakage and weak signal scenes, thereby solving the problems in the background art.
[0005] To achieve the above-mentioned purpose, the technical scheme of the present application is as follows: An infrared leakage segmentation and small target detection fusion method, executed by a computer device, comprises the following steps: S1, infrared image acquisition and preprocessing, acquiring the four-tube area image of the boiler by an infrared thermal imager, sequentially completing radiation rate calibration, black body correction and noise suppression, and then enhancing the contrast through the CLAHE algorithm to output a standardized infrared image; S2, inputting the standardized infrared image into the PaddleSeg system segmentation network, enhancing small target features through multi-scale feature fusion and attention mechanism, combining CRF edge refinement and morphological post-processing to generate mask features of the leakage area; S3, synchronously inputting the standardized infrared image into the RT-DETR and PP-YOLOE detection networks, extracting multi-scale features based on a 5-layer feature pyramid, and obtaining leakage candidate boxes including positions and confidence through rotation box reasoning; S4, match the segmentation mask features with the leakage candidate boxes on the feature pyramid, apply triple consistency constraints of IoU, boundary and confidence, fuse the mask features and the leakage candidate boxes by using a weighted voting mechanism, and output the final leakage position.
[0006] Optionally, the specific step flow of S1 is as follows: S11: image acquisition is completed on the four-tube area of the boiler by using an infrared thermal imager, meanwhile, the acquisition area is ensured to be complete by using a field angle lens, and original infrared image data containing temperature information is acquired; S12: for the original infrared image data, initial emissivity is set based on the material and surface state of the boiler pipe, temperature calibration is carried out by taking a blackbody radiation source as a benchmark, the emissivity deviation is back calculated by using a contact thermometer to measure the actual temperature of the pipe, multi-point calibration is carried out at multiple temperature points; finally, the emissivity is corrected by using a radiation compensation algorithm, and the calibrated infrared image data is obtained; S13: for the calibrated infrared image data, one-point correction is carried out by using an internal shutter to eliminate pixel offset error; two-point correction is carried out by using high and low temperature blackbodies every shift to construct the response curve of a single pixel and complete function correction, meanwhile, bad points in the image are detected, the bad point area is repaired by using a neighborhood interpolation method, and the repaired infrared image data is obtained; S14: for the repaired infrared image data, time average is carried out on 3-5 consecutive frames of images in the time domain, adaptive Gaussian filtering and bilateral filtering are applied in the spatial domain, and threshold denoising is implemented in the wavelet domain, and low-noise infrared image is output; S15: CLAHE algorithm is adopted on the low-noise infrared image to combine with the image entropy value to dynamically adjust the equalization parameters of the adaptive histogram, and preprocessed infrared image is generated.
[0007] Optionally, the specific implementation flow of S15 is as follows: S151: let the input low-noise infrared image be L, the size be , M and N represent the size, and the range of pixel values from large to small be ; image blocking is carried out, the input image L is divided into non-overlapping rectangular sub-blocks, and H and W are the number of sub-blocks divided in different directions; the size of each sub-block is , and the following conditions are met , ; let the sub-block in the i-th row and the j-th column be , and the pixel set thereof be:
[0008] wherein x represents a coordinate number belonging to 1 to M, y represents a coordinate number belonging to 1 to n, , ; S152: dynamically adjusting the CLAHE clipping threshold based on the image entropy value; Calculate the sub-block gray level histogram, for each sub-block Calculate the number of pixels with gray value k, k represents the gray level, that is, get the histogram
[0009] where, is the Dirac function, C represents the histogram data; S153: Calculate the normalized histogram, normalize the histogram to get the normalized ; , p represents the data after normalizing the histogram data, then calculate the sub-block image entropy value E, to avoid the logarithm meaningless when the gray probability is 0, introduce the minimum value , entropy value of is calculated as:
[0010] S154: dynamically adjust the clipping threshold, set the clipLimit reference value to , first calculate the mean value of all sub-block entropy values and standard deviation ; Adjust the clipping threshold of each sub-block based on the entropy value
[0011] S155: Calculate the clipping pixel number threshold, get the histogram pixels that exceed the threshold after clipping ; distribute the excess pixels clipped, calculate the total number of pixels clipped
[0012] S156: correct the sub-block histogram by uniformly distributing the excess pixels clipped, then accumulate and normalize the gray level of the corrected histogram to get the cumulative distribution function of the sub-block ; S157: sub-block pixel value transformation, for each pixel in , perform histogram equalization transformation:
[0013] S158: bilinear interpolation splicing sub-block, to eliminate the blocking effect of sub-block splicing, the pixel coordinates Performing bilinear interpolation, let it be located at the intersection of the i-th row and j-th column sub-block and the adjacent sub-block, the interpolation weight is:
[0014]
[0015] wherein u represents the weight coefficient with respect to row i, v represents the weight coefficient with respect to column j, : the final pixel value of pixel coordinate
[0016]
[0017] S159: Calculate the final pixel value of each pixel coordinate in the image to obtain a preprocessed infrared image after the CLAHE algorithm combined with the entropy dynamic adjustment .
[0018] Optionally, the specific implementation process in S156 is as follows: The excess pixels are evenly distributed to all gray levels, the number of pixels of each gray level after cropping is calculated first, then the total number of excess pixels generated by cropping in the sub-block is counted, and the excess pixels are evenly distributed to all gray levels, and finally the number of pixels after cropping of each gray level is added to the number of excess pixels allocated to the gray level, so as to obtain the adjusted histogram of each sub-block; For each sub-block, the number of pixels of the corresponding gray level in the adjusted histogram is accumulated from gray level 0 to the current gray level k, to obtain an accumulated sum; then the accumulated sum is divided by the total number of pixels of the sub-block , to calculate the cumulative distribution function value of the sub-block at gray level k .
[0019] Optionally, the specific step process in S2 is as follows: S21: Select a segmentation network based on the PaddleSeg framework, including HRNet-W48, OCRNet and PP-HumanSeg, adjust the input channel, convert the preprocessed infrared image into a single-channel temperature image or a pseudo-color three-channel image, and use a channel adaptation method based on an ImageNet pre-trained model to migrate the pre-trained weight; normalize and standardize the image temperature value to generate a segmentation network feature map; S23: Introduce a channel attention mechanism in the segmentation network feature map, highlight the feature channels by calculating the feature channel weight; add a spatial attention mechanism, and generate a spatial attention map as a preliminary segmentation result based on the importance of the pixel region; S24: Apply a conditional random field to the preliminary segmentation result output by the segmentation network to optimize the segmentation boundary based on the relationship between pixels; add a boundary-aware loss function in the network training process, and use a deep supervision strategy to supervise and train the multi-level output of the segmentation network to generate an optimized segmentation result; S25: Perform a morphological opening and closing operation on the optimized segmentation result to remove noise points and fill in area holes; perform connected component analysis to filter small noise regions with an area less than 50 pixels; use a B-spline curve to fit the edge of the segmentation region, smooth the jagged edge, and generate an accurate leakage region mask feature.
[0020] Optionally, the specific step process in S3 is as follows: S31: Based on the task requirements, select the backbone detection network to complete the configuration of the detection network infrastructure and parameters; S32: Perform multi-scale feature extraction on the preprocessed infrared image, construct a 5-layer feature pyramid, selectively fuse the features using an attention mechanism, perform nonlinear transformation through 1x1 convolution and a nonlinear activation function, complete the extraction and fusion of multi-scale features, and output the multi-scale fusion features of the preprocessed infrared image; S33: Use two kinds of rotated box representation methods, parameter representation and corner point representation, to generate rotated anchor boxes that adapt to the inclination angle of the leakage region by presetting rotated anchor box templates of different angles; Calculate the rotated IoU based on the polygon intersection-over-union ratio to select anchor boxes with high overlap with the true leakage region to obtain preliminary rotated candidate boxes, including position and confidence, using the two kinds of rotated box representation methods, and the candidate box set that has not been optimized; S34: Overlap optimization of the candidate box and the mask, calculate the intersection-over-union of the candidate box and the segmentation mask, complete the IoU matching; use the mask center point coordinates to adjust the candidate box position to guide the positioning of the mask; align the candidate box boundary with the mask boundary, apply boundary consistency constraints, and output the final leakage candidate box.
[0021] Optionally, the specific step process of S4 is as follows: S41: Based on the hierarchical correspondence relationship of the feature pyramid, match the leakage region mask feature and the leakage candidate box at each scale feature layer one by one to ensure that the leakage targets of different scales are associated on the corresponding feature layer level, and output the mask feature and the leakage candidate box matching pair on each feature layer level; S42: Based on the mask feature and the leakage candidate box matching pair on each feature layer level, apply three consistency constraints to each matching pair, remove the matching pairs that do not satisfy any constraint, and output the effective matching pairs that satisfy the three consistency constraints; S43: A weighted voting fusion strategy is adopted for the effective matching pairs, the detection candidate box weight and the segmentation mask feature weight are respectively set, the position information of the effective matching pairs is weighted and calculated, the weight proportion can be dynamically adjusted, and the weighted and fused leakage position candidate result is integrated and output. S44: Based on the leakage position candidate result, the effective result is screened according to a comprehensive confidence threshold, and the final leakage position including coordinate information is output.
[0022] Optionally, the triple consistency constraint in S42 includes: An IoU consistency constraint is used to check whether the region overlap degree of the candidate box and the mask meets a preset threshold; A boundary consistency constraint is used to ensure that the coincidence degree of the candidate box boundary and the mask boundary meets a standard; A confidence consistency constraint is used to control the deviation of the detection confidence and the segmentation confidence within an allowable range.
[0023] In another aspect, the application also discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to make the processor execute the steps of the above method.
[0024] In another aspect, the application also discloses a computer device, which comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the above method.
[0025] Compared with the prior art, the above technical solution has the following beneficial effects: 1. The application solves the problem of weak small target leakage features from two dimensions of feature extraction and edge optimization by combining multi-scale feature fusion, attention mechanism and CRF edge refinement technology in the segmentation network, significantly improves the accuracy of mask segmentation, and simultaneously uses a rotated box reasoning to adapt to a tilted leakage area, and realizes deep fusion of segmentation and detection results through triple consistency constraints, thereby greatly improving the detection accuracy of the leakage position from the aspects of positioning adaptability and fusion effectiveness.
[0026] 2. The application dynamically adjusts the equalization parameters of CLAHE based on image entropy values, realizes differentiated contrast enhancement of different sub-blocks of the infrared image, highlights the features of the leakage area and avoids excessive amplification of noise in the normal area, and further eliminates the blocking effect of the blocking processing by means of bilinear interpolation, thereby guaranteeing the quality of the preprocessed image and laying a good input foundation for efficient operation of the subsequent segmentation and detection network. BRIEF DESCRIPTION OF DRAWINGS
[0027] Figure 1 is a method step flowchart of the application; Figure 2is a specific calculation flowchart of the present application S15. DETAILED DESCRIPTION
[0028] To make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments of the present application.
[0029] The embodiments of the present application will be further described in detail below with reference to the drawings and embodiments. The following embodiments are used to illustrate the present application, but cannot be used to limit the scope of the present application.
[0030] The present application provides a fusion method of infrared leakage segmentation and small target detection, as shown in Figure 1 The following steps are executed by a computer device: S1, infrared image acquisition and preprocessing, the image of the four-tube area of the boiler is acquired by an infrared thermal imager, radiation rate calibration, black body correction and noise suppression are sequentially completed, and then the contrast is enhanced by a CLAHE algorithm, and a standardized infrared image is output; S2, the standardized infrared image is input into a PaddleSeg system segmentation network, small target features are enhanced through multi-scale feature fusion and attention mechanism, and a leakage area mask feature is generated through CRF edge refinement and morphological post-processing; S3, the standardized infrared image is simultaneously input into an RT-DETR and a PP-YOLOE detection network, multi-scale features are extracted based on a 5-layer feature pyramid, and leakage candidate boxes including positions and confidence are obtained through rotation box reasoning; S4, the segmentation mask feature and the leakage candidate box are matched on the feature pyramid, three consistency constraints of IoU, boundary and confidence are applied, the mask feature and the leakage candidate box are fused by using a weighted voting mechanism, and the final leakage position is output.
[0031] Embodiment: In this embodiment, the water-cooled wall area of the boiler of a certain thermal power plant frequently appears an inclined micro-leakage fault with an area of pixels, and when the traditional manual inspection combined with the basic infrared detection technology is used, three core problems appear: first, the micro-leakage small target features are easily covered due to the high-temperature noise interference of the boiler furnace in the infrared image, third, the segmentation and detection results are simply spliced, and the "mask area and candidate box misalignment" often occurs, and the leakage position cannot be accurately output.
[0032] Therefore, the power plant uses the fusion method of infrared leakage segmentation and small target detection of the present application, and executes the detection process through an industrial computer.
[0033] S11: Infrared image acquisition, an infrared thermal imager with a resolution of 640x512 is used, and a lens with a field of view angle of 25° is matched to ensure that the whole area of the water-cooled wall is covered, the image of the water-cooled wall area of the boiler is collected, and the original infrared image data containing temperature information is obtained.
[0034] S12: Radiance calibration, based on the carbon steel material of the water-cooled wall pipeline, the initial radiance is set to 0.85; based on the blackbody radiation source, multi-point calibration is carried out at three temperature points of 200℃, 350℃ and 500℃, the actual temperature of the pipeline is measured by a contact thermometer, the radiance deviation is back calculated, and the calibrated infrared image data is obtained through radiance compensation algorithm.
[0035] S13: Blackbody correction and bad pixel repair, one-point correction is carried out through the built-in shutter of the infrared thermal imager to eliminate pixel offset error; two-point correction is carried out once every shift using high and low temperature blackbodies of-20℃ and 600℃ to construct the response curve of a single pixel and complete function correction; three bad pixels are detected and repaired through 3x3 neighborhood interpolation method to obtain repaired infrared image data.
[0036] S14: Noise suppression, for the repaired image, first, time domain noise reduction is carried out through 3-frame time average, then, space domain adaptive Gaussian filter and bilateral filter are applied, and finally, soft threshold denoising is carried out in wavelet domain to output low-noise infrared image.
[0037] S15: CLAHE algorithm is used on the low-noise infrared image to generate preprocessed infrared image by dynamically adjusting the equalization parameters of adaptive histogram combined with image entropy value, as shown in Figure 2 .
[0038] S151: Let the input low-noise infrared image be L, the size be , M and N represent the size, and the range of pixel value from large to small be ; image blocking is carried out, the input image L is divided into non-overlapping rectangular sub-blocks, H and W are the number of sub-blocks divided in different directions; the size of each sub-block is , which satisfies , ; the sub-block in the i-th row and the j-th column is denoted as , and the pixel set thereof is:
[0039] wherein x represents the coordinate number belonging to 1 to M, y represents the coordinate number belonging to 1 to n, , ; S152: dynamically adjust the CLAHE clipping threshold value based on image entropy value; Calculate the sub-block gray scale histogram, for each sub-block Calculate the number of pixels with gray value k, k represents the gray level, that is, get the histogram :
[0040] Wherein, is the Dirac function, C represents the histogram data; S153: Calculate the normalized histogram, normalize the histogram to get the normalized ; , p represents the data after normalizing the histogram data, then calculate the sub-block image entropy value E, in order to avoid the logarithm meaningless when the gray probability is 0, introduce the minimum value , Entropy value of Calculate as:
[0041] S154: dynamically adjust the clipping threshold, set the clipLimit reference value as , first calculate the mean value of all sub-block entropy And the standard deviation ; Based on the entropy value, adjust the clipping threshold of each sub-block :
[0042] S155: calculate the clipping pixel number threshold, get the histogram pixels that exceed the threshold after clipping ; distribute the excess pixels clipped, calculate the total number of pixels clipped :
[0043] S156: evenly distribute the excess pixels to all gray levels, first calculate the number of pixels of each gray level after clipping for each sub-block, then count the total number of excess pixels generated in the sub-block due to clipping, evenly distribute this part of the excess pixels to all gray levels, finally add the number of excess pixels allocated to each gray level to the number of pixels after clipping for each gray level, to get the adjusted histogram of each sub-block; For each sub-block, from gray level 0 to the current gray level k, the number of pixels in the corresponding gray level in the adjusted histogram is added in turn to get the cumulative sum; then divide this cumulative sum by the total number of pixels in the sub-block , calculate the cumulative distribution function value of the sub-block at gray level k ; S157: sub-block pixel value transformation, for each pixel in , performing histogram equalization transformation:
[0044] S158: bilinear interpolation splicing sub-block, in order to eliminate the block effect of sub-block splicing, the pixel coordinates Perform bilinear interpolation, suppose it is located at the junction of the i-th row and the j-th column sub-block and the adjacent sub-block, the interpolation weight:
[0045]
[0046] Wherein, u represents the weight coefficient about row i, v represents the weight coefficient of column j, : then the pixel coordinates The final pixel value of is:
[0047]
[0048] S159: calculate the final pixel value of each pixel coordinate in the image, and obtain the preprocessed infrared image after the CLAHE algorithm combined with entropy dynamic adjustment .
[0049] The present application realizes high-entropy leakage area contrast enhancement, low-entropy normal area noise suppression, and micro-leakage target gray scale recognition degree re-improvement by distinguishing from the existing fixed CLAHE parameter processing, and dynamically adjusting the threshold value by entropy value, and solves the small target feature blur problem.
[0050] S21: segmentation network configuration and feature map generation, input the normalized infrared image into the HRNet-W48 segmentation network of the PaddleSeg framework, adjust the input channel to 3, and convert the single-channel temperature map to three channels; using the channel adaptation method based on the ImageNet pre-trained model to transfer the weight, the image temperature value is normalized to [0, 1] and Z-score standardized, and the segmentation network feature map is generated.
[0051] Compared with directly using the pre-trained weight, the channel adaptation transfer + temperature value double standardization makes the segmentation network adapt to the infrared image features faster, reduces the number of model training iterations, and at the same time improves the sensitivity of the feature map to temperature difference.
[0052] S23: Attention mechanism reinforced features. In the segmentation network feature map, the channel attention mechanism is introduced. The feature channel weight is calculated through the Squeeze-Excitation structure, and the related channel weight is increased to 1.5 times. The spatial attention mechanism is added, and the spatial attention map is generated based on the pixel area temperature gradient to strengthen the pixel weight of the leakage area, and the preliminary segmentation result is obtained.
[0053] S24: CRF edge refinement and depth supervision. The CRF is applied to the preliminary segmentation result, and the segmentation boundary is optimized based on the temperature and spatial relationship between pixels. The boundary perception loss function is added in the network training to supervise the training of the shallow, middle and deep layers of the network, and the optimized segmentation result is generated.
[0054] S25: Morphological post-processing. The optimized segmentation result is subjected to morphological opening operation of 3x3 rectangular structure element and morphological closing operation of 5x5 circular structure element. Then the B-spline curve is used to fit the edge of the segmentation area to smooth the jagged edge and generate the leakage area mask feature.
[0055] S31: Detection network configuration Operation process: Based on the micro-leakage detection requirement, RT-DETR (main ResNet50) and PP-YOLOE are selected as the detection network. The input resolution is configured as 640x640, and the feature extraction layer is C3-C5. The basic architecture and parameter configuration of the detection network are completed.
[0056] Innovation and benefits: Compared with a single detection network, the dual network configuration takes into account the long sequence feature extraction capability of RT-DETR and the real-time performance of PP-YOLOE, which lays a foundation for subsequent accurate and efficient candidate box generation.
[0057] S32: Multi-scale feature extraction fusion. For the standardized infrared image, a five-layer feature pyramid is constructed. The features of each layer are selectively fused through the attention gate mechanism, and then 1x1 convolution and ReLU non-linear activation are performed to output the multi-scale fusion features.
[0058] S33: Rotated box reasoning. For the multi-scale fusion features, two kinds of rotated box representations are used: parameter representation and angle point representation. Different angle rotated anchor box templates are preset to generate rotated anchor boxes that adapt to the inclination angle of the leakage area. The rotated IoU is calculated based on the polygon intersection over union, and the anchor boxes with high overlap with the real leakage area are selected to obtain the preliminary rotated candidate boxes, which include position and confidence. The candidate box set is not optimized. For example: 5 parameters, center point (x, y), width w, height h, and rotation angle θ. Corner; preset 15°, 30°, 45°, 60°, 75° rotation anchor frame template, generate rotation anchor frame; based on polygon IoU calculation, filter anchor frame with overlap degree > 0.7, obtain, for example: preliminary rotation candidate frame with position (x: 280, y: 190, w: 20, h: 15, θ: 30°, confidence 0.88.
[0059] Rotation anchor frame + polygon IoU, accurate adaptation to inclined leakage shape, solve the problem of large inclined leakage positioning deviation.
[0060] S34: candidate frame and mask overlap optimization, calculate the IoU value 0.72 of the preliminary candidate frame and the mask feature output by S2, complete the IoU matching; adjust the center point of the candidate frame to the center point of the mask, align the boundary of the candidate frame and the mask, the deviation is ≤2 pixels, apply boundary consistency constraint, output optimized leakage candidate frame.
[0061] By adjusting the candidate frame through mask matching, the positioning deviation is reduced from the preliminary 10 pixels to within 5 pixels, providing a high-precision candidate frame for subsequent fusion.
[0062] S41: feature pyramid level matching, on the five-layer feature pyramid constructed in S32, match the mask feature output by S2 with the candidate frame output by S34 one by one, ensure that the micro-leakage small target is accurately associated at the P4 layer, and output the mask-candidate frame matching pairs of each layer.
[0063] S42: triple consistency constraint, impose constraints on the three matching pairs: For example: ① IoU ≥ 0.6; ② boundary deviation ≤ 3 pixels; ③ detection confidence and segmentation confidence deviation ≤ 0.1.
[0064] Compared with unconstrained fusion, the triple constraint eliminates invalid matching pairs and reduces the interference of low-quality results.
[0065] S43: weighted voting fusion, set the detection candidate frame weight to 0.6 and the segmentation mask weight to 0.4, and weightedly calculate the position information of the two effective matching pairs to integrate and output the leakage position candidate result.
[0066] Unlike fixed weight splicing, dynamic adaptive weighted voting combines the advantages of double results, and the fusion result confidence is improved by 15% compared with single result.
[0067] In another aspect, the application also discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to make the processor execute the steps of the above method.
[0068] In still another aspect, the present application also discloses a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, causes the processor to perform the steps of the above method.
[0069] In yet another embodiment provided in the present application, a computer program product containing instructions, which, when executed on a computer, causes the computer to perform the method of fusing infrared leakage segmentation and small target detection of any of the above embodiments.
[0070] It can be understood that the system, device and storage medium provided by the embodiments of the present application correspond to the method provided by the embodiments of the present application, and the explanation, examples and beneficial effects of the related content can refer to the corresponding part in the above method.
[0071] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium.
[0072] For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media.
[0073] The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD) or a semiconductor medium (for example, solid state disk (SSD)) and the like.
[0074] It should be noted that in this paper, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between the entities or operations.
[0075] Also, the term "comprising" or "including" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.
[0076] Each of the embodiments in the specification is described in a relevant manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.
[0077] The embodiments of the present application are given for example and description, although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.
Claims
1. A fusion method for infrared leakage segmentation and small target detection, characterized in that, Perform the following steps using a computer device: S1, Infrared image acquisition and preprocessing: The image of the four tube areas of the boiler is acquired by an infrared thermal imager, and the emissivity calibration, blackbody correction and noise suppression are completed in sequence. Then, the contrast is enhanced by the CLAHE algorithm, and a standardized infrared image is output. S2, the standardized infrared image is input into the PaddleSeg system segmentation network, the small target features are enhanced through multi-scale feature fusion and attention mechanism, and the mask features of the leakage area are generated by combining CRF edge refinement and morphological post-processing. S3, simultaneously inputs the standardized infrared image into the RT-DETR and PP-YOLOE detection networks, extracts multi-scale features based on a 5-layer feature pyramid, and obtains leak candidate boxes including location and confidence through rotating box inference; S4 matches segmentation mask features and leakage candidate boxes on the feature pyramid, applies triple consistency constraints of IoU, boundary and confidence, and uses a weighted voting mechanism to fuse mask features and leakage candidate boxes to output the final leakage location.
2. The fusion method of infrared leakage segmentation and small target detection as described in claim 1, characterized in that: The specific steps of S1 are as follows: S11: The infrared thermal imager acquires images of the four tube areas of the boiler, while the field-of-view lens ensures complete coverage of the acquisition area and obtains raw infrared image data containing temperature information. S12: Based on the original infrared image data, the initial emissivity is set according to the material and surface condition of the boiler pipe. Temperature calibration is performed with a blackbody radiation source as a reference. The actual temperature of the pipe is measured using a contact thermometer to back-calculate the emissivity deviation. Multi-point calibration is carried out at multiple temperature points. Finally, the emissivity is corrected through an emissivity compensation algorithm to obtain the calibrated infrared image data. S13: For the calibrated infrared image data, a one-point correction is performed once through the built-in shutter to eliminate pixel offset error; a two-point correction is performed once per shift using a high and low temperature blackbody to construct the response curve of a single pixel to complete function correction. At the same time, bad pixels in the image are detected, and bad pixel areas are repaired by neighborhood interpolation to obtain repaired infrared image data. S14: In the time domain, the repaired infrared image data is averaged over 3-5 consecutive frames. In the spatial domain, adaptive Gaussian filtering and bilateral filtering are applied, and threshold denoising is performed in the wavelet domain to output a low-noise infrared image. S15: The CLAHE algorithm is used to dynamically adjust the equalization parameters of the adaptive histogram based on the image entropy value to generate a preprocessed infrared image from the low-noise infrared image.
3. The fusion method of infrared leakage segmentation and small target detection as described in claim 2, characterized in that: The specific implementation process of S15 is as follows: S151: Let the input low-noise infrared image be L, and its size be... M and N represent the size, and the pixel values range from largest to smallest as follows: ; The input image L is divided into blocks. Each non-overlapping rectangular sub-block, where H and W represent the number of sub-blocks in different directions; Each sub-block has a size of ,satisfy , Let the sub-block in row i and column j be denoted as . Its pixel set is: Where x represents the coordinate number belonging to the range 1 to M, and y represents the coordinate number belonging to the range 1 to n. , ; S152: Dynamically adjust the CLAHE cropping threshold based on image entropy value; Calculate the grayscale histogram of each sub-block. Calculate the number of pixels with a grayscale value of k, where k represents the grayscale level, to obtain the histogram. : in, Here, C represents the Dirac function, and C represents the histogram data. S153: Calculate the normalized histogram, normalize the histogram, and obtain the normalized histogram. ; p represents the data after normalization of the histogram data. Then, the entropy value E of the sub-block image is calculated. To avoid the logarithm being meaningless when the gray level probability is 0, a local minimum is introduced. , entropy value The calculation is as follows: S154: Dynamically adjust the clipping threshold, setting the clipLimit baseline value to... First, calculate the average entropy value of all sub-blocks. and standard deviation ; Adjust the clipping threshold for each sub-block based on entropy value. : S155: Calculate the cropping pixel threshold and obtain the histogram pixels that are cropped beyond the threshold. ; Allocate the excess pixels that have been cropped and calculate the total number of pixels to be cropped. : S156: The sub-block histogram is corrected by uniformly distributing and cropping redundant pixels. Then, the corrected histogram is accumulated at gray levels and normalized to obtain the cumulative distribution function of the sub-block. ; S157: Sub-block pixel value transformation, for Each pixel Perform histogram equalization transformation: S158: Bilinear interpolation for sub-block stitching. To eliminate the block effect in sub-block stitching, pixel coordinates are... Perform bilinear interpolation, assuming it is located at the boundary between the i-th row and j-th column sub-block and its adjacent sub-block. Interpolation weights: Where u represents the weight coefficient for row i, and v represents the weight coefficient compared to column j. Then pixel coordinates final pixel value for: S159: Calculate the final pixel value of each pixel coordinate in the image to obtain the preprocessed infrared image after dynamic adjustment using the CLAHE algorithm combined with entropy. .
4. The fusion method of infrared leakage segmentation and small target detection as described in claim 3, characterized in that: The specific implementation process in S156 is as follows: The excess pixels are evenly distributed to all gray levels. First, the number of pixels in each gray level after cropping each sub-block is calculated. Then, the total number of excess pixels in the sub-block due to cropping is counted. These excess pixels are evenly distributed to all gray levels. Finally, the number of pixels after cropping in each gray level is added to the number of excess pixels allocated to that gray level to obtain the adjusted histogram for each sub-block. For each sub-block, starting from gray level 0 to the current gray level k, the number of pixels at the corresponding gray level in the adjusted histogram is accumulated to obtain a total sum; then this total sum is divided by the total number of pixels in that sub-block. The cumulative distribution function value of the sub-block at gray level k is calculated. .
5. The fusion method of infrared leakage segmentation and small target detection as described in claim 4, characterized in that: The specific steps in S2 are as follows: S21: Select segmentation networks based on the PaddleSeg framework, including HRNet-W48, OCRNet, and PP-HumanSeg. Adjust the input channels to convert the preprocessed infrared image into a single-channel temperature map or a pseudo-color three-channel image. Use the channel adaptation method based on the ImageNet pre-trained model to transfer the pre-trained weights. Normalize and standardize the image temperature values to generate segmentation network feature maps. S23: Introduce a channel attention mechanism into the feature map of the segmentation network to highlight the feature channels by calculating the feature channel weights; A spatial attention mechanism is incorporated to generate a spatial attention map based on the importance of pixel regions as a preliminary segmentation result; S24: Apply a conditional random field to the initial segmentation results output by the segmentation network to optimize the segmentation boundary based on the relationship between pixels; During network training, a boundary-aware loss function is added, and a deep supervision strategy is adopted to supervise the training of multi-level outputs of the segmentation network, thereby generating optimized segmentation results. S25: Perform morphological opening and closing operations on the optimized segmentation results to remove noise and fill regional holes; conduct connected component analysis to filter out small noise regions with an area less than 50 pixels; use B-spline curves to fit the edges of the segmented regions, smooth jagged edges, and generate accurate leakage region mask features.
6. The fusion method of infrared leakage segmentation and small target detection as described in claim 5, characterized in that: The specific steps in S3 are as follows: S31: Based on task requirements, select the backbone detection network and complete the configuration of the detection network infrastructure and parameters; S32: Perform multi-scale feature extraction on the preprocessed infrared image, construct a 5-layer feature pyramid, selectively fuse features using an attention mechanism, and perform nonlinear transformation through 1×1 convolution and nonlinear activation function to complete the extraction and fusion of multi-scale features, and output the multi-scale fused features of the preprocessed infrared image. S33: Two rotating frame representation methods are used for multi-scale fusion features: parameter representation and corner point representation. Rotating anchor frame templates with different angles are preset to generate rotating anchor frames that adapt to the tilt angle of the leakage area. Based on the intersection-union ratio of polygons, the rotation IoU is calculated, and anchor boxes with high overlap with the actual leakage area are screened to obtain preliminary rotation candidate boxes, which include position and confidence. Two rotation box representation methods are used, and the candidate box set is not optimized. S34: Optimize the overlap between the candidate box and the mask, calculate the intersection-union ratio between the candidate box and the segmentation mask, and complete the IoU matching; use the coordinates of the mask center point to adjust the position of the candidate box to guide the mask positioning; align the candidate box boundary with the mask boundary, apply boundary consistency constraints, and output the final leak candidate box.
7. The fusion method for infrared leakage segmentation and small target detection as described in claim 6, characterized in that: The specific steps of S4 are as follows: S41: Based on the hierarchical correspondence of the feature pyramid, the mask features of the leakage area and the leakage candidate boxes are matched one by one at each feature layer of each scale to ensure that leakage targets at different scales are associated at the corresponding feature layers, and the output is the matching pair of mask features and leakage candidate boxes at each feature layer. S42: Based on the mask features and leak candidate box matching pairs at each feature level, apply triple consistency constraints to each matching pair, eliminate matching pairs that do not satisfy any of the constraints, and output the valid matching pairs that satisfy the triple consistency constraints. S43: A weighted voting fusion strategy is adopted for effective matching pairs. The weights of detection candidate boxes and segmentation mask features are set separately. The position information of effective matching pairs is weighted and calculated. The weight ratio can be dynamically adjusted. The weighted fusion result of leakage position candidate is then integrated and output. S44: Based on the candidate results of the leak location, the valid results are filtered according to the comprehensive confidence threshold, and the coordinate information is unified and output as the final leak location including coordinate information.
8. The fusion method for infrared leakage segmentation and small target detection as described in claim 7, characterized in that: The triple consistency constraint in S42 includes: IoU consistency constraint, verifying whether the overlap between the candidate box and the mask meets the preset threshold; Boundary consistency constraints ensure that the candidate box boundary matches the mask boundary to a certain standard; The confidence consistency constraint keeps the deviation between the detection confidence and the segmentation confidence within an acceptable range.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method as described in any one of claims 1 to 8.
10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the processor, it causes the processor to perform the steps of the method as described in any one of claims 1 to 8.