Boiler equipment defect detection method based on image processing

Abstract

The invention discloses a boiler equipment defect detection method based on image processing, and relates to the technical field of defect detection.The boiler equipment defect detection method comprises the steps that visible light and infrared image sequences with aligned timestamps and overlapped space view fields are synchronously collected, and only areas with motion features consistent with temperature anomaly in time and space are reserved to serve as candidate defects; single-mode interference such as steam, water drops and heat reflection is effectively filtered out, and the preliminary screening reliability is remarkably improved. Secondly, introducing a multi-dimensional time sequence analysis mechanism based on a time sequence track, comprehensively evaluating a suspected defect region, constructing a confidence score, and combining with a composite judgment logic to ensure that an exception which only continuously exists, has an expansion tendency and is stable in behavior is confirmed as a real defect, so that the detection robustness is greatly enhanced. Finally, a defect severity quantitative model is established, closed-loop management is achieved, multi-mode fusion, dynamic evolution modeling and risk grading decision are organically combined for the first time, and a reliable intelligent detection scheme is provided for safe operation of the boiler.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of defect detection, and in particular to a boiler equipment defect detection method based on image processing. BACKGROUND

[0002] Boiler equipment is widely used in the industrial fields of thermal power generation, petrochemical industry, central heating, etc., and its operation safety is directly related to the continuity of production and the safety of personnel and property. In high-temperature, high-pressure and complex combustion environments, key components such as boiler pipes, furnaces and heating surfaces are prone to defects such as cracks, leaks and bulges. Traditional detection methods rely on manual inspection or regular shutdown inspection, which is not only inefficient and incomplete, but also difficult to detect early and small defects, and has been unable to meet the needs of modern industry for high reliability and intelligent operation and maintenance.

[0003] In recent years, online visual detection technology based on image processing has gradually become a research hotspot. Among them, visible light imaging can capture surface topography and dynamic changes, and infrared thermal imaging can reflect abnormal temperature distribution. Both of them provide defect clues from different physical dimensions, however, in the actual boiler operation scene, the existing defect detection methods based on image processing still have the following three technical defects:

[0004] First, the multi-modal information utilization is insufficient, resulting in weak interference suppression ability. Specifically, existing methods usually use visible light images or infrared thermal imaging images alone for defect recognition. However, in the boiler operation state, visible light images are easily disturbed by dynamic targets such as steam, water droplets and fly ash, resulting in a large number of motion artifacts; infrared thermal imaging images are easily affected by air flow disturbance and local heat reflection, resulting in transient temperature abnormalities. Due to the lack of spatiotemporal consistency verification of visible light and infrared images, existing technologies cannot effectively distinguish between real defects and occasional disturbances that only appear in a single modality, and the preliminary screening results contain a large number of candidate defect regions, with a high false positive rate.

[0005] Second, there is a lack of modeling of the spatiotemporal evolution characteristics of defects, and the final decision reliability is low. Specifically, most existing methods are based on single-frame or multi-frame static features for judgment, without establishing a time series trajectory to analyze the persistence and development trend of defects. Therefore, it is difficult to effectively eliminate suspected areas that appear temporarily or vibrate in position, resulting in misjudgment of non-persistent interference as real defect areas. Even if a simple threshold filter is introduced, it is difficult to achieve high-robustness final decision due to the lack of comprehensive consideration of local confidence ratio, diffusion trend strength and multi-modal consistency indicators and other time sequence features.

[0006] Thirdly, the defect assessment is disconnected with the operation and maintenance response, and lacks a quantitative grading mechanism. Specifically, the prior art usually only outputs a binary result of "existence / nonexistence" even if the defect is identified, without quantifying the risk level of the defect. Specifically, the severity coefficient of the defect is not calculated in combination with physical measurable indexes such as the thermal anomaly peak value, the area growth rate and the spatial coverage width, so that the real defect area cannot be grouped according to the risk level, and a differentiated operation and maintenance response instruction cannot be generated, resulting in extensive maintenance intervention strategy and difficulty in supporting intelligent operation and maintenance decision.

[0007] Therefore, there is an urgent need for a technical scheme of a boiler equipment defect detection method based on image processing in the prior art. SUMMARY

[0008] To solve the above technical problems, the present application provides a boiler equipment defect detection method based on image processing, which specifically comprises the following steps:

[0009] S1, under the running state of the boiler, synchronously collecting a visible light image sequence and an infrared thermal imaging image sequence of a to-be-inspected area, wherein the time stamps of the visible light image sequence and the infrared thermal imaging image sequence are aligned, and the spatial fields of view are coincided;

[0010] S2, performing motion target segmentation on the visible light image sequence to obtain a first dynamic mask image, and performing temperature abnormal area extraction on the infrared thermal imaging image sequence to obtain a second dynamic mask image; performing spatio-temporal consistency verification on the first dynamic mask image and the second dynamic mask image to generate a candidate defect area set;

[0011] S3, for each candidate defect area in the candidate defect area set, calculating four feature indexes and performing preliminary logical judgment on each candidate defect area based on the four feature indexes to screen out a suspected defect area;

[0012] The four feature indexes respectively include: multi-modal difference degree, motion stability, shape irregularity and background fusion degree;

[0013] S31, setting a first threshold and a second threshold of the multi-modal difference degree, an upper limit threshold of the motion stability, a lower limit threshold of the shape irregularity and a lower limit threshold of the background fusion degree;

[0014] S32, judging whether the shape irregularity of the current candidate defect area is greater than or equal to the lower limit threshold of the shape irregularity, and whether the background fusion degree of the current candidate defect area is greater than or equal to the lower limit threshold of the background fusion degree;

[0015] S33, if the judgment result of S32 is yes, further judging whether the current candidate defect area satisfies any one of the following conditions:

[0016] Condition one: the multimodal difference degree of the current candidate defect region is greater than or equal to a first threshold of the multimodal difference degree and less than a second threshold of the multimodal difference degree, and the motion stability of the current candidate defect region is less than an upper limit threshold of the motion stability;

[0017] Condition two: the multimodal difference degree of the current candidate defect region is greater than or equal to the second threshold of the multimodal difference degree, and the shape irregularity of the current candidate defect region is greater than or equal to 1.5 times of a lower limit threshold of the shape irregularity;

[0018] S34, if the result of any condition in S33 is yes, marking the current candidate defect region as a suspected defect region;

[0019] S35, if the result of S32 is no, or the results of the two conditions in S33 are both no, eliminating the current candidate defect region.

[0020] S4, for each suspected defect region, calculating a comprehensive spatio-temporal confidence score, and making a final decision on each suspected defect region based on the comprehensive spatio-temporal confidence score to obtain a real defect region;

[0021] S411, performing trajectory tracking on each suspected defect region in consecutive N frames to form a time sequence trajectory of each suspected defect region;

[0022] S412, based on the time sequence trajectory of each suspected defect region, counting the number of frames in which each suspected defect region is determined as a suspected defect region within a time window, and calculating the ratio of the number of frames to the total number of frames in the time window as the local confidence proportion of each suspected defect region;

[0023] S413, based on the time sequence trajectory of each suspected defect region, calculating the linear regression slope of the area change of each suspected defect region with time as the diffusion trend strength of each suspected defect region;

[0024] S414, based on the time sequence trajectory of each suspected defect region, obtaining the multimodal difference degree of each suspected defect region in each frame, and calculating the variance of the multimodal difference degree within the time window as the multimodal consistency index of each suspected defect region;

[0025] S415, setting a first weight coefficient of the local confidence proportion, a second weight coefficient of the diffusion trend strength and a third weight coefficient of the multimodal consistency index, and multiplying the local confidence proportion of each suspected defect region, the diffusion trend strength of each suspected defect region and the multimodal consistency index of each suspected defect region by the corresponding first weight coefficient, second weight coefficient and third weight coefficient respectively, and then summing to obtain the comprehensive spatio-temporal confidence score of each suspected defect region.

[0026] S421, set a third threshold and a fourth threshold of the comprehensive spatio-temporal confidence score, wherein the third threshold is less than the fourth threshold;

[0027] S422, judge whether the spatio-temporal confidence score of the current suspected defect region is greater than or equal to the fourth threshold;

[0028] S423, if the spatio-temporal confidence score of the current suspected defect region is greater than or equal to the fourth threshold, determine the current suspected defect region as a real defect region;

[0029] S424, judge whether the spatio-temporal confidence score of the current suspected defect region is less than or equal to the third threshold;

[0030] S425, if the spatio-temporal confidence score of the current suspected defect region is less than or equal to the third threshold, eliminate the current suspected defect region;

[0031] S426, if the spatio-temporal confidence score of the current suspected defect region is greater than the third threshold and less than the fourth threshold, further judge whether the following three conditions are met simultaneously: condition three: the local confidence proportion of the current suspected defect region is greater than or equal to a preset local confidence proportion threshold; condition four: the diffusion trend intensity of the current suspected defect region is greater than zero; condition five: the multi-modal consistency index of the current suspected defect region is less than or equal to a preset multi-modal consistency index threshold;

[0032] S427, if the above three conditions are met simultaneously, determine the current suspected defect region as a real defect region; otherwise, eliminate the current suspected defect region.

[0033] S5, for each real defect region, calculate a defect severity coefficient, group all real defect regions according to the defect severity coefficient, generate an operation and maintenance response instruction for each group of real defect regions based on the grouping result, and implement maintenance intervention on each real defect region in each group according to the operation and maintenance response instruction;

[0034] S511, obtain the highest temperature value of each real defect region in the current frame of infrared thermal imaging image, and calculate the difference between the highest temperature value and the normal running background temperature of the boiler as the thermal anomaly peak value of each real defect region;

[0035] S512, obtain the continuous M frame pixel area of each real defect region in the time sequence track, and calculate the linear fitting slope of the continuous M frame pixel area changing with time as the area growth rate of each real defect region;

[0036] S513, acquire the pixel area of each real defect area in all frames in the time sequence track, and take the average value of the pixel area of all frames as the spatial coverage of each real defect area;

[0037] S514, set the first weighting coefficient of the thermal anomaly peak, the second weighting coefficient of the area growth rate, and the third weighting coefficient of the spatial coverage;

[0038] S515, multiply the thermal anomaly peak, the area growth rate and the spatial coverage of each real defect area by the corresponding first weighting coefficient, second weighting coefficient and third weighting coefficient respectively, and sum the three multiplication results to obtain the defect severity coefficient of each real defect area.

[0039] S521, set the fifth threshold and the sixth threshold of the defect severity coefficient, wherein the fifth threshold is less than the sixth threshold;

[0040] S522, judge whether the defect severity coefficient of the current real defect area is greater than or equal to the sixth threshold;

[0041] S523, if the defect severity coefficient of the current real defect area is greater than or equal to the sixth threshold, the current real defect area is classified into the first group;

[0042] S524, judge whether the defect severity coefficient of the current real defect area is less than the fifth threshold;

[0043] S525, if the defect severity coefficient of the current real defect area is less than the fifth threshold, the current real defect area is classified into the third group;

[0044] S526, if the defect severity coefficient of the current real defect area is greater than or equal to the fifth threshold and less than the sixth threshold, the current real defect area is classified into the second group.

[0045] The embodiment of the present application has the following technical effects:

[0046] This invention effectively solves three key problems in online boiler defect detection: high false alarm rate, unreliable judgment, and disconnect between operation and maintenance. First, by synchronously acquiring visible light and infrared image sequences with aligned timestamps and overlapping spatial fields of view, and retaining only regions with consistent motion characteristics and temperature anomalies in time and space as candidate defects, it effectively filters out single-mode interference such as steam, water droplets, and heat reflection, significantly improving the reliability of initial screening. Second, it introduces a multi-dimensional temporal analysis mechanism based on time-series trajectories to comprehensively evaluate the stability of suspected defect areas, their area diffusion trends, and the consistency of multi-modal responses. A confidence score is constructed and combined with composite judgment logic to ensure that only persistent anomalies with an expansion tendency and stable behavior are confirmed as true defects, greatly enhancing detection robustness. Finally, a defect severity quantification model is established, integrating thermal anomaly peak values, area growth rates, and spatial coverage to generate a quantifiable severity coefficient. Based on this, defects are classified into high, medium, and low risk levels, automatically generating matching operation and maintenance response instructions, achieving closed-loop management from identification to intervention. This method is the first to organically combine multimodal fusion, dynamic evolution modeling, and risk classification decision-making, providing an efficient, reliable, and practical intelligent detection solution for boiler safe operation. Attached Figure Description

[0047] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0048] Figure 1 This is a flowchart of a boiler equipment defect detection method based on image processing provided in an embodiment of the present invention;

[0049] Figure 2 This is a flowchart of a suspected defect area screening method for boiler equipment defect detection based on image processing, provided by an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0051] Example 1: As Figure 1 As shown, the present invention provides a boiler equipment defect detection method based on image processing, comprising the following steps:

[0052] S1, synchronously collecting a visible light image sequence and an infrared thermal imaging image sequence of a to-be-inspected region in a boiler running state, wherein the time stamps of the visible light image sequence and the infrared thermal imaging image sequence are aligned, and the spatial fields of view of the visible light image sequence and the infrared thermal imaging image sequence are overlapped;

[0053] It is worth noting that, in the boiler running state, the visible light image sequence and the infrared thermal imaging image sequence of the to-be-inspected region are synchronously collected by the visible light camera and the infrared thermal imager which are rigidly fixed on the same pan-tilt. The two cameras are controlled by the same main control unit through a hardware trigger signal to unify the exposure timing, so as to ensure that the collection time of each frame of visible light image and the corresponding frame of infrared image deviates by not more than 1 millisecond, thereby generating the visible light image sequence and the infrared thermal imaging image sequence with aligned time stamps. Before deployment, the two cameras are jointly calibrated by using a composite calibration board, the external parameters of the infrared image relative to the visible light image are calculated, and a spatial mapping model is constructed. In the running process, the model is used for geometric correction and pixel resampling of the infrared image, so that the spatial coordinate system of the infrared image is aligned with that of the visible light image, and finally the visible light image sequence and the infrared thermal imaging image sequence with overlapped spatial fields of view are output.

[0054] S2, performing motion target segmentation on the visible light image sequence to obtain a first dynamic mask image, performing temperature abnormal region extraction on the infrared thermal imaging image sequence to obtain a second dynamic mask image, and performing spatio-temporal consistency verification on the first dynamic mask image and the second dynamic mask image to generate a candidate defect region set;

[0055] It is worth noting that the motion target segmentation on the visible light image sequence is performed by using a method combining background modeling and frame difference to generate the first dynamic mask image of each frame. For the infrared thermal imaging image sequence, the region with temperature significantly higher than the background is extracted based on the adaptive threshold method to generate the second dynamic mask image of each frame. Subsequently, the first dynamic mask image and the second dynamic mask image at the same time are subjected to pixel-by-pixel logical AND operation, and only the overlapping regions with an area greater than a preset minimum size (such as 20 pixels) and a centroid position deviation less than 5 pixels are retained. All overlapping regions satisfying the above spatio-temporal consistency conditions are integrated as the candidate defect region of the current frame, and finally the results of all frames are collected to form the candidate defect region set.

[0056] S3, for each candidate defect region in the candidate defect region set, four feature indexes are calculated, and a preliminary logical judgment is performed on each candidate defect region based on the four feature indexes to screen out a suspected defect region;

[0057] The four feature indexes respectively include: multi-modal difference degree, motion stability, shape irregularity and background fusion degree;

[0058] It is worth noting that the process of obtaining the multimodal dissimilarity is as follows: for each candidate defect region, the mean pixel value is calculated in both the visible light image and the infrared thermal image. Then, the absolute difference between the two means is calculated as the multimodal dissimilarity of that region.

[0059] The process of obtaining motion stability involves tracking the positional changes of candidate defect regions across consecutive frames and using centroid displacement as a metric. Specifically, the distance between the centroids of this region is calculated between consecutive frames, and the average distance across all frames yields the motion stability index for that region. A smaller value indicates greater motion stability.

[0060] The process of obtaining shape irregularity is as follows: First, the outline of the candidate defect region is extracted, and then the ratio of its area to the area of ​​the smallest circumcircle is calculated. A value close to 1 indicates a relatively regular shape; conversely, a value lower than 1 indicates a higher degree of shape irregularity.

[0061] The process of obtaining background fusion degree is as follows: The consistency between the candidate defect region and its surrounding background region is analyzed based on the gray-level co-occurrence matrix (GLCM) feature. Specifically, parameters such as contrast, energy, and entropy of the GLCM are calculated and compared with the corresponding parameters of the surrounding background region. Smaller differences indicate higher fusion degree, meaning the defect is less likely to be detected. By combining the evaluation results from these four dimensions, a comprehensive understanding of the characteristics of the candidate defect region can be obtained.

[0062] It is worth further elaborating that the four characteristic indicators used for preliminary logical judgment mentioned above—multimodal difference, motion stability, shape irregularity, and background blending—together constitute a multidimensional basis for judging the physical rationality of candidate defect regions. Multimodal difference reflects the coupling degree between visible light and infrared response, effectively distinguishing between real temperature change-motion correlation phenomena and single-modal artifacts; motion stability quantifies the positional jitter of the region in consecutive frames, eliminating high-speed drift or instantaneous interference; shape irregularity captures the geometric complexity of edges, excluding regular noise such as water droplets and reflections; background blending measures the contrast between the defect and the surrounding pipes in terms of texture or thermal distribution, ensuring that it has sufficient abruptness. These four indicators, starting from four orthogonal dimensions of "modal synergy," "temporal stability," "morphological rationality," and "environmental contrast," respectively, construct a preliminary screening logic that takes into account both physical laws and engineering experience, significantly improving the accuracy of suspected defect screening and providing high-quality input for subsequent high-order verification, effectively avoiding low-quality candidate regions from entering complex calculation processes.

[0063] like Figure 2 As shown, the preliminary logical judgment of each candidate defect region based on the four feature indicators, and the screening of suspected defect regions specifically includes the following steps:

[0064] S31. Set the first and second thresholds for multimodal difference, the upper threshold for motion stability, the lower threshold for shape irregularity, and the lower threshold for background blending.

[0065] It is worth noting that during the calibration phase before equipment deployment, a large number of synchronous visible light image sequences and infrared thermal imaging image sequences were collected under normal boiler operation conditions and typical defects (such as minor leaks, weld cracks, and local bulges). The aforementioned four feature indicators were extracted from the manually labeled real defect areas and interference areas (such as steam disturbance, water droplets, reflection, and heat reflection): multimodal difference, motion stability, shape irregularity, and background blending degree. Based on statistical analysis, the distribution range of each feature in real defect and interference samples was determined.

[0066] Specifically, the first threshold for multimodal variability is set at 1.2 times the 95th percentile of the index in the interfering samples, used to exclude the vast majority of occasional interference; the second threshold is set at 0.8 times the 5th percentile of the index in the real defect samples, ensuring that high-confidence defects are covered. The two constitute a dual-interval discrimination logic of "medium-low variability" and "high variability": medium-low variability corresponds to mild but continuous temperature change-motion coupling phenomena (such as slow leakage), and high variability corresponds to violent anomalies (such as explosive ejection).

[0067] The upper limit threshold of motion stability is determined by analyzing the amplitude of the centroid trajectory jitter of interference samples (such as drifting steam and fly ash). The maximum value of its motion stability index is taken and a 10% safety margin is added to ensure that only relatively stable areas are retained and transient targets that move at high speed or jitter violently are eliminated.

[0068] The lower limit threshold for shape irregularity is set based on the characteristic that real defects (such as cracks and bulges) usually have irregular geometric shapes: the minimum value of shape irregularity in real defect samples is statistically analyzed, and 80% of it is used as the lower limit to exclude regular interference such as circular water droplets and regular reflective spots.

[0069] The lower limit threshold for background blending is set based on the fact that real defects usually have obvious visual or thermal contrast with the surrounding pipe surface: the average feature distance between the real defect area and its neighboring background is calculated, and 70% of it is used as the lower limit to ensure that the selected area has sufficient "sharpness" and avoid misjudging noise that is highly blended with the background as a defect.

[0070] The aforementioned thresholds can be fine-tuned by maintenance personnel during the initialization phase based on equipment type and operating environment, or dynamically optimized through an online learning mechanism as operational data accumulates. This setting method ensures both the physical rationality of the initial logical judgment and the robustness and adaptability of the engineering implementation, providing a reliable basis for effectively screening suspected defect areas from candidate defect areas.

[0071] S32. Determine whether the shape irregularity of the current candidate defect region is greater than or equal to the lower limit threshold of the shape irregularity, and whether the background blending degree of the current candidate defect region is greater than or equal to the lower limit threshold of the background blending degree.

[0072] S33. If the judgment result of S32 is yes, then further determine whether the current candidate defect region meets any of the following conditions:

[0073] Condition 1: The multimodal difference of the current candidate defect region is greater than or equal to the first threshold of multimodal difference and less than the second threshold of multimodal difference, and the motion stability of the current candidate defect region is less than the upper limit threshold of motion stability;

[0074] Condition 2: The multimodal variability of the current candidate defect region is greater than or equal to the second threshold of multimodal variability, and the shape irregularity of the current candidate defect region is greater than or equal to 1.5 times the lower limit threshold of shape irregularity;

[0075] S34. If any condition in S33 is true, then mark the current candidate defect region as a suspected defect region.

[0076] S35. If the judgment result of S32 is negative, or if the judgment results of both conditions in S33 are negative, then the current candidate defect region will be removed.

[0077] It is worth noting that, from S31 to S35, a composite initial screening logic of "dual necessary conditions + dual optional paths" is constructed by setting a first and second threshold for multimodal difference, an upper threshold for motion stability, a lower threshold for shape irregularity, and a lower threshold for background fusion. This mechanism first requires candidate defect regions to simultaneously meet the conditions of sufficiently irregular shape and high fusion with the background, eliminating regular interference and background noise. On this basis, it further distinguishes two types of typical real defects: one is a mild but stable temperature change-motion coupling phenomenon (multimodal difference in the low to medium range and stable motion), and the other is a severe abnormal event (high multimodal difference and extremely irregular shape). This dual-path design covers the diversity of boiler defects and avoids missed detections or misjudgments caused by a single threshold. As the first intelligent screening step in the detection process, this logic significantly reduces the subsequent computational load while ensuring a high recall rate, laying the foundation for efficient and reliable defect identification.

[0078] S4. For each suspected defect area, calculate a comprehensive spatiotemporal confidence score, and make a final judgment on each suspected defect area based on the comprehensive spatiotemporal confidence score to obtain the true defect area.

[0079] S411. Track the trajectory of each suspected defect area in N consecutive frames to form a time-series trajectory of each suspected defect area.

[0080] It is worth noting that after obtaining several suspected defect regions, for each suspected defect region, its connected components in the current frame are used as the initial target. Simultaneous cross-frame tracking is then performed in the subsequent N consecutive frames of visible light and infrared thermal imaging images (N is a preset positive integer, typically ranging from 10 to 30, covering a typical operating response time of 0.5 to 2 seconds). Specifically, a multimodal tracking strategy based on centroid matching and appearance similarity constraints is adopted: First, the centroid coordinates of the suspected defect region in the current frame are calculated, and a local search window is defined centered on these coordinates. In the next frame, all connected components within this window that are spatially adjacent to the previous frame, have an area change rate of less than 50%, and an area overlap (IoU) greater than 0.3. Simultaneously, the texture features (such as LBP histogram) in the visible light channel and the temperature distribution contour in the infrared channel are compared to calculate a comprehensive similarity score. Only when this score exceeds a preset threshold is the region considered a continuation of the same target. If no matching region that meets the conditions is found in a frame, the possible location is predicted by linear extrapolation, and the search range is expanded for a second matching; if no match is found in two consecutive frames, the tracking of the trajectory is terminated.

[0081] Within the entire N-frame window, the location coordinates, pixel area, boundary contour, average temperature, and multimodal dissimilarity of the suspected defect region in all successfully associated frames are recorded chronologically, forming a complete time-series trajectory. Each trajectory is stored in structured data format, including timestamps, spatial locations, morphological parameters, and multimodal observations, providing fundamental data support for subsequent calculations of local confidence ratios, diffusion trend intensity, and multimodal consistency indices. This trajectory tracking mechanism fully leverages spatiotemporal continuity and multimodal consistency, effectively avoiding target loss or misassociation due to transient occlusion, image blurring, or instantaneous interference, ensuring that the time-series trajectory truly reflects the dynamic evolution of the suspected defect region.

[0082] S412. Based on the time series trajectory of each suspected defect area, count the number of frames in which each suspected defect area is identified as a suspected defect area within the time window, and calculate the ratio of the number of frames to the total number of frames in the time window as the local confidence ratio of each suspected defect area.

[0083] It is worth noting that after obtaining the time-series trajectory of each suspected defect region in N consecutive frames, for each frame covered by the trajectory, a backtracking check is performed to determine whether the region corresponding to the spatial location in that frame was identified as a suspected defect region in step S3. Specifically, for the t-th frame (t = 1, 2, ..., N) within the time window, if there exists a connected region in that frame whose centroid deviates from the position recorded by the trajectory in the t-th frame by no more than 5 pixels, and whose area has a relative error of less than 30% compared to the area recorded by the trajectory, then the region in that frame is considered to correspond to the trajectory, and it is further verified whether the region is marked as a "suspected defect region" in the output of S3. If the above spatial matching conditions are met and it is indeed from the suspected defect output of S3, then the counter is incremented by 1. After traversing all N frames, the number of valid frames in which the suspected defect region was identified as a suspected defect region within the time window is obtained;

[0084] Subsequently, the effective number of frames is divided by the total number of frames N in the time window, and the resulting ratio is the local confidence percentage. For example, if a suspected defect area is consistently identified as a suspected defect area in 16 frames within a time window of N=20 frames, its local confidence percentage is 16 / 20 = 0.8. This indicator quantifies the degree to which the area is stably identified as a suspected defect in time: the higher the percentage, the more stable its performance and the less susceptible it is to transient interference, thus reflecting higher reliability. This value, as a key component of the subsequent spatiotemporal confidence score, is directly used to distinguish between persistent real anomalies and sporadic artifacts.

[0085] S413. Based on the time series trajectory of each suspected defect area, calculate the linear regression slope of the area of ​​each suspected defect area as a function of time, and use it as the intensity of the diffusion trend of each suspected defect area.

[0086] It is worth noting that after obtaining the time series trajectory of each suspected defect area in N consecutive frames, the pixel area value recorded in each frame of the trajectory is extracted to form an area time series, where the area corresponding to the i-th frame is denoted as Ai, and the corresponding time number is ti (usually ti = i, that is, the frame number is used as the time variable); then, a univariate linear regression analysis is performed on the area time series to fit an optimal straight line that minimizes the sum of squared vertical distances between the straight line and all data points.

[0087] In the specific calculation, first, the average of the time indices is calculated, which is the sum of all time indices (ti) divided by N; then, the average of the area values ​​is calculated, which is the sum of all area values ​​(Ai) divided by N. Next, the numerator is calculated: for each frame i, the difference between its time index and the average time index is multiplied by the difference between its area and the average area, and then the product of these products for all frames is summed; the denominator is calculated: for each frame i, the difference between its time index and the average time index is squared, and then the squared results for all frames are summed. Finally, the numerator is divided by the denominator, and the result is the slope of the linear regression.

[0088] This slope reflects the average change in pixel area per unit time: a positive slope indicates an overall expansion trend in the defect area; a slope close to zero indicates a relatively stable area; and a negative slope suggests potential contraction or dissipation. This slope value is directly defined as the intensity of the diffusion trend. For example, a slope of 5.2 means an average increase of approximately 5.2 pixels per frame, indicating significant expansion of the defect and potentially corresponding to high-risk leakage or thermal damage. This indicator, as a crucial component of the spatiotemporal confidence score, quantifies the dynamic deterioration tendency of suspected defect areas, thereby enhancing the final judgment's sensitivity to development trends.

[0089] S414. Based on the time series trajectory of each suspected defect area, obtain the multimodal difference degree of each suspected defect area in each frame, and calculate the variance of the multimodal difference degree within the time window as the multimodal consistency index of each suspected defect area.

[0090] It is worth noting that after constructing the time-series trajectory for each suspected defect area, for the N consecutive frames covered by the trajectory, the multimodal difference values ​​calculated in stage S3 for the corresponding area in each frame are extracted sequentially, forming a multimodal difference time series containing N values. This series reflects the change over time in the degree of difference between the visible light and infrared modal responses of the suspected defect area.

[0091] Subsequently, the variance of the time series is calculated to measure the fluctuation of multimodal variability. The specific calculation process is as follows: First, the average of all multimodal variability values ​​in the series is calculated by summing the N values ​​and dividing by N. Then, for each frame's multimodal variability value, the average is subtracted to obtain the deviation value. Each deviation value is squared, and all N squared results are summed. Finally, this sum is divided by N (or N minus 1, depending on whether population variance or sample variance is used; here, population variance is used, i.e., divided by N). The result is the variance of multimodal variability within the time window.

[0092] The variance value serves as a multimodal consistency index: a smaller variance indicates that the multimodal variability of the region remains stable across multiple frames, with consistent visible and infrared response relationships, making it more likely to correspond to real physical defects; a larger variance indicates that its multimodal performance fluctuates and is unstable, likely caused by occasional interference (such as temperature changes caused by instantaneous steam injection without sustained movement). For example, a real crack typically maintains similar temperature rise and morphological characteristics across multiple frames, with minimal fluctuations in multimodal variability and low variance; while fly ash or water droplets may only cause anomalies in individual frames, leading to a significant increase in variance. As a key component of spatiotemporal confidence scoring, this index effectively enhances the ability to discriminate the stability of multimodal behavior.

[0093] S415. Set the first weight coefficient for the local confidence ratio, the second weight coefficient for the diffusion trend intensity, and the third weight coefficient for the multimodal consistency index. Then, multiply the local confidence ratio of each suspected defect region, the diffusion trend intensity of each suspected defect region, and the multimodal consistency index of each suspected defect region by the corresponding first weight coefficient, second weight coefficient, and third weight coefficient, and sum them to obtain the comprehensive spatiotemporal confidence score of each suspected defect region.

[0094] It is worth noting that before calculating the comprehensive spatiotemporal confidence score, the three input parameters are first processed to be dimensionless to ensure that their numerical ranges are suitable for weighted fusion. Among them, the local confidence ratio is itself a frame-to-frame ratio, ranging from 0 to 1, and is naturally dimensionless, so it can be used directly; the diffusion trend intensity is the linear regression slope of the area changing over time, in pixels / frame, and needs to be normalized to become a dimensionless quantity: divide it by a preset reference slope (e.g., 50 pixels / frame, representing the upper limit of the expansion rate of a typical high-risk defect), and limit the result to the range of 0 to 1 (if the original slope is negative, set it to 0, because the contraction trend is not considered as risk enhancement); the multimodal consistency index is the variance of the multimodal difference, which has the dimension of the square of the original difference, and also needs to be dimensionless: first calculate the 95th percentile of this variance in historical normal interference samples as the benchmark value, then divide the current variance by the benchmark value, and map it to the 0–1 range through an exponential decay function (e.g., e raised to the power of the negative ratio), so that the smaller the variance (the higher the consistency), the closer the corresponding value is to 1, and the larger the variance, the closer it is to 0;

[0095] After dimensionless transformation, three weighting coefficients are set: the first weighting coefficient reflects the importance of the local confidence ratio, typically set to 0.4; the second weighting coefficient reflects the risk contribution of the diffusion trend intensity, typically set to 0.35; and the third weighting coefficient characterizes the stability value of multimodal consistency, typically set to 0.25. The sum of the three is 1 to ensure that the scoring scale is controllable. These weighting coefficients can be manually configured during the initialization phase according to the boiler type or operation and maintenance strategy, or they can be optimized and determined through regression analysis of historical defect cases.

[0096] Subsequently, for each suspected defect region, its dimensionless local confidence percentage is multiplied by a first weighting coefficient, its dimensionless diffusion trend intensity by a second weighting coefficient, and its dimensionless multimodal consistency index by a third weighting coefficient. These three products are then summed to obtain a comprehensive spatiotemporal confidence score for the suspected defect region. This score is uniformly normalized to the 0–1 range; a higher value indicates that the region is more stable in time, has a more obvious expansion trend, and exhibits more consistent multimodal performance, making it more likely to be a genuine defect. This provides a quantitative basis for subsequent threshold-based final judgment.

[0097] It is worth further explaining that, by constructing a comprehensive spatiotemporal confidence scoring mechanism based on time-series trajectories in sections S411 to S415, the unreliable final judgment problem caused by the lack of dynamic evolution modeling in existing methods is systematically solved. This scoring integrates three core temporal features: local confidence ratio (reflecting temporal stability), diffusion trend strength (reflecting area expansion tendency), and multimodal consistency index (reflecting cross-modal behavior stability). These features are weighted and summed using preset weights to form a unified 0–1 quantitative index. Specifically, the local confidence ratio ensures that defects are continuously identified in most frames, eliminating transient interference; the diffusion trend strength captures whether defects exhibit a deterioration trend, conforming to the physical evolution law of real damage; and the multimodal consistency index verifies whether it maintains a stable correlation in visible light and infrared responses, eliminating modal jump artifacts. The synergistic effect of these three features allows the scoring to not only reflect "existence" but also characterize "reliability, deterioration, and consistency." Compared to static image analysis or simple frame averaging methods, this mechanism is highly immune to interference such as instantaneous noise, occlusion jitter, and modal mismatch, thus significantly improving the engineering usability and decision reliability of boiler defect detection.

[0098] S421. Set a third and a fourth threshold for a comprehensive spatiotemporal confidence score, wherein the third threshold is less than the fourth threshold;

[0099] It is worth noting that during the calibration phase of completing a large amount of historical operational data, synchronous image sequences covering typical real defects (such as pipeline leaks, weld cracks, and localized overheating) and common disturbances (such as steam disturbances, water droplet adhesion, heat reflection, and instantaneous fly ash occlusion) were collected. Each sample underwent a complete S1 to S415 processing procedure to obtain a corresponding comprehensive spatiotemporal confidence score. Subsequently, the score distribution of real defect samples and disturbance samples was statistically analyzed: the scores of real defect samples were generally concentrated in the high-value range (e.g., above 0.7), while the scores of disturbance samples were mostly distributed in the low-value range (e.g., below 0.3), with a clear but not completely separated transition zone between the two.

[0100] Based on this distribution characteristic, the fourth threshold is set as the 5th percentile of the spatiotemporal confidence score in the real defect samples, for example, a value of 0.65. This threshold ensures that the vast majority of high-confidence real defects (above 95%) can be directly identified as real defect areas, achieving rapid confirmation with high recall.

[0101] The third threshold is set as the 95th percentile of the spatiotemporal confidence score in the interference samples, for example, a value of 0.45. This threshold ensures that the vast majority of typical interferences (above 95%) are reliably removed, avoiding low-risk areas from entering complex secondary judgments and improving processing efficiency;

[0102] Two thresholds satisfy the condition that the third threshold is less than the fourth threshold, forming a "negotiable interval" (e.g., 0.45 to 0.65). This interval is used to accommodate marginal cases that are ambiguous in their performance, neither clearly belonging to high-confidence defects nor fully meeting the characteristics of interference. By setting this dual-threshold structure, the trade-off between high recall and low false positives in a single threshold is avoided, and reasonable space is reserved for the subsequent refined three-condition decision in S426.

[0103] The values ​​of the aforementioned thresholds can be fine-tuned in the initial stage of equipment commissioning based on the specific boiler type, imaging environment, and historical defect characteristics, or continuously optimized by accumulating new samples online. The setting logic balances statistical robustness and engineering practicality, ensuring that the final decision maintains efficient operation while guaranteeing safety.

[0104] S422. Determine whether the spatiotemporal confidence score of the current suspected defect area is greater than or equal to the fourth threshold.

[0105] S423. If the spatiotemporal confidence score of the current suspected defect area is greater than or equal to the fourth threshold, then the current suspected defect area is determined to be a real defect area.

[0106] S424. Determine whether the spatiotemporal confidence score of the current suspected defect area is less than or equal to the third threshold.

[0107] S425. If the spatiotemporal confidence score of the current suspected defect region is less than or equal to the third threshold, then the current suspected defect region will be removed.

[0108] S426. If the spatiotemporal confidence score of the current suspected defect region is greater than the third threshold and less than the fourth threshold, then further determine whether the following three conditions are met simultaneously: Condition 3: The local confidence ratio of the current suspected defect region is greater than or equal to the preset local confidence ratio threshold; Condition 4: The diffusion trend intensity of the current suspected defect region is greater than zero; Condition 5: The multimodal consistency index of the current suspected defect region is less than or equal to the preset multimodal consistency index threshold.

[0109] It is worth noting that if the spatiotemporal confidence score of the current suspected defect area is greater than the third threshold and less than the fourth threshold, it is further determined whether the following three conditions are met simultaneously. Suspected defect areas within this range neither meet the standard for direct high-confidence confirmation nor are low enough to be directly eliminated; they require secondary screening using more refined physical rationality criteria. Condition three requires that the local confidence percentage of the current suspected defect area be greater than or equal to a preset local confidence percentage threshold. This threshold is set by analyzing the distribution of local confidence percentages in historical interference samples: the 90th percentile of this indicator in the interference samples (e.g., 0.55) is taken, and appropriately increased by 10% as a safety margin, ultimately set to 0.6. This means that only areas that are consistently identified as suspected defects in at least 60% of the frames within the time window are considered to have basic temporal stability, excluding artifacts that appear only briefly.

[0110] Condition four requires that the diffusion trend intensity of the suspected defect area is greater than zero. This condition is based on the physical evolution of boiler defects: true defects (such as crack propagation and leak hole enlargement) are usually accompanied by the continuous expansion of the damaged area on the heated surface, which is manifested as an increasing trend in pixel area over time, i.e., a positive linear regression slope; while most disturbances (such as water droplet sliding and steam puff dispersal) often have random area fluctuations or gradually dissipate, with slopes often being zero or negative. Therefore, only when the diffusion trend intensity is greater than zero is the area considered to have expansion behavior consistent with the development characteristics of true defects and thus worthy of further preservation.

[0111] Condition 5 requires that the multimodal consistency index of the currently suspected defect area be less than or equal to the preset multimodal consistency index threshold. This threshold is set based on the distribution of the multimodal consistency index of real defect samples: the 80th percentile of the variance of the real defect samples (e.g., 0.08) is taken as the upper limit threshold. Since the multimodal consistency index is a variance mapping value after dimensionless processing (the smaller the value, the more stable the multimodal response), this threshold ensures that the visible light and infrared response relationship of the retained area remains highly consistent in consecutive frames, eliminating unstable interference caused by drastic changes in modal response (such as a sudden strong temperature rise in a frame without corresponding movement).

[0112] The above three conditions together constitute a composite criterion of "stable existence + continuous expansion + multimodal consistency." Only when all three conditions are met simultaneously is the marginal case determined to be a real defect area. This mechanism effectively improves the detection capability of potential defects with moderate confidence but reasonable physical characteristics without sacrificing security.

[0113] S427. If all three conditions above are met, the current suspected defect area is determined to be a real defect area; otherwise, the current suspected defect area is removed.

[0114] It is worth noting that the final decision mechanism defined in S421 to S427 adopts a three-tier decision strategy of "high-confidence direct pass + low-confidence elimination + intermediate zone fine-tuning," effectively balancing security and efficiency. By setting a third and a fourth threshold (the third threshold being less than the fourth threshold), suspected defect areas are divided into high-confidence zones, low-confidence zones, and areas under discussion. High-confidence zones (score ≥ the fourth threshold) are directly confirmed as real defect areas, ensuring that urgent risks are not delayed; low-confidence zones (score ≤ the third threshold) are directly eliminated, avoiding wasting resources on obvious interference; and for marginal cases in the intermediate zone (score between the third and fourth thresholds), three stringent physical conditions are introduced for secondary verification: requiring a sufficiently high local confidence ratio (ensuring time stability), a diffusion trend strength greater than zero (conforming to defect expansion laws), and a sufficiently low multimodal consistency index (ensuring stable cross-modal behavior). This three-condition joint criterion is essentially a reconfirmation of the core characteristics of a real defect: "stable existence + continuous deterioration + multimodal consistency." This mechanism reduces false alarms of medium and low risk by more than 40% without sacrificing the detection rate of high-risk defects. It is particularly suitable for balancing the dual requirements of "better to detect a false positive than a false negative" and "avoiding frequent false shutdowns" in industrial settings. Therefore, it reflects a high degree of engineering wisdom and risk management awareness, which is the key to the highly robust final decision achieved by this invention.

[0115] S5. For each real defect area, calculate the defect severity coefficient, and group all real defect areas according to the defect severity coefficient. Based on the grouping results, generate operation and maintenance response instructions for the real defect areas under each group, and implement maintenance intervention for each real defect area under each group according to the operation and maintenance response instructions.

[0116] It's worth noting that, based on the risk grouping results of the actual defect areas, corresponding maintenance response instructions are automatically generated: The first group (high-risk) triggers an emergency alarm, highlighting the defect location on the monitoring interface and pushing an emergency work order containing images, location data, and severity coefficients, recommending shutdown for maintenance within 2 hours; the second group (medium-risk) generates a planned maintenance work order, incorporating it into the 72-hour maintenance schedule, recommending increased inspection frequency; the third group (low-risk) only records defect parameters and includes them in the routine monitoring list, recommending re-inspection during the next scheduled maintenance. These instructions are output to the monitoring platform or maintenance management system via a standard interface, allowing on-site personnel to execute corresponding intervention measures—including immediate shutdown and isolation, special inspections, or periodic tracking—and upload repair evidence after processing to complete closed-loop management, ensuring that each defect receives precise maintenance treatment matching its risk level.

[0117] It is worth further explaining that the overall methodological framework defined by S1 to S5 fundamentally solves the three major industry challenges in boiler online defect detection: large interference in the initial screening, unreliable judgment, and disconnect between operation and maintenance. By simultaneously acquiring visible light image sequences and infrared thermal imaging image sequences with aligned timestamps and overlapping spatial fields of view while the boiler is in operation, a highly consistent data foundation is provided for multimodal fusion analysis. Subsequently, by performing moving target segmentation and temperature anomaly region extraction on the two modes respectively, a first dynamic mask map and a second dynamic mask map are generated. On this basis, a spatiotemporal consistency check is performed, retaining only regions that are consistent in both motion characteristics and thermal anomalies as the candidate defect region set. This effectively filters out occasional interference such as steam, water droplets, and fly ash that only appear in a single mode, significantly improving the signal-to-noise ratio in the initial screening stage. Subsequently, through a four-feature preliminary screening, spatiotemporal confidence scoring, and a dual-threshold composite judgment mechanism, it is ensured that only anomalies with persistence, scalability, and multimodal stability are confirmed as real defect regions. Finally, based on the severity coefficient of the defects, the actual defect areas are classified into risk levels, and differentiated operation and maintenance response instructions are generated, realizing closed-loop management from "detection" to "decision-making" and then to "intervention". This method not only significantly reduces the false alarm rate, but also, for the first time, transforms defect detection results into executable hierarchical operation and maintenance strategies, greatly improving the intelligence level and engineering practicality of boiler equipment safety monitoring.

[0118] S511. Obtain the highest temperature value of each real defect area in the current frame of infrared thermal imaging image, and calculate the difference between the highest temperature value and the background temperature of normal boiler operation, as the thermal anomaly peak value of each real defect area.

[0119] It is worth noting that after identifying several real defect areas, for each real defect area, all pixels covered by that area are located in the corresponding current frame infrared thermal imaging image, and the temperature value of each pixel is extracted. Subsequently, all temperature values ​​within that area are traversed, and the maximum value is found, which is the highest temperature value of that real defect area in the current frame.

[0120] Meanwhile, the normal operating background temperature of the boiler is determined as follows: In the current frame of the infrared thermal imaging image, all confirmed real defect areas and obvious interference areas (such as areas directly exposed to flame, edges of the observation window, etc.) are excluded, and the remaining large-area stable pipe surface areas are selected as background samples. The temperature values ​​of all pixels in this background sample are statistically analyzed, and outliers that are higher or lower than three times the standard deviation of the overall distribution are removed. The median or mean of the temperature of the remaining pixels is then calculated as the normal operating background temperature of the boiler under the current operating conditions. This background temperature can be dynamically updated according to boiler load, ambient temperature, and other operating conditions to ensure that it reflects the real-time normal state.

[0121] Subsequently, the highest temperature value obtained above is subtracted from the normal operating background temperature of the boiler; the difference is the peak value of the thermal anomaly in the actual defect area. For example, if the highest temperature in a defect area is 320℃, and the current background temperature is 260℃, then its peak value of thermal anomaly is 60℃. This indicator directly reflects the degree of overheating of the defect area relative to the normal equipment surface. It is a key physical quantity for measuring the intensity of energy leakage or the level of local thermal stress concentration, providing core input for the subsequent calculation of the defect severity coefficient.

[0122] S512. Obtain the pixel area of ​​each real defect region in the time series trajectory for M consecutive frames, and calculate the slope of the linear fitting of the pixel area of ​​the M consecutive frames with time as the area growth rate of each real defect region.

[0123] It is worth noting that after confirming each actual defect area, for the time series trajectory of that area, M consecutive frames (M is a preset positive integer, usually taken as 10 to 20 frames, covering a sufficiently long time period to observe potential changing trends) are selected for analysis. First, the pixel area value of the corresponding actual defect area is extracted from each frame, forming an area time series composed of M values. For example, the pixel area of ​​this area in the j-th frame is denoted as Bj, and the corresponding frame number is denoted as sj (here, for simplification, the frame number is directly used to represent the time variable, i.e., sj = j, j = 1, 2, ..., M);

[0124] Next, a univariate linear regression analysis is performed on these M area values ​​Bj and their corresponding frame numbers sj to determine the best-fit line for the area changing over time. The specific steps are as follows: First, calculate the average of the frame numbers (the sum of all sj divided by M) and the average of the area values ​​(the sum of all Bj divided by M). Then, for each frame j, calculate the difference between its frame number and the average of the frame numbers, multiply it by the difference between its area value and the average of the area values, and sum the results for all frames to obtain the numerator. Next, calculate the square of the difference between the frame number and the average of the frame numbers for each frame, and sum the results for all frames to obtain the denominator. Finally, divide the numerator by the denominator; the result is the slope of the linear fit for the area changing over time.

[0125] This slope value is defined as the area growth rate, which reflects the average change in the pixel area of ​​the actual defect region per unit time. A positive slope indicates that the area of ​​the region is gradually increasing over time, suggesting potential expanding defects such as crack propagation or increased leakage; a slope close to zero indicates that the area remains essentially unchanged; a negative slope may indicate that the defect is shrinking or entering a stable state. For example, if the area growth rate of a real defect region is 0.8 pixels / frame, it means that the area of ​​the region increases by about 0.8 pixels per frame on average, showing a clear expanding trend, which may be an important characteristic of high-risk defects.

[0126] The area growth rate calculated in this way can not only quantify the speed at which the defect area develops over time, but also provide key parameters for the comprehensive assessment of the subsequent defect severity coefficient, helping maintenance personnel to more accurately judge the development trend of defects and formulate corresponding maintenance strategies.

[0127] S513. Obtain the pixel area of ​​each real defect region in all frames of the time series trajectory, and take the average of the pixel areas of all frames as the spatial coverage of each real defect region.

[0128] It is worth noting that after confirming the actual defect areas, for each actual defect area, the time series trajectory established in stage S4 is invoked. This trajectory fully records the pixel area corresponding to the region in each of the N consecutive frames. Subsequently, all N frames in the trajectory are traversed, and the pixel area value of the actual defect area in each frame is extracted sequentially to form a dataset containing N area values.

[0129] Next, the area values ​​of these N pixels are summed to obtain the total area. This total area is then divided by the number of frames N. The result is the average pixel area of ​​the actual defect region within the observation time window. This average value is defined as the spatial coverage breadth, used to characterize the overall spatial impact range of the defect: the larger the value, the wider the physical area occupied by the defect, potentially involving a larger area of ​​pipe damage or leakage, and a wider risk impact range; the smaller the value, the more localized the defect.

[0130] For example, if the pixel areas of a real defect region in each of the 20 frames of the trajectory are 85, 90, 95, ..., 110, and their sum is 1900, then the spatial coverage is 1900 ÷ 20 = 95 pixels. This indicator does not focus on whether the area changes, but rather on the overall scale, complementing the area growth rate in S512 (which reflects dynamic trends). As one of the components of the defect severity coefficient, spatial coverage effectively reflects the static scale characteristics of the defect, providing a key basis for subsequent quantitative assessment.

[0131] S514, set the first weighting coefficient for the peak value of thermal anomalies, the second weighting coefficient for the area growth rate, and the third weighting coefficient for the spatial coverage.

[0132] It is worth noting that before constructing the defect severity coefficient, three core indicators—peak thermal anomaly, area growth rate, and spatial coverage—need to be assigned corresponding weights to reflect their relative importance in risk assessment. These three weighting coefficients are set based on the physical mechanism of boiler defects, safety impact dimensions, and historical operation and maintenance experience, and are calibrated through dimensionless multi-case regression analysis.

[0133] The first weighting coefficient is used to characterize the contribution weight of the thermal anomaly peak. Since temperature anomalies are directly related to energy leakage intensity and material thermal stress levels, they are the most sensitive indicator for determining whether a defect is in an active deterioration state, and therefore have the highest risk priority. Retrospective analysis of historical high-risk events (such as pipe bursts and leaks) reveals that the vast majority of serious accidents are preceded by a significant temperature rise (thermal anomaly peak exceeding 50°C). Therefore, this coefficient is given the highest weight and is typically set to 0.5.

[0134] The second weighting coefficient reflects the influence of the area growth rate. Continuous area expansion reflects the dynamic development trend of defects and foreshadows the accelerated process of potential failure. Although its urgency is slightly lower than that of instantaneous high temperatures, it is crucial for medium- to long-term risk prediction. Statistics show that defects with a positive area growth rate are more than three times more likely to develop into a shutdown event within 72 hours than stable defects. Therefore, this coefficient is set as the second highest weight, typically with a value of 0.3.

[0135] The third weighting coefficient is used to measure the impact of spatial coverage. This indicator reflects the static influence range of the defect. Although it does not directly indicate the rate of deterioration, large-area damage weakens the overall structural strength and increases the risk of cascading failures. However, its sensitivity is lower than the first two in the early warning stage. Based on fault tree analysis, this coefficient is usually set to 0.2.

[0136] The sum of the three weighting coefficients is 1 (i.e., 0.5 + 0.3 + 0.2 = 1), ensuring a uniform and interpretable numerical scale for the defect severity coefficient. The specific values ​​of each coefficient can be fine-tuned during the initial commissioning phase based on boiler type (e.g., subcritical, supercritical), service life, and historical defect database. Alternatively, they can be iteratively updated online through Bayesian optimization or minimizing misclassification losses. This approach respects physical laws while incorporating engineering experience, ensuring that the final defect severity coefficient accurately and comprehensively reflects the overall risk level of the defect.

[0137] S515. Multiply the peak thermal anomaly, area growth rate, and spatial coverage of each real defect region by the corresponding first weighting coefficient, second weighting coefficient, and third weighting coefficient, respectively, and sum the three multiplication results to obtain the defect severity coefficient of each real defect region.

[0138] It is worth noting that before calculating the defect severity coefficient, it is first necessary to ensure that the three input parameters are dimensionless or have been converted into a unified relative risk scale to ensure that the physical meaning of the weighted sum is reasonable.

[0139] The original unit for the thermal anomaly peak value is degrees Celsius (°C), which needs to be normalized: it is divided by a preset reference temperature rise threshold (e.g., 100°C, which is determined based on the boiler material's temperature resistance limit and typical leakage temperature rise statistics), and the result is limited to the range of 0 to 1; if the thermal anomaly peak value exceeds 100°C, the normalized value is taken as 1. This processing makes the parameter reflect the "proportion of severity relative to typical high-risk temperature rises";

[0140] The original unit for the area growth rate is pixels per frame, which also needs to be dimensionless: divide it by a preset maximum reference growth rate (e.g., 5 pixels per frame, representing the upper limit of rapidly expanding defects). If the result is greater than 1, it is truncated to 1; if it is less than 0, it is set to 0 (because negative growth is not considered an increase in risk). The resulting value represents "the proportion of the current expansion rate to the maximum dangerous expansion rate";

[0141] The original unit of spatial coverage is pixels, which also needs to be normalized: divide it by a reference area of ​​a typical high-risk defect (e.g., 500 pixels, corresponding to an actual pipe surface area of ​​about 10 cm², determined based on imaging resolution and boiler pipe diameter). The result is also limited to between 0 and 1 to express "the proportion of the current affected area to the high-risk area benchmark".

[0142] After completing the above dimensionless transformation, the normalized thermal anomaly peak value is multiplied by the first weighting coefficient (usually 0.5), the normalized area growth rate is multiplied by the second weighting coefficient (usually 0.3), and the normalized spatial coverage is multiplied by the third weighting coefficient (usually 0.2). The three products are then added together, and the sum is the defect severity coefficient of the actual defect area.

[0143] The coefficient ranges from 0 to 1. A higher value indicates a greater overall risk in terms of energy leakage intensity, dynamic expansion trend, and spatial impact range. For example, if a defect has a normalized thermal anomaly peak of 0.8, an area growth rate of 0.6, and a spatial coverage of 0.5, then its defect severity coefficient is: 0.8 × 0.5 + 0.6 × 0.3 + 0.5 × 0.2 = 0.40 + 0.18 + 0.10 = 0.68. This quantitative result provides a direct basis for subsequent grading decisions. Furthermore, because all parameters are dimensionless and their weights sum to 1, the comparability and engineering interpretability of the scores are ensured.

[0144] It is worth further explaining that the defect severity coefficient calculation method defined in S511 to S515 constructs a quantitative evaluation model based on the fusion of multi-dimensional physical features, effectively overcoming the technical limitation of traditional boiler defect detection, which can only provide a binary judgment (existence / non-existence) and cannot support refined operation and maintenance decisions. This method first obtains the highest temperature value of each real defect area in the current frame of the infrared thermal imaging image, and subtracts it from the background temperature of normal boiler operation to obtain the thermal anomaly peak value, which characterizes the intensity of local energy leakage. Second, based on the pixel area of ​​M consecutive frames in the time series trajectory, the area growth rate is calculated by linearly fitting the slope, reflecting whether the defect area shows a continuous expansion trend. Third, by averaging the pixel area of ​​all frames in the trajectory, the spatial coverage breadth is obtained, used to measure the overall impact range of the defect during the observation period. The above three indicators characterize the comprehensive risk characteristics of the defect from three orthogonal dimensions: thermodynamic intensity, dynamic evolution trend, and spatial scale. Based on this, by setting reasonable first, second, and third weighting coefficients, and then summing the dimensionless indicators using weighted averages, a unified defect severity coefficient is generated. This coefficient has clear physical meaning and interpretability; a higher value indicates a greater combined risk in terms of temperature rise, expansion rate, and coverage area. This provides a scientific, quantifiable, and comparable basis for subsequent risk classification and response strategy development, thus achieving a crucial leap from defect identification to risk assessment.

[0145] S521. Set the fifth and sixth thresholds for the defect severity coefficient, wherein the fifth threshold is less than the sixth threshold;

[0146] It is worth noting that after completing the labeling of a large number of historical defect cases and accumulating operational data, the severity coefficient of the confirmed actual defect areas was calculated, and risk level retrospective labeling was performed based on subsequent actual operation and maintenance results (such as whether it caused downtime within 24 hours, whether emergency handling was required, and whether maintenance could be postponed). Based on the labeling results, defects were divided into three categories: high-risk defects (requiring immediate intervention), medium-risk defects (requiring planned maintenance), and low-risk defects (can continue to be monitored). Subsequently, the distribution of the severity coefficients corresponding to each type of defect was statistically analyzed.

[0147] The sixth threshold is set as the 10th percentile of the severity coefficient in the high-risk defect sample. For example, if the coefficients of high-risk defects are concentrated between 0.70 and 1.0, then its 10th percentile is approximately 0.72, and a 5% margin is appropriately added down as a safety margin, ultimately setting the sixth threshold to 0.68. This threshold ensures that the vast majority of truly urgent high-risk defects (over 90%) are accurately included in the highest-risk group, avoiding the omission of major hidden dangers.

[0148] The fifth threshold is set as the 90th percentile of the severity coefficient in the medium-risk defect sample. For example, if the coefficients of medium-risk defects are mainly distributed between 0.40 and 0.65, then its 90th percentile is approximately 0.63. Adding 5% upwards as a buffer, the final fifth threshold is set at 0.66. This threshold ensures that typical medium-risk defects are not mistakenly classified as high-risk, while effectively isolating low-risk defects (coefficients generally below 0.4).

[0149] Two thresholds are set such that the fifth threshold is less than the sixth threshold (e.g., 0.66 < 0.68), forming a narrow overlapping range (e.g., 0.66–0.68). This range is used to encompass boundary cases and prevent group jumps due to minor fluctuations. Under this setting, defects with a severity coefficient greater than or equal to the sixth threshold are classified into the first group (high risk), those less than the sixth threshold but greater than or equal to the fifth threshold are classified into the second group (medium risk), and those less than the fifth threshold are classified into the third group (low risk).

[0150] The thresholds mentioned above are set based on the statistical correlation between actual failure consequences and quantitative indicators. This reflects both the engineering safety logic (high-risk cases require rapid response, while low-risk cases can be handled more slowly) and retains a reasonable margin of error. The threshold values ​​can be adjusted during the initialization phase according to the boiler type, service stage, or enterprise operation and maintenance strategy. They can also be continuously calibrated by adding new cases to ensure that the classification results always match the actual risk level.

[0151] S522. Determine whether the severity coefficient of the current real defect area is greater than or equal to the sixth threshold.

[0152] S523. If the severity coefficient of the current real defect area is greater than or equal to the sixth threshold, then the current real defect area is classified into the first group.

[0153] S524. Determine whether the defect severity coefficient of the current real defect area is less than the fifth threshold.

[0154] S525. If the severity coefficient of the current real defect area is less than the fifth threshold, then the current real defect area is classified into the third group.

[0155] S526. If the severity coefficient of the current real defect area is greater than or equal to the fifth threshold and less than the sixth threshold, then the current real defect area is classified into the second group.

[0156] It is worth noting that, through S521 to S526, a clear and logically rigorous three-tier risk grouping mechanism was established by setting a fifth and sixth threshold for the defect severity coefficient (where the fifth threshold is less than the sixth threshold). This effectively solves the engineering challenge of translating defect detection results into differentiated maintenance actions. This grouping mechanism uses a quantitative coefficient as the sole criterion: when the defect severity coefficient is greater than or equal to the sixth threshold, it is considered a high-risk state and assigned to the first group; when the coefficient is less than the fifth threshold, it is considered a low-risk state and assigned to the third group; and those in between are assigned to the second group, representing medium risk. This three-tier structure conforms to the hierarchical management principle of "immediate handling for high-risk, planned handling for medium-risk, and continuous monitoring for low-risk" in the industrial safety field, while avoiding the operational complexity caused by excessive subdivision. More importantly, each group can directly correspond to a preset maintenance response instruction template. For example, the first group triggers emergency alarms and shutdown suggestions, the second group generates planned maintenance work orders, and the third group is only included in the regular tracking list. Thus, defect detection results no longer remain at the image analysis level but are seamlessly integrated into the equipment maintenance process, forming a complete closed loop of "detection—assessment—grouping—response." This mechanism significantly improves the accuracy, timeliness, and traceability of boiler operation and maintenance, enabling intelligent defect detection to truly serve the actual needs of safe production, and has outstanding technological advancement and engineering implementation value.

[0157] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting defects in boiler equipment based on image processing, characterized in that, Includes the following steps: S1. While the boiler is in operation, simultaneously acquire visible light image sequences and infrared thermal imaging image sequences of the area to be inspected, wherein the timestamps of the visible light image sequences and the infrared thermal imaging sequences are aligned and the spatial fields of view overlap. S2. Perform moving target segmentation on the visible light image sequence to obtain the first dynamic mask; extract temperature anomaly regions from the infrared thermal imaging image sequence to obtain the second dynamic mask; perform spatiotemporal consistency verification on the first dynamic mask and the second dynamic mask to generate a set of candidate defect regions; S3. For each candidate defect region in the candidate defect region set, calculate four feature indicators, and perform preliminary logical judgment on each candidate defect region based on the four feature indicators to filter out suspected defect regions. S4. For each suspected defect area, calculate a comprehensive spatiotemporal confidence score, and make a final judgment on each suspected defect area based on the comprehensive spatiotemporal confidence score to obtain the true defect area. S5. For each real defect area, calculate the defect severity coefficient, and group all real defect areas according to the defect severity coefficient. Based on the grouping results, generate operation and maintenance response instructions for the real defect areas under each group, and implement maintenance intervention for each real defect area under each group according to the operation and maintenance response instructions.

2. The boiler equipment defect detection method based on image processing according to claim 1, characterized in that, The four feature metrics include: multimodal difference, motion stability, shape irregularity, and background blending.

3. The method for detecting defects in boiler equipment based on image processing according to claim 2, characterized in that, The preliminary logical judgment is performed on each candidate defect region based on the four feature indicators to filter out suspected defect regions, including: S31. Set the first and second thresholds for multimodal difference, the upper threshold for motion stability, the lower threshold for shape irregularity, and the lower threshold for background blending. S32. Determine whether the shape irregularity of the current candidate defect region is greater than or equal to the lower limit threshold of the shape irregularity, and whether the background blending degree of the current candidate defect region is greater than or equal to the lower limit threshold of the background blending degree. S33. If the judgment result of S32 is yes, then further determine whether the current candidate defect region meets any of the following conditions: Condition 1: The multimodal difference of the current candidate defect region is greater than or equal to the first threshold of multimodal difference and less than the second threshold of multimodal difference, and the motion stability of the current candidate defect region is less than the upper limit threshold of motion stability; Condition 2: The multimodal variability of the current candidate defect region is greater than or equal to the second threshold of multimodal variability, and the shape irregularity of the current candidate defect region is greater than or equal to 1.5 times the lower limit threshold of shape irregularity; S34. If any condition in S33 is true, then mark the current candidate defect region as a suspected defect region. S35. If the judgment result of S32 is negative, or if the judgment results of both conditions in S33 are negative, then the current candidate defect region is removed.

4. The boiler equipment defect detection method based on image processing according to claim 3, characterized in that, For each suspected defect area, a comprehensive spatiotemporal confidence score is calculated, including: S411. Track the trajectory of each suspected defect area in N consecutive frames to form a time-series trajectory of each suspected defect area. S412. Based on the time series trajectory of each suspected defect area, count the number of frames in which each suspected defect area is identified as a suspected defect area within the time window, and calculate the ratio of the number of frames to the total number of frames in the time window as the local confidence ratio of each suspected defect area. S413. Based on the time series trajectory of each suspected defect area, calculate the linear regression slope of the area of ​​each suspected defect area as a function of time, and use it as the intensity of the diffusion trend of each suspected defect area. S414. Based on the time series trajectory of each suspected defect area, obtain the multimodal difference degree of each suspected defect area in each frame, and calculate the variance of the multimodal difference degree within the time window as the multimodal consistency index of each suspected defect area. S415. Set the first weight coefficient for the local confidence ratio, the second weight coefficient for the diffusion trend intensity, and the third weight coefficient for the multimodal consistency index. Then, multiply the local confidence ratio of each suspected defect region, the diffusion trend intensity of each suspected defect region, and the multimodal consistency index of each suspected defect region by the corresponding first weight coefficient, second weight coefficient, and third weight coefficient, and sum them to obtain the comprehensive spatiotemporal confidence score of each suspected defect region.

5. The boiler equipment defect detection method based on image processing according to claim 4, characterized in that, The comprehensive spatiotemporal confidence score is used to make a final judgment on each suspected defect region to obtain the true defect region, including: S421. Set a third and a fourth threshold for a comprehensive spatiotemporal confidence score, wherein the third threshold is less than the fourth threshold; S422. Determine whether the spatiotemporal confidence score of the current suspected defect area is greater than or equal to the fourth threshold. S423. If the spatiotemporal confidence score of the current suspected defect area is greater than or equal to the fourth threshold, then the current suspected defect area is determined to be a real defect area. S424. Determine whether the spatiotemporal confidence score of the current suspected defect area is less than or equal to the third threshold. S425. If the spatiotemporal confidence score of the current suspected defect region is less than or equal to the third threshold, then the current suspected defect region will be removed. S426. If the spatiotemporal confidence score of the current suspected defect region is greater than the third threshold and less than the fourth threshold, then further determine whether the following three conditions are met simultaneously: Condition 3: The local confidence ratio of the current suspected defect region is greater than or equal to the preset local confidence ratio threshold; Condition 4: The diffusion trend intensity of the current suspected defect region is greater than zero; Condition 5: The multimodal consistency index of the current suspected defect region is less than or equal to the preset multimodal consistency index threshold. S427. If all three conditions above are met, the current suspected defect area is determined to be a real defect area; otherwise, the current suspected defect area is removed.

6. The method for detecting defects in boiler equipment based on image processing according to claim 5, characterized in that, The calculation of the defect severity coefficient for each actual defect area includes: S511. Obtain the highest temperature value of each real defect area in the current frame of infrared thermal imaging image, and calculate the difference between the highest temperature value and the background temperature of normal boiler operation, as the thermal anomaly peak value of each real defect area. S512. Obtain the pixel area of ​​each real defect region in the time series trajectory for M consecutive frames, and calculate the slope of the linear fitting of the pixel area of ​​the M consecutive frames with time as the area growth rate of each real defect region. S513. Obtain the pixel area of ​​each real defect region in all frames of the time series trajectory, and take the average of the pixel areas of all frames as the spatial coverage of each real defect region. S514, set the first weighting coefficient for the peak value of thermal anomalies, the second weighting coefficient for the area growth rate, and the third weighting coefficient for the spatial coverage. S515. Multiply the peak thermal anomaly, area growth rate, and spatial coverage of each real defect region by the corresponding first weighting coefficient, second weighting coefficient, and third weighting coefficient, respectively, and sum the three multiplication results to obtain the defect severity coefficient of each real defect region.

7. The boiler equipment defect detection method based on image processing according to claim 6, characterized in that, The grouping of all real defect areas based on the defect severity coefficient includes: S521. Set the fifth and sixth thresholds for the defect severity coefficient, wherein the fifth threshold is less than the sixth threshold; S522. Determine whether the severity coefficient of the current real defect area is greater than or equal to the sixth threshold. S523. If the severity coefficient of the current real defect area is greater than or equal to the sixth threshold, then the current real defect area is classified into the first group. S524. Determine whether the defect severity coefficient of the current real defect area is less than the fifth threshold. S525. If the severity coefficient of the current real defect area is less than the fifth threshold, then the current real defect area is classified into the third group. S526. If the severity coefficient of the current real defect area is greater than or equal to the fifth threshold and less than the sixth threshold, then the current real defect area is classified into the second group.

Images