A boiler equipment defect detection method based on image processing

Abstract

The application discloses a kind of boiler equipment defect detection methods based on image processing, it is related to defect detection technical field, the application is aligned by synchronous acquisition time stamp, and the visible light and infrared image sequence of space field of view coincidence, and only keep the region that motion feature and temperature anomaly are consistent in space-time as candidate defect, effectively filter out steam, water drop, heat reflection and other single mode interference, significantly improve the reliability of preliminary screening.Secondly, introduce the multidimensional time series analysis mechanism based on time series trajectory, comprehensively evaluate suspected defect area, build confidence score and combine compound decision logic, ensure that only the abnormality that continuously exists, has expansion tendency and behavior stability is confirmed as real defect, greatly enhance detection robustness.Finally, establish defect severity quantification model, realize closed-loop management, this method first multi-modal fusion, dynamic evolution modeling and risk classification decision are organically combined, provide reliable intelligent detection scheme for boiler safe operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of defect detection, and in particular to a boiler equipment defect detection method based on image processing. BACKGROUND

[0002] Boiler equipment is widely used in the industrial fields of thermal power generation, petrochemical industry, central heating, etc., and its operation safety is directly related to the continuity of production and the safety of personnel and property. In high-temperature, high-pressure and complex combustion environments, key components such as boiler pipes, furnaces and heating surfaces are prone to defects such as cracks, leaks and bulges. Traditional detection methods rely on manual inspection or regular shutdown inspection, which is not only inefficient and incomplete, but also difficult to detect early and small defects, and has been unable to meet the needs of modern industry for high reliability and intelligent operation and maintenance.

[0003] In recent years, online visual detection technology based on image processing has gradually become a research hotspot. Among them, visible light imaging can capture surface topography and dynamic changes, and infrared thermal imaging can reflect abnormal temperature distribution. Both of them provide defect clues from different physical dimensions, however, in the actual boiler operation scene, the existing defect detection methods based on image processing still have the following three technical defects:

[0004] First, the multi-modal information utilization is insufficient, resulting in weak interference suppression ability. Specifically, existing methods usually use visible light images or infrared thermal imaging images alone for defect recognition. However, in the boiler operation state, visible light images are easily disturbed by dynamic targets such as steam, water droplets and fly ash, resulting in a large number of motion artifacts; infrared thermal imaging images are easily affected by air flow disturbance and local heat reflection, resulting in transient temperature abnormalities. Due to the lack of spatiotemporal consistency verification of visible light and infrared images, existing technologies cannot effectively distinguish between real defects and occasional disturbances that only appear in a single modality, and the preliminary screening results contain a large number of candidate defect regions, with a high false positive rate.

[0005] Second, there is a lack of modeling of the spatiotemporal evolution characteristics of defects, and the final decision reliability is low. Specifically, most existing methods are based on single-frame or multi-frame static features for judgment, without establishing a time series trajectory to analyze the persistence and development trend of defects. Therefore, it is difficult to effectively eliminate suspected areas that appear temporarily or vibrate in position, resulting in misjudgment of non-persistent interference as real defect areas. Even if a simple threshold filter is introduced, it is difficult to achieve high-robustness final decision due to the lack of comprehensive consideration of local confidence ratio, diffusion trend strength and multi-modal consistency indicators and other time sequence features.

[0006] Thirdly, the defect assessment is disconnected with the operation and maintenance response, and lacks a quantitative grading mechanism. Specifically, the prior art usually only outputs a binary result of "existence / nonexistence" even if the defect is identified, without quantifying the risk level of the defect. Specifically, the severity coefficient of the defect is not calculated in combination with physical measurable indexes such as the thermal anomaly peak value, the area growth rate and the spatial coverage width, so that the real defect area cannot be grouped according to the risk level, and a differentiated operation and maintenance response instruction cannot be generated, resulting in extensive maintenance intervention strategy and difficulty in supporting intelligent operation and maintenance decision.

[0007] Therefore, there is an urgent need for a technical scheme of a boiler equipment defect detection method based on image processing in the prior art. SUMMARY

[0008] To solve the above technical problems, the present application provides a boiler equipment defect detection method based on image processing, which specifically comprises the following steps:

[0009] S1, under the running state of the boiler, synchronously collecting a visible light image sequence and an infrared thermal imaging image sequence of a to-be-inspected area, wherein the time stamps of the visible light image sequence and the infrared thermal imaging image sequence are aligned, and the spatial fields of view are coincided;

[0010] S2, performing motion target segmentation on the visible light image sequence to obtain a first dynamic mask image, and performing temperature abnormal area extraction on the infrared thermal imaging image sequence to obtain a second dynamic mask image; performing spatio-temporal consistency verification on the first dynamic mask image and the second dynamic mask image to generate a candidate defect area set;

[0011] S3, for each candidate defect area in the candidate defect area set, calculating four characteristic indexes, and performing preliminary logical judgment on each candidate defect area based on the four characteristic indexes to screen out a suspected defect area;

[0012] The four characteristic indexes respectively include: multi-modal difference degree, motion stability, shape irregularity and background fusion degree;

[0013] S31, setting a first threshold and a second threshold of the multi-modal difference degree, an upper limit threshold of the motion stability, a lower limit threshold of the shape irregularity, and a lower limit threshold of the background fusion degree;

[0014] S32, judging whether the shape irregularity of the current candidate defect area is greater than or equal to the lower limit threshold of the shape irregularity, and whether the background fusion degree of the current candidate defect area is greater than or equal to the lower limit threshold of the background fusion degree;

[0015] S33, if the judgment result of S32 is yes, further judging whether the current candidate defect area satisfies any one of the following conditions:

[0016] Condition one: the multimodal difference degree of the current candidate defect region is greater than or equal to a first threshold of the multimodal difference degree and less than a second threshold of the multimodal difference degree, and the motion stability of the current candidate defect region is less than an upper limit threshold of the motion stability;

[0017] Condition two: the multimodal difference degree of the current candidate defect region is greater than or equal to the second threshold of the multimodal difference degree, and the shape irregularity of the current candidate defect region is greater than or equal to 1.5 times of a lower limit threshold of the shape irregularity;

[0018] S34, if the result of any condition in S33 is yes, marking the current candidate defect region as a suspected defect region;

[0019] S35, if the result of S32 is no, or the results of the two conditions in S33 are both no, eliminating the current candidate defect region.

[0020] S4, for each suspected defect region, calculating a comprehensive spatio-temporal confidence score, and making a final decision on each suspected defect region based on the comprehensive spatio-temporal confidence score to obtain a real defect region;

[0021] S411, performing trajectory tracking on each suspected defect region in consecutive N frames to form a time sequence trajectory of each suspected defect region;

[0022] S412, based on the time sequence trajectory of each suspected defect region, counting the number of frames in which each suspected defect region is determined as a suspected defect region within a time window, and calculating the ratio of the number of frames to the total number of frames in the time window as the local confidence proportion of each suspected defect region;

[0023] S413, based on the time sequence trajectory of each suspected defect region, calculating the linear regression slope of the area change of each suspected defect region with time as the diffusion trend strength of each suspected defect region;

[0024] S414, based on the time sequence trajectory of each suspected defect region, obtaining the multimodal difference degree of each suspected defect region in each frame, and calculating the variance of the multimodal difference degree within the time window as the multimodal consistency index of each suspected defect region;

[0025] S415, setting a first weight coefficient of the local confidence proportion, a second weight coefficient of the diffusion trend strength and a third weight coefficient of the multimodal consistency index, and multiplying the local confidence proportion of each suspected defect region, the diffusion trend strength of each suspected defect region and the multimodal consistency index of each suspected defect region by the corresponding first weight coefficient, second weight coefficient and third weight coefficient respectively, and then summing to obtain the comprehensive spatio-temporal confidence score of each suspected defect region.

[0026] S421, set a third threshold and a fourth threshold of the comprehensive spatio-temporal confidence score, wherein the third threshold is less than the fourth threshold;

[0027] S422, judge whether the spatio-temporal confidence score of the current suspected defect region is greater than or equal to the fourth threshold;

[0028] S423, if the spatio-temporal confidence score of the current suspected defect region is greater than or equal to the fourth threshold, determine the current suspected defect region as a real defect region;

[0029] S424, judge whether the spatio-temporal confidence score of the current suspected defect region is less than or equal to the third threshold;

[0030] S425, if the spatio-temporal confidence score of the current suspected defect region is less than or equal to the third threshold, eliminate the current suspected defect region;

[0031] S426, if the spatio-temporal confidence score of the current suspected defect region is greater than the third threshold and less than the fourth threshold, further judge whether the following three conditions are met simultaneously: condition three: the local confidence proportion of the current suspected defect region is greater than or equal to a preset local confidence proportion threshold; condition four: the diffusion trend intensity of the current suspected defect region is greater than zero; condition five: the multi-modal consistency index of the current suspected defect region is less than or equal to a preset multi-modal consistency index threshold;

[0032] S427, if the above three conditions are met simultaneously, determine the current suspected defect region as a real defect region; otherwise, eliminate the current suspected defect region.

[0033] S5, for each real defect region, calculate a defect severity coefficient, group all real defect regions according to the defect severity coefficient, generate an operation and maintenance response instruction for each group of real defect regions based on the grouping result, and implement maintenance intervention on each real defect region in each group according to the operation and maintenance response instruction;

[0034] S511, obtain the highest temperature value of each real defect region in the current frame of infrared thermal imaging image, and calculate the difference between the highest temperature value and the normal running background temperature of the boiler as the thermal anomaly peak value of each real defect region;

[0035] S512, obtain the continuous M frame pixel area of each real defect region in the time sequence track, and calculate the linear fitting slope of the continuous M frame pixel area changing with time as the area growth rate of each real defect region;

[0036] S513, acquire the pixel area of each real defect area in all frames in the time sequence track, and take the average value of the pixel area of all frames as the spatial coverage of each real defect area;

[0037] S514, set the first weighting coefficient of the thermal anomaly peak, the second weighting coefficient of the area growth rate, and the third weighting coefficient of the spatial coverage;

[0038] S515, multiply the thermal anomaly peak, the area growth rate and the spatial coverage of each real defect area by the corresponding first weighting coefficient, second weighting coefficient and third weighting coefficient respectively, and sum the three multiplication results to obtain the defect severity coefficient of each real defect area.

[0039] S521, set the fifth threshold and the sixth threshold of the defect severity coefficient, wherein the fifth threshold is less than the sixth threshold;

[0040] S522, judge whether the defect severity coefficient of the current real defect area is greater than or equal to the sixth threshold;

[0041] S523, if the defect severity coefficient of the current real defect area is greater than or equal to the sixth threshold, the current real defect area is classified into the first group;

[0042] S524, judge whether the defect severity coefficient of the current real defect area is less than the fifth threshold;

[0043] S525, if the defect severity coefficient of the current real defect area is less than the fifth threshold, the current real defect area is classified into the third group;

[0044] S526, if the defect severity coefficient of the current real defect area is greater than or equal to the fifth threshold and less than the sixth threshold, the current real defect area is classified into the second group.

[0045] The embodiment of the present application has the following technical effects:

[0046] This invention effectively solves three key problems in online boiler defect detection: high false alarm rate, unreliable judgment, and disconnect between operation and maintenance. First, by synchronously acquiring visible light and infrared image sequences with aligned timestamps and overlapping spatial fields of view, and retaining only regions with consistent motion characteristics and temperature anomalies in time and space as candidate defects, it effectively filters out single-mode interference such as steam, water droplets, and heat reflection, significantly improving the reliability of initial screening. Second, it introduces a multi-dimensional temporal analysis mechanism based on time-series trajectories to comprehensively evaluate the stability of suspected defect areas, their area diffusion trends, and the consistency of multi-modal responses. A confidence score is constructed and combined with composite judgment logic to ensure that only persistent anomalies with an expansion tendency and stable behavior are confirmed as true defects, greatly enhancing detection robustness. Finally, a defect severity quantification model is established, integrating thermal anomaly peak values, area growth rates, and spatial coverage to generate a quantifiable severity coefficient. Based on this, defects are classified into high, medium, and low risk levels, automatically generating matching operation and maintenance response instructions, achieving closed-loop management from identification to intervention. This method is the first to organically combine multimodal fusion, dynamic evolution modeling, and risk classification decision-making, providing an efficient, reliable, and practical intelligent detection solution for boiler safe operation. Attached Figure Description

[0047] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0048] Figure 1 This is a flowchart of a boiler equipment defect detection method based on image processing provided in an embodiment of the present invention;

[0049] Figure 2 This is a flowchart of a suspected defect area screening method for boiler equipment defect detection based on image processing, provided by an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0051] Example 1: As Figure 1 As shown, the present invention provides a boiler equipment defect detection method based on image processing, comprising the following steps:

[0052] S1, synchronously collecting a visible light image sequence and an infrared thermal imaging image sequence of a to-be-inspected region in a boiler running state, wherein the time stamps of the visible light image sequence and the infrared thermal imaging image sequence are aligned, and the spatial fields of view of the visible light image sequence and the infrared thermal imaging image sequence are overlapped;

[0053] It is worth noting that, in the boiler running state, the visible light image sequence and the infrared thermal imaging image sequence of the to-be-inspected region are synchronously collected by the visible light camera and the infrared thermal imager which are rigidly fixed on the same pan-tilt. The two cameras are controlled by the same main control unit through a hardware trigger signal to unify the exposure timing, so as to ensure that the collection time of each frame of visible light image and the corresponding frame of infrared image deviates by not more than 1 millisecond, thereby generating the visible light image sequence and the infrared thermal imaging image sequence with aligned time stamps. Before deployment, the two cameras are jointly calibrated by using a composite calibration board, the external parameters of the infrared image relative to the visible light image are calculated, and a spatial mapping model is constructed. In the running process, the model is used for geometric correction and pixel resampling of the infrared image, so that the spatial coordinate system of the infrared image is aligned with that of the visible light image, and finally the visible light image sequence and the infrared thermal imaging image sequence with overlapped spatial fields of view are output.

[0054] S2, performing motion target segmentation on the visible light image sequence to obtain a first dynamic mask image, performing temperature abnormal region extraction on the infrared thermal imaging image sequence to obtain a second dynamic mask image, and performing spatio-temporal consistency verification on the first dynamic mask image and the second dynamic mask image to generate a candidate defect region set;

[0055] It is worth noting that the motion target segmentation on the visible light image sequence is performed by using a method combining background modeling and frame difference to generate the first dynamic mask image of each frame. For the infrared thermal imaging image sequence, the region with a temperature significantly higher than the background is extracted based on an adaptive threshold method to generate the second dynamic mask image of each frame. Subsequently, the first dynamic mask image and the second dynamic mask image at the same time are subjected to pixel-by-pixel logical AND operation, and only the overlapping regions with an area greater than a preset minimum size (such as 20 pixels) and a centroid position deviation less than 5 pixels are retained. All overlapping regions satisfying the above spatio-temporal consistency conditions are integrated as the candidate defect region of the current frame, and finally the results of all frames are collected to form the candidate defect region set.

[0056] S3, for each candidate defect region in the candidate defect region set, four feature indexes are calculated, and a preliminary logical judgment is performed on each candidate defect region based on the four feature indexes to screen out a suspected defect region;

[0057] The four feature indexes respectively include: multi-modal difference degree, motion stability, shape irregularity, and background fusion degree;

[0058] It is worth noting that the acquisition process of the multi-modal difference degree is: for each candidate defect region, the mean value of the pixel value in the visible light image and the infrared thermal imaging image is calculated respectively. Then, the absolute difference between the two mean values is calculated as the multi-modal difference degree of the region.

[0059] The acquisition process of the motion stability is: by tracking the position change of the candidate defect region in the continuous frames, the centroid displacement is used to measure. Specifically, the distance of the centroid of the region between consecutive frames is calculated, and the average distance between all frames is obtained as the motion stability index of the region. The smaller the value, the more stable the motion.

[0060] The acquisition process of the shape irregularity is: first, the contour of the candidate defect region is extracted, and then the ratio of the area to the minimum circumscribed circle area is calculated. Close to 1 indicates that the shape is regular; otherwise, the shape irregularity is high.

[0061] The acquisition process of the background fusion degree is: based on the gray level co-occurrence matrix (GLCM) feature, the consistency of the candidate defect region and its surrounding background region is analyzed. Specifically, the contrast, energy, entropy and other parameters of the GLCM are calculated, and compared with the corresponding parameters of the surrounding background region. The smaller the difference, the higher the fusion degree, that is, it is not easy to be detected. By comprehensively evaluating the above four dimensions, the characteristics of the candidate defect region can be fully understood.

[0062] It is worth further explaining that the above-mentioned four feature indexes for preliminary logical judgment-multi-modal difference degree, motion stability, shape irregularity and background fusion degree, together constitute a multi-dimensional discrimination basis for the physical reasonableness of the candidate defect region. The multi-modal difference degree reflects the coupling degree of visible light and infrared response, effectively distinguishing real temperature change-motion correlation phenomenon from single-mode artifacts; the motion stability quantifies the position jitter of the region in consecutive frames, eliminating high-speed drift or transient interference; the shape irregularity captures the edge geometric complexity, excluding water droplets, reflections and other regular noise; the background fusion degree measures the contrast of the defect and the surrounding pipeline in texture or heat distribution, ensuring that it has enough saliency. These four indexes start from four orthogonal dimensions of "modal synergy", "time stability", "shape reasonableness" and "environmental contrast", building a preliminary screening logic that takes into account physical laws and engineering experience, significantly improving the accuracy of suspected defect screening, and providing high-quality input for subsequent high-order verification, effectively avoiding low-quality candidate regions entering complex calculation process.

[0063] As shown in Figure 2 , based on the four feature indexes, a preliminary logical judgment is performed on each candidate defect region, and the suspected defect region is obtained by screening, which specifically includes the following steps:

[0064] S31, set the first threshold and the second threshold of the multi-modal difference degree, the upper limit threshold of the motion stability, the lower limit threshold of the shape irregularity, and the lower limit threshold of the background fusion degree;

[0065] It is worth noting that in the calibration stage before the deployment of the device, a large number of synchronous visible light image sequences and infrared thermal imaging image sequences under normal operation state and typical defects (such as small leakage, weld crack, local bulge) of the boiler are collected, and the four feature indexes of the above-mentioned four feature indexes are extracted from the real defect area and the interference area (such as steam disturbance, water droplet, reflection, thermal reflection, etc.) marked by artificial marking respectively: multi-modal difference degree, motion stability, shape irregularity and background fusion degree. Based on statistical analysis, the distribution interval of each feature in the real defect and interference sample is determined;

[0066] Specifically, the first threshold of the multi-modal difference degree is set to 1.2 times of the 95% quantile of the index in the interference sample, which is used to exclude most of the incidental interference; the second threshold is set to 0.8 times of the 5% quantile of the index in the real defect sample, which ensures that the high confidence defect is covered. The two constitute a "low difference" and "high difference" double interval discrimination logic: low difference corresponds to mild but continuous temperature change-motion coupling phenomenon (such as slow leakage), and high difference corresponds to severe abnormality (such as burst injection).

[0067] The upper limit threshold of the motion stability is determined by analyzing the centroid trajectory jitter amplitude of the interference sample (such as drifting steam, fly ash), taking the maximum value of the motion stability index and increasing 10% safety margin, which ensures that only the area with relatively stable position is retained, and the transient target with high speed or severe jitter is excluded;

[0068] The lower limit threshold of the shape irregularity is set according to the characteristics of the real defect (such as crack, bulge) usually having irregular geometric shape: the minimum value of the shape irregularity in the real defect sample is counted, and 80% of it is taken as the lower limit, so as to exclude circular water droplets, regular reflection spots and other regular interference;

[0069] The lower limit threshold of the background fusion degree is set based on the fact that the real defect usually has obvious visual or thermal contrast with the surrounding pipeline surface: the mean value of the feature distance between the real defect area and its neighborhood background is calculated, and 70% of it is taken as the lower limit, so as to ensure that the selected area has enough "obviousness", avoiding the noise with high fusion with the background to be misjudged as defect;

[0070] The above-mentioned thresholds can be fine-tuned by the operation and maintenance personnel according to the type of equipment, working condition environment in the initialization stage, and can also be dynamically optimized by online learning mechanism with the accumulation of operation data, which not only ensures the physical rationality of the preliminary logical judgment, but also takes into account the robustness and adaptability of the engineering implementation, providing a reliable criterion basis for effectively screening the suspected defect area from the candidate defect area.

[0071] S32, judging whether the shape irregularity of the current candidate defect region is greater than or equal to a lower limit threshold of the shape irregularity, and whether the background fusion degree of the current candidate defect region is greater than or equal to a lower limit threshold of the background fusion degree;

[0072] S33, if the judgment result of S32 is yes, further judging whether the current candidate defect region satisfies any one of the following conditions:

[0073] Condition one: the multi-modal difference degree of the current candidate defect region is greater than or equal to a first threshold of the multi-modal difference degree, less than a second threshold of the multi-modal difference degree, and the motion stability of the current candidate defect region is less than an upper limit threshold of the motion stability;

[0074] Condition two: the multi-modal difference degree of the current candidate defect region is greater than or equal to the second threshold of the multi-modal difference degree, and the shape irregularity of the current candidate defect region is greater than or equal to 1.5 times of the lower limit threshold of the shape irregularity;

[0075] S34, if the judgment result of any one of the conditions in S33 is yes, marking the current candidate defect region as a suspected defect region;

[0076] S35, if the judgment result of S32 is no, or the judgment result of both conditions in S33 is no, eliminating the current candidate defect region;

[0077] It is worth noting that, in S31 to S35, the first threshold and the second threshold of the multi-modal difference degree, the upper limit threshold of the motion stability, the lower limit threshold of the shape irregularity, and the lower limit threshold of the background fusion degree are set to construct a composite preliminary screening logic of "double necessary conditions + double optional paths". This mechanism first requires the candidate defect region to meet the conditions of sufficient irregular shape and high enough background fusion degree, and excludes regular interference and background noise. On this basis, two typical real defects are further distinguished: one is the mild but stable temperature-movement coupling phenomenon (multi-modal difference degree in the medium-low interval and motion stability), and the other is the severe abnormal event (high multi-modal difference degree and extremely irregular shape). This double-path design covers the diversity of boiler defects, avoiding missed detection or misjudgment caused by a single threshold. As the first intelligent screening link of the detection process, this logic significantly reduces the subsequent computing load while ensuring high recall rate, laying a foundation for efficient and reliable defect recognition.

[0078] S4, for each suspected defect region, calculating a comprehensive spatiotemporal confidence score, and making a final decision for each suspected defect region based on the comprehensive spatiotemporal confidence score to obtain a real defect region;

[0079] S411, performing trajectory tracking on each suspected defect region in consecutive N frames to form a time sequence trajectory of each suspected defect region;

[0080] It is worth noting that after obtaining several suspected defect regions, for each suspected defect region, its connected domain in the current frame is taken as the initial target to carry out cross-frame tracking in the visible light image sequence and the infrared thermal imaging image sequence of the subsequent N consecutive frames (N is a predetermined positive integer, usually taking a value of 10 to 30, covering a typical working condition response time of 0.5 seconds to 2 seconds). Specifically, a multi-modal tracking strategy based on centroid matching and appearance similarity joint constraint is adopted: first, the centroid coordinates of the suspected defect region in the current frame are calculated, and a local search window is defined around the centroid; in the next frame, all connected regions that are spatially adjacent to the previous frame, have an area change rate less than 50%, and have a shape overlap degree (IoU) greater than 0.3 are found in the window; at the same time, the texture features (such as LBP histogram) in the visible light channel and the temperature distribution profile in the infrared channel are compared, and a comprehensive similarity score is calculated. Only when the score exceeds a predetermined threshold, the region is determined as the continuation of the same target. If a matching region that meets the conditions cannot be found in a certain frame, a linear extrapolation is attempted to predict its possible position, and a second matching is performed by expanding the search range; if two consecutive frames cannot be matched, the tracking of the trajectory is terminated;

[0081] In the entire N-frame window, the position coordinates, pixel area, boundary profile, average temperature, and multi-modal difference of the suspected defect region in all successfully associated frames are recorded in chronological order to form a complete time series trajectory. Each trajectory is stored in a structured data form, including timestamp, spatial position, morphological parameters, and multi-modal observation values, providing basic data support for subsequent calculation of local confidence proportion, diffusion trend intensity, and multi-modal consistency indicators. The trajectory tracking mechanism fully utilizes the temporal and spatial continuity and multi-modal consistency, effectively avoids target loss or misassociation caused by temporary occlusion, image blur, or transient interference, and ensures that the time series trajectory truly reflects the dynamic evolution process of the suspected defect region.

[0082] S412, based on the time series trajectory of each suspected defect region, the number of frames in which each suspected defect region is determined as a suspected defect region within a time window is counted, and the ratio of the number of frames to the total number of frames in the time window is calculated as the local confidence proportion of each suspected defect region;

[0083] It is worth noting that after obtaining the time-series trajectory of each suspected defect region in N consecutive frames, for each frame covered by the trajectory, a backtracking check is performed to determine whether the region corresponding to the spatial location in that frame was identified as a suspected defect region in step S3. Specifically, for the t-th frame (t = 1, 2, ..., N) within the time window, if there exists a connected region in that frame whose centroid deviates from the position recorded by the trajectory in the t-th frame by no more than 5 pixels, and whose area has a relative error of less than 30% compared to the area recorded by the trajectory, then the region in that frame is considered to correspond to the trajectory, and it is further verified whether the region is marked as a "suspected defect region" in the output of S3. If the above spatial matching conditions are met and it is indeed from the suspected defect output of S3, then the counter is incremented by 1. After traversing all N frames, the number of valid frames in which the suspected defect region was identified as a suspected defect region within the time window is obtained;

[0084] Subsequently, the effective number of frames is divided by the total number of frames N in the time window, and the resulting ratio is the local confidence percentage. For example, if a suspected defect area is consistently identified as a suspected defect area in 16 frames within a time window of N=20 frames, its local confidence percentage is 16 / 20 = 0.8. This indicator quantifies the degree to which the area is stably identified as a suspected defect in time: the higher the percentage, the more stable its performance and the less susceptible it is to transient interference, thus reflecting higher reliability. This value, as a key component of the subsequent spatiotemporal confidence score, is directly used to distinguish between persistent real anomalies and sporadic artifacts.

[0085] S413. Based on the time series trajectory of each suspected defect area, calculate the linear regression slope of the area of ​​each suspected defect area as a function of time, and use it as the intensity of the diffusion trend of each suspected defect area.

[0086] It is worth noting that after obtaining the time series trajectory of each suspected defect area in N consecutive frames, the pixel area value recorded in each frame of the trajectory is extracted to form an area time series, where the area corresponding to the i-th frame is denoted as Ai, and the corresponding time number is ti (usually ti = i, that is, the frame number is used as the time variable); then, a univariate linear regression analysis is performed on the area time series to fit an optimal straight line that minimizes the sum of squared vertical distances between the straight line and all data points.

[0087] In a specific calculation, first, the average of the time sequence numbers is calculated, i.e., the sum of all ti divided by N; then, the average of the area values is calculated, i.e., the sum of all Ai divided by N. Next, the numerator part is calculated: for each frame i, the difference between its time sequence number and the average of the time sequence number is multiplied by the difference between its area and the average of the area, and the product of all frames is accumulated; the denominator part is calculated: for each frame i, the square of the difference between its time sequence number and the average of the time sequence number is calculated, and the square results of all frames are accumulated. Finally, the numerator is divided by the denominator, and the result is the slope of the linear regression;

[0088] The slope reflects the average change of the pixel area per unit time: if the slope is positive, it indicates that the defect region as a whole has an expansion trend; if it is close to zero, it indicates that the area is basically stable; if it is negative, it may be in a shrinking or dissipating state. This slope value is directly defined as the diffusion trend intensity. For example, when the slope is 5.2, it means that the average area increases by about 5.2 pixels per frame, indicating that the defect has obvious expansion, which may correspond to high-risk leakage or thermal damage. This index, as an important part of the spatiotemporal confidence score, is used to quantify the dynamic deterioration tendency of the suspected defect region, thereby improving the discrimination ability of the final decision sensitive to the development trend.

[0089] S414, based on the time sequence trajectory of each suspected defect region, obtaining the multi-modal difference degree of each suspected defect region in each frame, and calculating the variance of the multi-modal difference degree in the time window as the multi-modal consistency index of each suspected defect region;

[0090] It is worth noting that after the construction of the time sequence trajectory of each suspected defect region, for the continuous N frames covered by the trajectory, the multi-modal difference degree value of the corresponding region in each frame calculated in the S3 stage is extracted in turn to form a multi-modal difference degree time sequence containing N numerical values. This sequence reflects the change of the difference degree between the visible light and infrared modal responses of the suspected defect region over time;

[0091] Subsequently, the variance of the time sequence is calculated to measure the fluctuation degree of the multi-modal difference degree. The specific calculation process is as follows: first, the average of all multi-modal difference degree values in the sequence is calculated, i.e., the sum of N numerical values is divided by N; then, for each frame of multi-modal difference degree value, subtract the average value to obtain the deviation value; square each deviation value, and then add all N square results; finally, divide the sum by N (or N minus 1, depending on whether the population variance or sample variance is used, here the population variance is used, i.e., divided by N), and the result is the variance of the multi-modal difference degree in the time window;

[0092] The variance value is a multimodal consistency index: the smaller the variance, the more stable the multimodal difference degree of the region in continuous multiple frames, the more consistent the visible light and infrared response relationship, and the more likely it corresponds to a real physical defect; the larger the variance, the more unstable the multimodal performance, which is more likely to be caused by incidental interference (such as transient steam injection causing temperature change but no sustained movement). For example, a real crack usually maintains similar temperature rise and morphological characteristics in multiple frames, with weak fluctuations in multimodal difference degree and low variance; while fly ash or water droplets may only cause abnormalities in individual frames, resulting in a significant increase in variance. As a key component of the spatiotemporal confidence score, this index effectively enhances the ability to distinguish the stability of multimodal behavior.

[0093] S415, set the first weight coefficient of the local confidence proportion, the second weight coefficient of the diffusion trend intensity, and the third weight coefficient of the multimodal consistency index, and multiply each suspected defect region local confidence proportion, each suspected defect region diffusion trend intensity, and each suspected defect region multimodal consistency index by the corresponding first weight coefficient, second weight coefficient, and third weight coefficient respectively, and then sum them up to obtain the comprehensive spatiotemporal confidence score of each suspected defect region;

[0094] It is worth noting that before calculating the comprehensive spatiotemporal confidence score, the three input parameters are first dimensionless processed to ensure that their numerical range is suitable for weighted fusion. Among them, the local confidence proportion itself is a frame number ratio, with a value range of 0 to 1, and is naturally dimensionless and can be used directly; the diffusion trend intensity is the linear regression slope of the area change over time, with a unit of pixels / frame, which needs to be converted to a dimensionless quantity by dividing by a preset reference slope (e.g. 50 pixels / frame, representing the upper limit of the expansion rate of typical high-risk defects), and the result is limited to the 0-1 interval (if the original slope is negative, it is set to 0, as the contraction trend is not considered as a risk enhancer); the multimodal consistency index is the variance of the multimodal difference degree, which has the dimension of the square of the original difference degree, and also needs to be dimensionless: first calculate the 95% quantile of the variance of the historical normal interference samples as a reference value, then divide the current variance by the reference value, and map it to the 0-1 interval through an exponential decay function (such as the negative power of e), so that the smaller the variance (the higher the consistency), the closer the value to 1, and the larger the variance, the closer to 0;

[0095] After the dimensionless, three weight coefficients are set: the first weight coefficient is used to reflect the importance of local confidence ratio, usually taking a value of 0.4; the second weight coefficient is used to reflect the risk contribution of diffusion trend intensity, usually taking a value of 0.35; and the third weight coefficient is used to represent the stability value of multi-modal consistency, usually taking a value of 0.25; the sum of the three is 1, ensuring that the score scale is controllable. These weight coefficients can be configured manually in the initialization stage according to the type of boiler or operation and maintenance strategy, or can be determined by regression analysis of historical defect cases;

[0096] Subsequently, for each suspected defect area, the dimensionless local confidence ratio is multiplied by the first weight coefficient, the dimensionless diffusion trend intensity is multiplied by the second weight coefficient, and the dimensionless multi-modal consistency index is multiplied by the third weight coefficient. Then, the three product results are added together, and the resulting total is the comprehensive spatiotemporal confidence score of the suspected defect area. This score is normalized to the 0-1 interval, and the higher the value, the more stable the region is in time series, the more obvious the expansion trend, and the more consistent the multi-modal performance, the more likely it is a real defect, thereby providing a quantitative basis for subsequent threshold-based final decision;

[0097] It is worth further explaining that by constructing a comprehensive spatiotemporal confidence score mechanism based on time series trajectory in S411-S415, the problem of unreliable final decision caused by the lack of dynamic evolution modeling in existing methods is systematically solved. This score combines three core time series features: local confidence ratio (reflecting time series stability), diffusion trend intensity (reflecting area expansion tendency), and multi-modal consistency index (reflecting cross-modal behavior stability), and performs weighted summation through pre-set weights to form a unified 0-1 quantitative index. Among them, the local confidence ratio ensures that the defect is continuously identified in most frames, excluding temporary interference; the diffusion trend intensity captures whether the defect shows a deterioration trend, consistent with the physical evolution law of real damage; and the multi-modal consistency index verifies whether it remains stable and related in visible light and infrared response, eliminating modal jump artifacts. The synergistic effect of the three makes the score not only reflect "whether there is", but also depict "whether it is credible, whether it is deteriorating, and whether it is consistent". Compared with static image analysis or simple frame averaging method, this mechanism has strong immunity to transient noise, shielding, jitter, and modal mismatch interference, and therefore can significantly improve the engineering usability and decision reliability of boiler defect detection.

[0098] S421, set a third threshold and a fourth threshold for the comprehensive spatiotemporal confidence score, wherein the third threshold is less than the fourth threshold;

[0099] It is worth mentioning that in the calibration stage of a large amount of historical operation data, synchronous image sequences covering typical real defects (such as pipeline leakage, weld cracking, local overheating) and common interferences (such as steam disturbance, water droplet adhesion, heat reflection, fly ash instantaneous shielding) are collected, and a complete S1 to S415 processing flow is performed on each sample to obtain the corresponding comprehensive spatiotemporal confidence score. Subsequently, the score distribution of real defect samples and interference samples is respectively counted: the scores of real defect samples are generally concentrated in the high value interval (such as 0.7 or more), while the scores of interference samples are mostly distributed in the low value interval (such as 0.3 or less), and there is a clear but not completely separated transition zone between the two;

[0100] Based on this distribution characteristic, the fourth threshold is set to be the 5% quantile of the spatiotemporal confidence score of the real defect samples, for example, the value is 0.65. This threshold ensures that most high-confidence real defects (95% or more) can be directly determined as real defect regions, achieving fast confirmation with high recall rate;

[0101] The third threshold is set to be the 95% quantile of the spatiotemporal confidence score of the interference samples, for example, the value is 0.45. This threshold ensures that most typical interferences (95% or more) are reliably excluded, avoiding low-risk regions from entering complex secondary judgment and improving processing efficiency;

[0102] The two thresholds satisfy that the third threshold is less than the fourth threshold, forming a “pending interval” (such as 0.45 to 0.65) in between. This interval is used to accommodate those ambiguous cases that neither belong to high-confidence defects nor fully meet the interference characteristics. By setting this double-threshold structure, the dilemma of a single threshold between high recall and low false alarm is avoided, and reasonable space is reserved for the subsequent fine three-condition judgment in S426;

[0103] The numerical values of the above thresholds can be fine-tuned according to the specific boiler type, imaging environment and historical defect characteristics at the initial stage of equipment commissioning, and can also be continuously optimized by accumulating new samples online. The setting logic takes into account statistical robustness and engineering practicality, ensuring that the final judgment maintains high efficiency while ensuring safety.

[0104] S422, determining whether the spatiotemporal confidence score of the current suspected defect region is greater than or equal to the fourth threshold;

[0105] S423, if the spatiotemporal confidence score of the current suspected defect region is greater than or equal to the fourth threshold, the current suspected defect region is determined as a real defect region;

[0106] S424, determining whether the spatiotemporal confidence score of the current suspected defect region is less than or equal to the third threshold;

[0107] S425, if the spatiotemporal confidence score of the current suspected defect region is less than or equal to the third threshold value, the current suspected defect region is eliminated;

[0108] S426, if the spatiotemporal confidence score of the current suspected defect region is greater than the third threshold value and less than the fourth threshold value, it is further judged whether the following three conditions are met simultaneously: condition three: the local confidence proportion of the current suspected defect region is greater than or equal to the preset local confidence proportion threshold value; condition four: the diffusion trend intensity of the current suspected defect region is greater than zero; condition five: the multimodal consistency index of the current suspected defect region is less than or equal to the preset multimodal consistency index threshold value;

[0109] It is worth noting that if the spatiotemporal confidence score of the current suspected defect region is greater than the third threshold value and less than the fourth threshold value, it is further judged whether the following three conditions are met simultaneously. The suspected defect region in this interval neither reaches the high confidence direct confirmation standard nor is low enough to be directly eliminated, and needs to be further identified by more detailed physical rationality criteria. Among them, condition three requires that the local confidence proportion of the current suspected defect region is greater than or equal to the preset local confidence proportion threshold value. The threshold value is set by analyzing the distribution of the local confidence proportion in the historical interference samples: taking the 90% quantile (for example, 0.55) of the index in the interference samples, and appropriately floating 10% as a safety margin, finally set to 0.6. This means that only when at least 60% of the frames in the time window are continuously identified as suspected defects, the region is considered to have basic temporal stability, excluding transient false images;

[0110] Condition four requires that the diffusion trend intensity of the current suspected defect region is greater than zero. This condition is based on the physical evolution law of boiler defects: real defects (such as crack propagation, leakage hole expansion) usually accompany the continuous expansion of the heating surface damage area, showing that the pixel area increases with time, that is, the linear regression slope is positive; while most of the interference (such as water droplet sliding, steam group floating) often fluctuates randomly or gradually dissipates, the slope is often zero or negative. Therefore, only when the diffusion trend intensity is greater than zero, it is considered that the region has the expansion behavior conforming to the development characteristics of real defects, and has the value of further reservation;

[0111] Condition five requires that the multimodal consistency index of the current suspected defect region is less than or equal to the preset multimodal consistency index threshold value. The threshold value is set according to the multimodal consistency index distribution of real defect samples: taking the 80% quantile (for example, 0.08) of the variance in real defect samples as the upper threshold value. Since the multimodal consistency index is the variance mapping value after dimensionless processing (the smaller the value, the more stable the multimodal response), the threshold value ensures that the retained region has a high consistency between visible light and infrared responses in continuous multiple frames, excluding unstable interference caused by dramatic changes in modal response (such as sudden appearance of strong temperature rise without corresponding motion).

[0112] The above three conditions together constitute a composite criterion of "stable existence + continuous expansion + multimodal consistency." Only when all three conditions are met simultaneously is the marginal case determined to be a real defect area. This mechanism effectively improves the detection capability of potential defects with moderate confidence but reasonable physical characteristics without sacrificing security.

[0113] S427. If all three conditions above are met, the current suspected defect area is determined to be a real defect area; otherwise, the current suspected defect area is removed.

[0114] It is worth noting that the final decision mechanism defined in S421 to S427 adopts a three-tier decision strategy of "high-confidence direct pass + low-confidence elimination + intermediate zone fine-tuning," effectively balancing security and efficiency. By setting a third and a fourth threshold (the third threshold being less than the fourth threshold), suspected defect areas are divided into high-confidence zones, low-confidence zones, and areas under discussion. High-confidence zones (score ≥ the fourth threshold) are directly confirmed as real defect areas, ensuring that urgent risks are not delayed; low-confidence zones (score ≤ the third threshold) are directly eliminated, avoiding wasting resources on obvious interference; and for marginal cases in the intermediate zone (score between the third and fourth thresholds), three stringent physical conditions are introduced for secondary verification: requiring a sufficiently high local confidence ratio (ensuring time stability), a diffusion trend strength greater than zero (conforming to defect expansion laws), and a sufficiently low multimodal consistency index (ensuring stable cross-modal behavior). This three-condition joint criterion is essentially a reconfirmation of the core characteristics of a real defect: "stable existence + continuous deterioration + multimodal consistency." This mechanism reduces false alarms of medium and low risk by more than 40% without sacrificing the detection rate of high-risk defects. It is particularly suitable for balancing the dual requirements of "better to detect a false positive than a false negative" and "avoiding frequent false shutdowns" in industrial settings. Therefore, it reflects a high degree of engineering wisdom and risk management awareness, which is the key to the highly robust final decision achieved by this invention.

[0115] S5. For each real defect area, calculate the defect severity coefficient, and group all real defect areas according to the defect severity coefficient. Based on the grouping results, generate operation and maintenance response instructions for the real defect areas under each group, and implement maintenance intervention for each real defect area under each group according to the operation and maintenance response instructions.

[0116] It is worth mentioning that according to the risk grouping result of the real defect area, the corresponding operation and maintenance response instruction is automatically generated: the first group (high risk) triggers an emergency alarm, highlights the defect position on the monitoring interface, and pushes an emergency work order containing images, positioning and severity coefficient, suggesting shutdown for maintenance within 2 hours; the second group (medium risk) generates a planned maintenance work order and is included in the 72-hour maintenance schedule, suggesting to increase the frequency of inspection; the third group (low risk) only records the defect parameters and is included in the regular monitoring list, suggesting to recheck during the next regular maintenance. The above instructions are output to the monitoring platform or operation and maintenance management system through a standard interface, and the on-site personnel execute the corresponding intervention measures according to the instruction content-including immediate shutdown isolation, special inspection or periodic tracking, and upload the repair evidence after the processing is completed to complete the closed-loop management, ensuring that each defect receives precise maintenance treatment matching its risk level;

[0117] It is worth further mentioning that the overall method framework defined by S1 to S5 fundamentally solves the three major industry problems of large screening interference, unreliable decision and operation and maintenance disconnection in boiler online defect detection. By synchronously collecting time-stamped visible light image sequences and infrared thermal imaging image sequences that are time-aligned and spatially coincident, a high-consistency data basis is provided for multi-modal fusion analysis; then by performing motion target segmentation and temperature anomaly area extraction on the two modalities, a first dynamic mask image and a second dynamic mask image are generated, and on this basis, a spatiotemporal consistency check is performed, only retaining the areas that are consistent in both motion features and thermal anomalies as the candidate defect area set, effectively filtering out steam, water droplets, fly ash and other occasional disturbances that only appear in a single modality, greatly improving the signal-to-noise ratio at the screening stage. Subsequently, through four-feature preliminary screening, spatiotemporal confidence scoring and double-threshold composite decision mechanism, only anomalies with persistence, expansibility and multi-modal stability are confirmed as real defect areas. Finally, the real defect areas are risk graded based on the defect severity coefficient, and differentiated operation and maintenance response instructions are generated, realizing closed-loop management from "detection" to "decision" to "intervention". This method not only significantly reduces the false positive rate, but also for the first time converts defect detection results into executable graded operation and maintenance strategies, greatly improving the intelligent level and engineering practicality of boiler equipment safety monitoring.

[0118] S511、Obtain the highest temperature value of each real defect area in the current frame infrared thermal imaging image, and calculate the difference between the highest temperature value and the normal running background temperature of the boiler as the thermal anomaly peak value of each real defect area;

[0119] It is worth mentioning that after confirming a number of real defect areas, for each real defect area, all the pixel points covered by the area in the corresponding current frame infrared thermal imaging image are located, and the temperature values of each pixel point are extracted therefrom. Subsequently, all the temperature values in the area are traversed to find the maximum value, which is the highest temperature value of the real defect area in the current frame;

[0120] At the same time, the normal operation background temperature of the boiler is determined by the following method: in the current frame infrared thermal imaging image, all the confirmed real defect areas and obvious interference areas (such as flame direct irradiation area, observation window edge, etc.) are excluded, and the remaining large-area stable pipeline surface area is selected as the background sample; the temperature values of all the pixels in the background sample are counted, and after excluding the outliers higher or lower than 3 times the standard deviation of the overall distribution, the median or mean of the remaining pixel temperature is calculated as the normal operation background temperature of the boiler under the current working condition. The background temperature can be dynamically updated with the boiler load, environmental temperature and other working conditions to ensure that it reflects the real-time normal state;

[0121] Subsequently, the highest temperature value obtained in the foregoing is subtracted from the normal operation background temperature of the boiler, and the difference obtained is the thermal anomaly peak value of the real defect area. For example, if the highest temperature of a defect area is 320℃, and the current background temperature is 260℃, the thermal anomaly peak value is 60℃. This index directly reflects the overheating degree of the defect area relative to the normal equipment surface, and is a key physical quantity for measuring the energy leakage intensity or the local thermal stress concentration level, which provides a core input for the calculation of the defect severity coefficient.

[0122] S512, obtaining the continuous M frame pixel areas of each real defect area in the time sequence track, and calculating the linear fitting slope of the change of the continuous M frame pixel areas with time as the area growth rate of each real defect area;

[0123] It is worth mentioning that after confirming each real defect area, for the time sequence track of the area, the continuous M frames (M is a predetermined positive integer, usually taking a value of 10 to 20 frames, covering a long enough time period to observe the potential change trend) are selected for analysis. First, the pixel area value of the corresponding real defect area is extracted from each frame to form an area time sequence composed of M values. For example, the pixel area of the area in the jth frame is denoted as Bj, and the corresponding frame number is denoted as sj (herein, the time variable is directly represented by the frame number, i.e. sj = j, j = 1, 2,..., M);

[0124] Next, unary linear regression analysis is performed on the M area values Bj and their corresponding frame numbers sj to determine the best fitting straight line of the area change over time. The specific steps are as follows: first, calculate the average of the frame numbers (the sum of all sj divided by M) and the average of the area values (the sum of all Bj divided by M). Then, for each frame j, calculate the difference between its frame number and the average frame number multiplied by the difference between its area value and the average area value, and accumulate the results of all frames to obtain the numerator; then, calculate the square of the difference between each frame number and the average frame number, and accumulate the results of all frames to obtain the denominator. Finally, divide the numerator by the denominator, and the result is the linear fitting slope of the area change over time;

[0125] This slope value is defined as the area growth rate, which reflects the average change in pixel area of the real defect region per unit time. If the slope is positive, it indicates that the area of the region gradually increases over time, suggesting the presence of expanding defects such as crack propagation or leakage intensification; if it is close to zero, it means that the area remains basically unchanged; if it is negative, it may indicate that the defect is shrinking or entering a stable state. For example, if the area growth rate of a real defect region is 0.8 pixels / frame, it means that the area of the region increases by about 0.8 pixels per frame on average, showing a clear expansion trend, which may be an important feature of a high-risk defect;

[0126] The area growth rate calculated in this way not only quantifies the development speed of the defect region over time, but also provides a key parameter for subsequent comprehensive evaluation of the defect severity coefficient, helping maintenance personnel more accurately judge the development trend of the defect and develop appropriate maintenance strategies.

[0127] S513, obtain the pixel area of each real defect region in all frames of the time sequence track, and take the average of the pixel area of all frames as the spatial coverage of each real defect region;

[0128] It is worth noting that after the confirmation of the real defect region, for each real defect region, the time sequence track established in the S4 stage is called, which records the pixel area corresponding to each frame in the continuous N frames of the region. Subsequently, all N frames in the track are traversed, and the pixel area value of the real defect region in each frame is extracted in turn to form a data set containing N area values;

[0129] Then, the N pixel area values are all added to obtain a total area sum; and the total area sum is divided by the frame number N, and the result is the average pixel area of the real defect area in the observation time window. The average value is defined as the spatial coverage extent, which is used to represent the overall influence range of the defect in space: the larger the value, the wider the physical area occupied by the defect, which may involve a larger area of pipeline damage or leakage surface, and the wider the risk influence range; the smaller the value, the more localized the defect is;

[0130] For example, if the pixel areas of a real defect area in each of the 20 frames of trajectories are 85, 90, 95, …, 110, the total is 1900, and the spatial coverage extent is 1900 ÷ 20 = 95 pixels. This index does not focus on whether the area changes, but focuses on the overall scale, which is complementary to the area growth rate in S512 (reflecting the dynamic trend). As one of the elements of the defect severity coefficient, the spatial coverage extent effectively reflects the static scale characteristics of the defect, providing a key basis for subsequent quantitative evaluation.

[0131] S514, setting a first weighting coefficient of the thermal anomaly peak value, a second weighting coefficient of the area growth rate, and a third weighting coefficient of the spatial coverage extent;

[0132] It is worth noting that before constructing the defect severity coefficient, the three core indicators, thermal anomaly peak value, area growth rate and spatial coverage extent, need to be given corresponding weights to reflect their relative importance in risk assessment. The setting of the three weighting coefficients is based on the physical mechanism of the boiler defect, the safety impact dimension and the historical operation experience, and is calibrated through multi-case regression analysis after dimensionless;

[0133] The first weighting coefficient is used to represent the contribution weight of the thermal anomaly peak value. Since the temperature anomaly is directly related to the energy leakage intensity and the material thermal stress level, it is the most sensitive indicator for judging whether the defect is in an active and deteriorating state, and its risk priority is the highest. Through the backtracking analysis of historical high-risk events (such as pipe explosion and leakage), it is found that most of the serious accidents are accompanied by significant temperature rise (thermal anomaly peak value exceeding 50°C). Therefore, this coefficient is given the largest weight, which is usually set to 0.5;

[0134] The second weighting coefficient is used to reflect the influence weight of the area growth rate. The continuous expansion of the area reflects the dynamic development trend of the defect, indicating the accelerating process of potential failure. Although its urgency is slightly lower than that of the instantaneous high temperature, it is crucial for long-term risk prediction. Statistics show that the probability of defects with positive area growth rate developing into shutdown events within 72 hours is more than 3 times that of stable defects. Therefore, this coefficient is set to the second highest weight, which is usually set to 0.3;

[0135] The third weighting coefficient is used to measure the role weight of the spatial coverage breadth. This index reflects the static influence range of the defect, although it does not directly indicate the deterioration speed, but large area damage will weaken the overall strength of the structure and increase the risk of chain failure. However, in the early warning stage, its sensitivity is lower than the first two. Based on fault tree analysis, this coefficient is usually set to 0.2;

[0136] The sum of the above three weighting coefficients is 1 (i.e. 0.5 + 0.3 + 0.2 = 1), which ensures that the numerical scale of the defect severity coefficient is unified and can be explained. The specific values of each coefficient can be fine-tuned at the initial stage of the device operation according to the boiler type (such as subcritical, supercritical), service life and historical defect database, or can be updated online by Bayesian optimization or minimizing the misclassification loss. This setting method respects the physical law and combines engineering experience, so that the final defect severity coefficient can truly and balancedly reflect the comprehensive risk level of the defect.

[0137] S515, multiply the thermal anomaly peak value, area growth rate and spatial coverage breadth of each real defect area by the corresponding first weighting coefficient, second weighting coefficient and third weighting coefficient respectively, and sum the three multiplication results to obtain the defect severity coefficient of each real defect area;

[0138] It is worth noting that before calculating the defect severity coefficient, first ensure that the three input parameters are dimensionless or have been converted to a unified relative risk scale to ensure that the physical meaning of weighted summation is reasonable;

[0139] The thermal anomaly peak value is originally in Celsius (℃), which needs to be normalized: divide it by a preset reference temperature rise threshold (for example, 100℃, which is determined based on the temperature resistance limit of boiler materials and typical leakage temperature rise statistics), and limit the result to the interval of 0 to 1; if the thermal anomaly peak value exceeds 100℃, the normalized value is taken as 1. This processing makes this parameter reflect the "serious proportion relative to the typical high-risk temperature rise";

[0140] The area growth rate is originally in pixels / frame, which also needs to be dimensionless: divide it by a preset maximum reference growth rate (for example, 5 pixels / frame, representing the upper limit of rapid expansion defects), if the result is greater than 1, it is truncated to 1, and if it is less than 0, it is set to 0 (because negative growth is not considered as risk aggravation). The resulting value represents "the proportion of the current expansion speed to the maximum dangerous expansion speed";

[0141] The spatial coverage breadth, originally in pixel, also needs to be normalized: divide it by a reference area of a typical high-risk defect (e.g. 500 pixels, corresponding to an actual pipe surface area of about 10 cm2, calibrated according to the imaging resolution and the pipe diameter of the boiler), the result is also limited between 0 and 1, to express the "proportion of the current impact range to the high-risk area benchmark";

[0142] After the above non-dimensionalization, multiply the normalized thermal anomaly peak by a first weighting coefficient (usually 0.5), multiply the normalized area growth rate by a second weighting coefficient (usually 0.3), multiply the normalized spatial coverage breadth by a third weighting coefficient (usually 0.2), and then add the three products to obtain the defect severity coefficient of the real defect area;

[0143] The coefficient value ranges from 0 to 1, the higher the value, the greater the comprehensive risk of the defect in terms of energy leakage intensity, dynamic expansion trend and spatial impact range. For example, a defect with a normalized thermal anomaly peak of 0.8, an area growth rate of 0.6, and a spatial coverage breadth of 0.5, its defect severity coefficient is: 0.8x0.5 + 0.6x0.3 + 0.5x0.2 = 0.40 + 0.18 + 0.10 = 0.68. This quantitative result provides a direct basis for subsequent grading decisions, and since each parameter has been non-dimensionalized and the weight sum is 1, it ensures the comparability and engineering interpretability of the score;

[0144] It is worth further explaining that the defect severity coefficient calculation method defined in S511 to S515 constructs a quantitative evaluation model based on multi-dimensional physical feature fusion, effectively overcoming the technical limitations of traditional boiler defect detection which can only provide binary judgment (existence / nonexistence) and cannot support refined operation and maintenance decisions. The method first obtains the highest temperature value of each real defect area in the current frame of infrared thermal imaging image, and subtracts the normal operation background temperature of the boiler to obtain the thermal anomaly peak value, which represents the local energy leakage intensity. Secondly, based on the pixel area of M consecutive frames in the time series trajectory, the area growth rate is calculated by linear fitting slope, reflecting whether the defect area presents a continuous expansion trend. Thirdly, the spatial coverage is obtained by averaging the pixel area of all frames in the trajectory, which is used to measure the overall influence range of the defect in the observation period. The above three indexes respectively depict the comprehensive risk characteristics of the defect from the three orthogonal dimensions of thermodynamic intensity, dynamic evolution trend and spatial scale. On this basis, by setting reasonable first weighting coefficient, second weighting coefficient and third weighting coefficient, and weighting the dimensionless indexes, a unified defect severity coefficient is finally generated. This coefficient has clear physical meaning and interpretability, and the higher the value represents the greater the risk of the defect in the three aspects of temperature rise amplitude, expansion speed and coverage range, providing a scientific, quantifiable and comparable basis for subsequent risk classification and response strategy formulation, thus realizing the key transition from defect identification to risk assessment.

[0145] S521, set a fifth threshold and a sixth threshold of the defect severity coefficient, wherein the fifth threshold is less than the sixth threshold;

[0146] It is worth noting that after completing a large number of historical defect cases and accumulating operation data, the defect severity coefficient of the confirmed real defect area is calculated, and the risk level is back annotated combined with the subsequent actual operation and maintenance results (such as whether it causes shutdown within 24 hours, whether it needs emergency treatment, whether it can be delayed for maintenance, etc.). According to the annotation results, the defects are divided into three categories: high-risk defects (need immediate intervention), medium-risk defects (need planned maintenance) and low-risk defects (can continue to monitor). Then, the defect severity coefficient distribution corresponding to each type of defect is counted;

[0147] The sixth threshold is set as the 10th percentile of the defect severity coefficient of the high-risk defect sample. For example, if the coefficients of high-risk defects are concentrated between 0.70 and 1.0, the 10th percentile is about 0.72, and a suitable 5% downward adjustment is made as a safety margin, and the sixth threshold is finally set as 0.68. This threshold ensures that most high-risk defects that really need emergency treatment (more than 90%) are accurately included in the highest risk group, avoiding missing major hidden dangers;

[0148] The fifth threshold is set as the 90th percentile of the defect severity coefficients of the medium-risk defect samples. For example, if the coefficients of the medium-risk defects are mainly distributed between 0.40 and 0.65, the 90th percentile is about 0.63, and the fifth threshold is set as 0.66 after being increased by 5% as a buffer. The threshold ensures that typical medium-risk defects are not misclassified into the high-risk group, and effectively isolates low-risk defects (coefficients generally lower than 0.4);

[0149] The two thresholds satisfy that the fifth threshold is less than the sixth threshold (e.g., 0.66 < 0.68), and a narrow overlapping interval (e.g., 0.66-0.68) is formed in the middle, which is used to contain boundary cases to avoid grouping jumps caused by slight fluctuations. Under this setting, defects with a defect severity coefficient greater than or equal to the sixth threshold are classified into the first group (high-risk), defects with a defect severity coefficient less than the sixth threshold but greater than or equal to the fifth threshold are classified into the second group (medium-risk), and defects with a defect severity coefficient less than the fifth threshold are classified into the third group (low-risk);

[0150] The above-mentioned threshold settings are based on the statistical correlation between real failure consequences and quantitative indicators, which not only reflect the engineering safety logic (high-risk must respond quickly, and low-risk can be handled slowly), but also reserve reasonable tolerance space. The threshold values can be adjusted in the initialization stage according to the type of the boiler, the service stage, or the enterprise operation strategy, and can also be continuously calibrated by adding new cases to ensure that the classification results always match the actual risk level.

[0151] S522, determining whether the defect severity coefficient of the current real defect area is greater than or equal to the sixth threshold;

[0152] S523, if the defect severity coefficient of the current real defect area is greater than or equal to the sixth threshold, the current real defect area is classified into the first group;

[0153] S524, determining whether the defect severity coefficient of the current real defect area is less than the fifth threshold;

[0154] S525, if the defect severity coefficient of the current real defect area is less than the fifth threshold, the current real defect area is classified into the third group;

[0155] S526, if the defect severity coefficient of the current real defect area is greater than or equal to the fifth threshold and less than the sixth threshold, the current real defect area is classified into the second group;

[0156] It is worth mentioning that by setting the fifth threshold and the sixth threshold of the defect severity coefficient in S521 to S526 (wherein the fifth threshold is less than the sixth threshold), a three-grade risk grouping mechanism with clear structure and rigorous logic is established, effectively solving the engineering problem that the defect detection result is difficult to be converted into differentiated operation and maintenance action. The grouping mechanism takes the quantitative coefficient as the only criterion: when the defect severity coefficient is greater than or equal to the sixth threshold, it is determined as a high-risk state and is classified into the first group; when the coefficient is less than the fifth threshold, it is considered as a low-risk state and is classified into the third group; the intermediate between the two is classified into the second group, representing a medium risk. This three-part structure not only conforms to the grading management principle of "high risk immediately disposed, medium risk planned for processing, and low risk continuously monitored" in the field of industrial safety, but also avoids the complexity of operation caused by excessive subdivision. More importantly, each group can directly correspond to a preset operation and maintenance response instruction template, for example, the first group triggers an emergency alarm and shutdown suggestion, the second group generates a planned maintenance work order, and the third group is only included in the regular tracking list. Thus, the defect detection result no longer stays at the image analysis level, but seamlessly connects to the equipment maintenance process, forming a complete closed loop of "detection-evaluation-grouping-response". This mechanism significantly improves the precision, timeliness and traceability of boiler operation and maintenance, making intelligent defect detection truly serve the actual needs of safety production, and has outstanding technical advancement and engineering landing value.

[0157] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting defects in boiler equipment based on image processing, characterized in that, Includes the following steps: S1. While the boiler is running, simultaneously acquire visible light image sequences and infrared thermal imaging image sequences of the area to be inspected, wherein the timestamps of the visible light image sequences and the infrared thermal imaging sequences are aligned and the spatial fields of view overlap. S2. Perform moving target segmentation on the visible light image sequence to obtain the first dynamic mask; extract temperature anomaly regions from the infrared thermal imaging image sequence to obtain the second dynamic mask; perform spatiotemporal consistency verification on the first dynamic mask and the second dynamic mask to generate a set of candidate defect regions; S3. For each candidate defect region in the candidate defect region set, calculate four feature indicators: multimodal difference, motion stability, shape irregularity, and background fusion. Based on the four characteristic indicators, a preliminary logical judgment is performed on each candidate defect region to filter out suspected defect regions, including: S31. Set the first and second thresholds for multimodal difference, the upper threshold for motion stability, the lower threshold for shape irregularity, and the lower threshold for background blending. S32. Determine whether the shape irregularity of the current candidate defect region is greater than or equal to the lower limit threshold of the shape irregularity, and whether the background blending degree of the current candidate defect region is greater than or equal to the lower limit threshold of the background blending degree. S33. If the judgment result of S32 is yes, then further determine whether the current candidate defect region meets any of the following conditions: Condition 1: The multimodal difference of the current candidate defect region is greater than or equal to the first threshold of multimodal difference and less than the second threshold of multimodal difference, and the motion stability of the current candidate defect region is less than the upper limit threshold of motion stability; Condition 2: The multimodal variability of the current candidate defect region is greater than or equal to the second threshold of multimodal variability, and the shape irregularity of the current candidate defect region is greater than or equal to 1.5 times the lower limit threshold of shape irregularity; S34. If any condition in S33 is true, then mark the current candidate defect region as a suspected defect region. S35. If the judgment result of S32 is negative, or if the judgment results of both conditions in S33 are negative, then the current candidate defect region will be removed. S4. For each suspected defect area, calculate a comprehensive spatiotemporal confidence score, and make a final judgment on each suspected defect area based on the comprehensive spatiotemporal confidence score to obtain the true defect area. S5. For each real defect area, calculate the defect severity coefficient, and group all real defect areas according to the defect severity coefficient. Based on the grouping results, generate operation and maintenance response instructions for the real defect areas under each group, and implement maintenance intervention for each real defect area under each group according to the operation and maintenance response instructions.

2. The boiler equipment defect detection method based on image processing according to claim 1, characterized in that, For each suspected defect area, a comprehensive spatiotemporal confidence score is calculated, including: S411. Track the trajectory of each suspected defect area in N consecutive frames to form a time-series trajectory of each suspected defect area. S412. Based on the time series trajectory of each suspected defect area, count the number of frames in which each suspected defect area is identified as a suspected defect area within the time window, and calculate the ratio of the number of frames to the total number of frames in the time window as the local confidence ratio of each suspected defect area. S413. Based on the time series trajectory of each suspected defect area, calculate the linear regression slope of the area of ​​each suspected defect area as a function of time, and use it as the intensity of the diffusion trend of each suspected defect area. S414. Based on the time series trajectory of each suspected defect area, obtain the multimodal difference degree of each suspected defect area in each frame, and calculate the variance of the multimodal difference degree within the time window as the multimodal consistency index of each suspected defect area. S415. Set the first weight coefficient for the local confidence ratio, the second weight coefficient for the diffusion trend intensity, and the third weight coefficient for the multimodal consistency index. Then, multiply the local confidence ratio of each suspected defect region, the diffusion trend intensity of each suspected defect region, and the multimodal consistency index of each suspected defect region by the corresponding first weight coefficient, second weight coefficient, and third weight coefficient, and sum them to obtain the comprehensive spatiotemporal confidence score of each suspected defect region.

3. The method for detecting defects in boiler equipment based on image processing according to claim 2, characterized in that, The comprehensive spatiotemporal confidence score is used to make a final judgment on each suspected defect area to obtain the true defect area, including: S421. Set a third and a fourth threshold for a comprehensive spatiotemporal confidence score, wherein the third threshold is less than the fourth threshold; S422. Determine whether the spatiotemporal confidence score of the current suspected defect area is greater than or equal to the fourth threshold. S423. If the spatiotemporal confidence score of the current suspected defect area is greater than or equal to the fourth threshold, then the current suspected defect area is determined to be a real defect area. S424. Determine whether the spatiotemporal confidence score of the current suspected defect area is less than or equal to the third threshold. S425. If the spatiotemporal confidence score of the current suspected defect region is less than or equal to the third threshold, then the current suspected defect region will be removed. S426. If the spatiotemporal confidence score of the current suspected defect area is greater than the third threshold and less than the fourth threshold, then further determine whether the following three conditions are met simultaneously: Condition 3: The local confidence percentage of the current suspected defect area is greater than or equal to the preset local confidence percentage threshold; Condition 4: The current diffusion trend intensity of the suspected defect area is greater than zero; Condition 5: The multimodal consistency index of the current suspected defect area is less than or equal to the preset multimodal consistency index threshold; S427. If all three conditions above are met, the current suspected defect area is determined to be a real defect area; otherwise, the current suspected defect area is removed.

4. The boiler equipment defect detection method based on image processing according to claim 3, characterized in that, For each actual defect area, the defect severity coefficient is calculated, including: S511. Obtain the highest temperature value of each real defect area in the current frame of infrared thermal imaging image, and calculate the difference between the highest temperature value and the background temperature of normal boiler operation, as the thermal anomaly peak value of each real defect area. S512. Obtain the pixel area of ​​each real defect region in the time series trajectory for M consecutive frames, and calculate the slope of the linear fitting of the pixel area of ​​the M consecutive frames with time as the area growth rate of each real defect region. S513. Obtain the pixel area of ​​each real defect region in all frames of the time series trajectory, and take the average of the pixel areas of all frames as the spatial coverage of each real defect region. S514, set the first weighting coefficient for the peak value of thermal anomalies, the second weighting coefficient for the area growth rate, and the third weighting coefficient for the spatial coverage. S515. Multiply the peak thermal anomaly, area growth rate, and spatial coverage of each real defect region by the corresponding first weighting coefficient, second weighting coefficient, and third weighting coefficient, respectively, and sum the three multiplication results to obtain the defect severity coefficient of each real defect region.

5. The boiler equipment defect detection method based on image processing according to claim 4, characterized in that, The grouping of all real defect areas based on the defect severity coefficient includes: S521. Set the fifth and sixth thresholds for the defect severity coefficient, wherein the fifth threshold is less than the sixth threshold; S522. Determine whether the severity coefficient of the current real defect area is greater than or equal to the sixth threshold. S523. If the severity coefficient of the current real defect area is greater than or equal to the sixth threshold, then the current real defect area is classified into the first group. S524. Determine whether the defect severity coefficient of the current real defect area is less than the fifth threshold. S525. If the severity coefficient of the current real defect area is less than the fifth threshold, then the current real defect area is classified into the third group. S526. If the severity coefficient of the current real defect area is greater than or equal to the fifth threshold and less than the sixth threshold, then the current real defect area is classified into the second group.

Images