Boiler coke accumulation detection system based on unmanned aerial vehicle inspection

Abstract

The invention relates to the technical field of image data processing, in particular to a boiler coke accumulation detection system based on unmanned aerial vehicle inspection, the system comprises a processor and a memory, and the processor executes a computer program of the memory to realize the following steps: obtaining at least two correction images and one fusion image of any to-be-detected part in a boiler, if it is detected that a suspected focus area exists in the fused image, a reference area corresponding to the suspected focus area is obtained from each corrected image, and the corrected image is obtained according to the gray value difference of pixel points in the suspected focus area and each reference area, the gray value change features of the pixel points in the suspected focus area and the gray value difference. Obtaining the focus accumulation confidence degree of each suspected focus accumulation area in the fused image; according to the focus accumulation confidence degree of each suspected focus accumulation area, the total weighted frequency number of each gray value in the fused image is obtained, histogram equalization is utilized to carry out enhancement processing on the fused image, an equalized image is obtained, and the accuracy of focus accumulation detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a boiler coke accumulation detection system based on drone inspection. Background Art

[0002] Coke accumulation can occur inside boilers due to factors such as incomplete fuel combustion, uneven flue gas flow, and structural dead corners. Once coke is deposited on heat transfer pipes, convection heating surfaces, or return flow areas, it not only severely reduces heat exchange efficiency but can also cause localized overheating, equipment fatigue, and even fire hazards. Therefore, detecting and evaluating the coke accumulation status inside boilers is a key measure to ensure safe operation and extend the service life of the equipment.

[0003] In existing technologies, drones are usually used to continuously shoot multiple different parts of the boiler along a preset path to obtain a comprehensive static image sequence. Image enhancement techniques (such as histogram equalization) are then used to redistribute the grayscale of the image and improve the global contrast, providing a higher-quality data foundation for subsequent image segmentation, edge detection, and feature recognition, thereby improving the system's recognition accuracy and reducing misjudgments and missed judgments.

[0004] However, during image enhancement using histogram equalization, coke accumulation typically occurs in areas of the boiler with uneven heat exchange, strong airflow disturbance, or blind spots during cleaning, such as pipe elbows, under convection heating surfaces, and in furnace corners. Due to long-term high-temperature operation, oxidation reactions occur in these areas due to the continuous effects of thermal stress, humidity, and chemical components such as oxygen and sulfides in the flue gas. This leads to the formation of dark, loosely structured, and irregularly textured oxide deposits (oxidation spots) on the surface. In grayscale images, oxidation spots exhibit similar grayscale and texture characteristics to coke accumulation. Because histogram equalization assigns equal weight to all pixels during image enhancement and uses this statistical frequency to calculate the cumulative distribution function to map the original grayscale values, both oxidation spots and coke accumulation are enhanced to the same extent, amplifying the grayscale and texture characteristics of oxidation spots, interfering with the accurate identification of coke accumulation. This may lead to misjudgment in subsequent identification, reducing the accuracy and reliability of boiler coke accumulation detection.

[0005] Therefore, how to retain the key features of focus accumulation while suppressing the interference of oxidation spots during the image enhancement process has become an urgent problem that needs to be solved. Summary of the Invention

[0006] In view of this, an embodiment of the present invention provides a boiler coke accumulation detection system based on drone inspection to solve the problem of how to retain the key features of coke accumulation while suppressing the interference of oxidation spots during the image enhancement process.

[0007] An embodiment of the present invention provides a boiler coke accumulation detection system based on drone inspection, comprising a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the following steps are implemented: For any part to be inspected inside the boiler, using a drone to obtain initial images of the part to be inspected at at least two shooting angles, performing correction processing on each initial image to obtain a corrected image, and performing fusion processing on all the corrected images to obtain a fused image; If at least one suspected out-of-focus area is detected in the fused image, for any suspected out-of-focus area, a reference area corresponding to the suspected out-of-focus area is obtained in each corrected image, and a multi-angle difference index of the suspected out-of-focus area is obtained based on the grayscale value difference between the pixel points in the suspected out-of-focus area and each reference area; Obtaining a focus confidence level for any suspected focus area based on grayscale value variation characteristics and grayscale value differences of pixels in any suspected focus area, and multi-angle difference indicators of any suspected focus area; Obtain a focus confidence level for each suspected focus area in the fused image, obtain a total weighted frequency of each grayscale value in the fused image based on the focus confidence level of each suspected focus area, use the total weighted frequency of each grayscale value in the fused image as the frequency of the grayscale value in a histogram equalization process, perform enhancement processing on the fused image using histogram equalization, and obtain an equalized image of any part to be detected, for use in performing focus detection on any part to be detected.

[0008] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The present invention uses a drone to obtain initial images of any part to be detected inside the boiler at at least two shooting angles, performs correction processing on each initial image to obtain a corrected image, and performs fusion processing on all the corrected images to obtain a fused image; if at least one suspected focus area is detected in the fused image, then for any suspected focus area, a reference area corresponding to the any suspected focus area is obtained in each corrected image, and according to the grayscale value difference between the pixel points in the any suspected focus area and each reference area, a multi-angle difference index of the any suspected focus area is obtained; according to the grayscale value difference between the pixel points in the any suspected focus area and each reference area, a multi-angle difference index of the any suspected focus area is obtained; The method comprises the following steps: obtaining the grayscale value change characteristics and grayscale value differences of the pixels in the focal area, and the multi-angle difference index of any suspected focal area, to obtain the focus confidence level of any suspected focal area; obtaining the focus confidence level of each suspected focal area in the fused image, obtaining the total weighted frequency of each grayscale value in the fused image according to the focus confidence level of each suspected focal area, using the total weighted frequency of each grayscale value in the fused image as the frequency of the grayscale value in the histogram equalization processing, and using the histogram equalization to enhance the fused image to obtain the equalized image of any part to be detected, so as to perform focus detection on any part to be detected. Among them, the focus confidence level of any suspected focus area is obtained according to the grayscale value difference of the pixels in any suspected focus area and each reference area, as well as the grayscale value change characteristics and grayscale value difference of the pixels in any suspected focus area, so as to distinguish the focus area and other interference areas in the fused image, and then the total weighted frequency of each grayscale value in the fused image is obtained according to the focus confidence level of each suspected focus area, so that when the fused image is enhanced by histogram equalization, the focus area can dominate the cumulative histogram calculation of the fused image, allowing the equalization process to focus on the real focus area, making the features of the focus area clearer, the contrast improvement more significant, and providing a higher quality equalized image, thereby significantly improving the accuracy and stability of focus identification in UAV inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0010] Figure 1 This is a flow chart of a boiler coke accumulation detection method based on drone inspection provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0011] The embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to be used to explain the present disclosure, but should not be understood as limiting the present disclosure.

[0012] It should be noted that the terms "first," "second," and the like in the specification of the present disclosure and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure.

[0013] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0014] An embodiment of the present invention provides a boiler coke accumulation detection system based on drone inspection, comprising a processor and a memory, wherein the processor executes a computer program in the memory to implement a boiler coke accumulation detection method based on drone inspection, such as Figure 1 As shown, the method includes the following steps: In step S101, for any part to be inspected inside the boiler, an initial image of the part to be inspected is obtained from at least two shooting angles using a drone, each initial image is corrected to obtain a corrected image, and all corrected images are fused to obtain a fused image.

[0015] During long-term operation, coke can accumulate inside boilers due to factors such as incomplete fuel combustion, uneven flue gas flow, and structural dead corners. Once coke is deposited in heat transfer pipes, convection heating surfaces, or return areas, it not only severely reduces heat exchange efficiency but can also cause localized overheating, equipment fatigue, and even fire hazards. Therefore, detecting and evaluating the coke accumulation status inside boilers is a key measure to ensure safe operation and extend the service life of the equipment.

[0016] In existing technologies, drones are usually used to continuously shoot multiple different parts of the boiler along a preset path to obtain a comprehensive static image sequence to fully reflect the structural features and details of the image, ensuring the complete capture of the focused details. Then, image enhancement technology (such as histogram equalization) is used to redistribute the grayscale of the image, making the image brightness level richer and the local contrast more significant, providing a higher-quality data foundation for subsequent image segmentation, edge detection and feature recognition, improving the system's recognition accuracy and reducing misjudgments and missed judgments.

[0017] Since there are multiple parts to be inspected inside the boiler (i.e., parts that require coke accumulation detection, such as the dense tube bundle below the convection heating surface, the back of the furnace water-cooled wall tube row, etc.), this embodiment takes any part to be inspected as an example and performs subsequent coke accumulation detection analysis on any part to be inspected.

[0018] Due to the complex lighting conditions inside the boiler, with the presence of shadows, reflections and local dark corners, a single perspective may not be able to fully capture the shape and position of the accumulated focus. At the same time, in order to prevent occlusion and blind spot problems, and reduce interference from reflections and shadows, in this embodiment, the shooting angle of the drone is adjusted around the center of the part to be detected by slightly rotating the shooting angle left and right (such as ±5°~10°) and fine-tuning the flight altitude up and down (such as ±10cm~20cm). There is no restriction here. According to the specific implementation scenario settings, the initial images of any part to be detected at at least two shooting angles can be obtained to ensure that the images have high consistency. An initial image is obtained at each shooting angle. Due to the limited light source inside the boiler, in order to reduce image noise in low-light environments, the initial image is denoised by bilateral filtering to avoid interference with subsequent grayscale distribution analysis and enhancement processing. Bilateral filtering belongs to the existing technology and will not be repeated here.

[0019] Because the internal environment of the boiler is mainly monochromatic and low-light, the focus of focus accumulation detection is on the difference in structural brightness and darkness, texture contrast, etc., rather than color information. Therefore, the grayscale image can directly reflect the brightness characteristics, which is more conducive to the subsequent analysis of the contrast difference between the focus accumulation area and the background. Therefore, if the drone uses a color camera for shooting, the initial image needs to be grayscaled to obtain a grayscale image. If an industrial black and white camera is used, there is no need to grayscale the initial image. Grayscale processing is a prior art and will not be repeated here.

[0020] Because the drone is limited by the small space, airflow interference and structural occlusion when flying inside the boiler, the initial image is prone to slight blur and jitter. In order to retain the clear texture details in each initial image and improve the image clarity, the initial images need to be fused to obtain a fused image. The method for obtaining the fused image is as follows: (1) extracting key points with stable texture in each initial image through the SIFT algorithm; (2) matching the key points in each initial image to establish a corresponding relationship through key point matching, calculating the homography matrix through the matching point pairs, and determining the transformation relationship from one initial image to another initial image; (3) using affine transformation to perform spatial correction on each initial image so that the pixels in each initial image are aligned to the same coordinate system to obtain a corrected image; (4) using the weighted image pyramid fusion algorithm to fuse all the corrected images (the weighted image pyramid fusion algorithm can give full play to the high-quality features of different regions in each image, balance brightness, enhance details, and support the suppression of blurred frames or abnormal brightness areas. Therefore, in this embodiment, the weighted image pyramid fusion algorithm is used to fuse all the corrected images. There is no restriction here and it can be set according to the specific implementation scenario); SIFT algorithm, key point matching, spatial correction using affine transformation and weighted image pyramid fusion algorithm belong to the existing technology and will not be repeated here.

[0021] At this point, N initial images, N corrected images and one fused image of any part to be detected are obtained.

[0022] In the existing technology, coke accumulation often occurs in areas of the boiler with uneven heat exchange, strong airflow disturbance, or blind cleaning zones, such as pipe elbows, under convection heating surfaces, and in furnace corners. Due to long-term high-temperature operation, the local metal areas in these areas undergo oxidation reactions due to the continuous effects of thermal stress, humidity, and chemical components such as oxygen and sulfides in the flue gas. This in turn forms dark, loosely structured, and irregularly textured oxide deposits (oxidation spots) on the surface. In grayscale images, oxidation spots exhibit grayscale and texture characteristics similar to coke accumulation. Because histogram equalization assigns equal weight to all pixels during image enhancement and uses this statistical frequency to calculate the cumulative distribution function to map the original grayscale values, both oxidation spots and coke accumulation are enhanced to the same extent, amplifying the grayscale and texture characteristics of oxidation spots, interfering with the accurate identification of coke accumulation. This may lead to misjudgment in subsequent identification, reducing the accuracy and reliability of boiler coke accumulation detection.

[0023] Therefore, after detecting the existence of suspected focus areas in the fused image, this embodiment obtains reference areas corresponding to the suspected focus areas in each corrected image respectively, and obtains the focus confidence level of each suspected focus area in the fused image based on the grayscale value difference between the pixels in the suspected focus area and each reference area, the grayscale value change characteristics and grayscale value difference of the pixels in the suspected focus area. Then, based on the focus confidence level of each suspected focus area, the total weighted frequency of each grayscale value in the fused image is obtained, and the fused image is enhanced using histogram equalization to obtain an equalized image, thereby improving the accuracy of focus detection for any part to be detected.

[0024] Because coke and oxidation spots typically exhibit image characteristics such as sudden grayscale changes and clear structural boundaries, resulting in distinct edge contours and a grayscale gradient difference from the background, this embodiment first performs edge detection on the fused image using the Canny edge detection algorithm. The Canny edge detection algorithm outputs a binary image, with white representing edges and black representing background. Connected domain analysis is then used to extract connected domains, marking connected white pixel regions as suspected coke areas. The Canny edge detection algorithm and connected domain analysis are state-of-the-art and will not be further described here.

[0025] In step S102, if at least one suspected focus area is detected in the fused image, a reference area corresponding to any suspected focus area is obtained in each corrected image respectively, and a multi-angle difference index of any suspected focus area is obtained based on the grayscale value difference between the pixel points in any suspected focus area and each reference area.

[0026] If suspected coking areas are detected in the fused image, it means that coking or oxidation spots may exist in the inspected area. In order to reduce the interference of oxidation spots on coking detection and identification and improve the accuracy and reliability of boiler coking detection, it is necessary to analyze the suspected coking areas in the fused image to determine whether the suspected coking areas are actually coking areas.

[0027] Since coke deposits usually exist in the form of accumulation and compaction, they are mostly thick and protruding, with a rough surface and a porous structure, and the specular reflection of visible light is weak. Therefore, when the drone changes the angle of shooting, in each corrected image, the grayscale values ​​of each pixel at the same position maintain a high degree of consistency, and the overall morphology fluctuation is low. Oxidation spots are mostly composed of oxide layers (such as iron oxide) formed after the metal surface undergoes oxidation reaction with high-temperature gas and steam. The structure is relatively shallow, and the oxide layer still has a metal-like structure and has a certain degree of specular reflection. Therefore, oxidation spots are more dependent on the visual performance of the lighting angle. Affected by the shooting angle, the grayscale values ​​of each pixel at the same position in each corrected image fluctuate greatly. At the same time, the overall morphology shows greater differences than coke deposits due to the different incident light at the shooting angle.

[0028] Therefore, for any suspected focus area, a reference area corresponding to any suspected focus area is obtained in each corrected image, and a multi-angle difference index of any suspected focus area is obtained based on the grayscale value difference of the pixel points in any suspected focus area and each reference area.

[0029] The method for obtaining the reference area corresponding to any suspected focus area in each corrected image is as follows: After each initial image is corrected, a spatial coordinate system is obtained, and the spatial coordinates of each pixel point in any suspected focus area are obtained in the spatial coordinate system. Based on the spatial coordinates of each pixel point in any suspected focus area, pixel points with the same spatial coordinates are obtained in each corrected image to form a reference area in each corrected image.

[0030] Furthermore, based on the grayscale value difference between any suspected focus area and each reference area, a method for obtaining a multi-angle difference index for any suspected focus area is as follows: A two-dimensional rectangular coordinate system is constructed with the lower left corner of each corrected image as the origin, the horizontal direction as the abscissa, and the vertical direction as the ordinate. Any corrected image is selected from all corrected images as a reference image. In this embodiment, the corrected image corresponding to the primary viewing angle (i.e., the corrected image corresponding to the first capture of the part to be inspected) is selected as the reference image. This is not limited here and can be set according to specific implementation scenarios. For any pixel point in the reference area of ​​the reference image, the grayscale value of each pixel point with the same coordinates as the any pixel point in all corrected images is obtained to form a grayscale value sequence for the any pixel point. Obtaining a grayscale value sequence for each pixel in a reference area of ​​the benchmark image, obtaining a standard deviation of each grayscale value sequence, obtaining a corresponding standard deviation cumulative value, and normalizing the standard deviation cumulative value to obtain a grayscale difference value for any suspected focus accumulation area; A two-dimensional rectangular coordinate system is constructed with the lower left corner of the fused image as the origin, the horizontal direction as the abscissa, and the vertical direction as the ordinate. Based on the coordinates of the pixel points in each reference area and the coordinates of the pixel points in any suspected focus area, the Hausdorff distance between each reference area and any suspected focus area is obtained, and a corresponding Hausdorff distance cumulative value is obtained. The Hausdorff distance cumulative value is normalized to obtain a distance difference value of any suspected focus area. The Hausdorff distance belongs to the prior art and will not be described in detail here. An average value between the grayscale difference value and the distance difference value is obtained to obtain a multi-angle difference index of any suspected focus accumulation area.

[0031] In one embodiment, the For example, the suspected focal area The calculation formula for the multi-angle difference index of a suspected focus area is:

[0032] in, For the Multi-angle difference indicators of suspected focus areas; is the standard deviation of the i-th gray value sequence; n is the number of gray value sequences; is the jth reference area and the Hausdorff distance of the suspected focal accumulation area; For the The jth reference area of ​​the suspected focal accumulation area; The first suspected focus areas; N is the number of reference areas (that is, the number of correction images); is the normalization function.

[0033] It should be noted that For the Grayscale difference value of the suspected focus area, The larger the The greater the fluctuation of the grayscale value of the pixel in the suspected focus area at multiple angles, the greater the difference in the pixel changes at different angles. The bigger it is, the the bigger it is; For the The distance difference value of the suspected focus area, The larger the value, the greater the difference in shape and position of the area when photographed at different angles. The bigger it is, the the bigger it is; The larger the There are significant differences in the images taken at multiple angles for the suspected focus area, indicating that this area is more likely to be an oxidation spot area.

[0034] At this point, the multi-angle difference index of any suspected focus accumulation area is obtained.

[0035] Step S103 , obtaining the focus confidence level of any suspected focus area according to the grayscale value variation characteristics and grayscale value differences of the pixels in any suspected focus area, and the multi-angle difference index of any suspected focus area.

[0036] Coke is a charred substance deposited on the boiler's heating surface over a long period of time due to incomplete fuel combustion, impurity accumulation, or obstructed heat exchange. This substance contains a variety of particulate impurities or cracks, resulting in an uneven surface with varying thicknesses. This leads to extremely uneven absorption and reflection of light during photography. Oxidation spots, on the other hand, are formed when metal materials oxidize in a high-temperature environment, forming a relatively uniform oxide film or oxidation product. These oxides usually adhere to the metal surface, so in the final fused image, the grayscale variations of the pixels in the coke area are more dramatic than those in the oxidation spot area. Furthermore, because the interior of a boiler is typically composed of metal or high-temperature layers, they are more evenly affected by lighting under a stable light source. Therefore, in the fused image, the grayscale values ​​of the pixels in the background are relatively uniform, while the grayscale values ​​of the pixels in the coke area are more complex. This means that the number of grayscale levels in the coke area is significantly higher than that in the oxidation spot area.

[0037] Because of the physical process of coke formation and accumulation, coke particles accumulate over time, and the accumulation time in the center is longer, forming a pyramid-like structure. Therefore, the grayscale value in the center of the coke area is usually higher than that in the edge area, and there is a grayscale change from the center to the edge. The formation of oxidation spots mainly depends on local chemical reactions and temperature conditions, and is less affected by airflow and particle deposition. It does not have a clear three-dimensional structure like the coke area, and usually has no obvious hierarchical structure as a whole.

[0038] Therefore, the focus confidence level of any suspected focus area can be obtained based on the grayscale value change characteristics and grayscale value differences of the pixels in any suspected focus area, as well as the multi-angle difference indicators of any suspected focus area, to determine whether any suspected focus area is a focus area.

[0039] The method for obtaining the focus confidence level of any suspected focus area is as follows based on the grayscale value variation characteristics and grayscale value differences of the pixels in any suspected focus area, as well as the multi-angle difference index of any suspected focus area: (1) Eliminate all suspected defocused areas in the fused image to obtain normal areas, and obtain the grayscale change degree of any suspected defocused area based on the grayscale value change characteristics of the pixels in any suspected defocused area and the grayscale distribution characteristics of any suspected defocused area and the normal area.

[0040] Specifically, a grayscale value mean of pixels in any suspected focus accumulation area is obtained, and the absolute value of the difference between the grayscale value of each pixel in any suspected focus accumulation area and the grayscale value mean is obtained, and the mean of the absolute values ​​of the differences is obtained accordingly. The mean of the absolute values ​​of the differences is normalized to obtain a first grayscale distribution disorder degree of any suspected focus accumulation area; Obtaining the number of gray levels in any suspected focus accumulation area, obtaining the number of gray levels in the normal area, calculating the difference between the number of gray levels in any suspected focus accumulation area and the number of gray levels in the normal area to obtain a gray level number difference, substituting the inverse of the gray level number difference into an exponential function with a natural constant as the base to obtain a second gray level distribution disorder degree of any suspected focus accumulation area; The average of the first grayscale distribution disorder degree and the second grayscale distribution disorder degree is obtained to obtain the grayscale change degree of any suspected focus accumulation area.

[0041] In one embodiment, the For example, the suspected focal area The calculation formula for the grayscale change degree of a suspected focus area is:

[0042] in, For the The degree of grayscale change in the suspected focus accumulation area; For the The grayscale value of the kth pixel in the suspected focus area; m is the grayscale value of the kth pixel in the suspected focus area; The number of pixels in the suspected focus area; For the The number of gray levels in the suspected focus accumulation area; is the number of gray levels in the normal area; is the normalization function; It is an exponential function with a natural constant as the base, used for inverse proportional normalization; is the absolute value symbol.

[0043] It should be noted that For the The degree of disorder of the first grayscale distribution of the suspected focus accumulation area, The larger the The grayscale value of each pixel in the suspected focus area is The greater the difference between the grayscale mean values ​​of the suspected focus areas, the greater the difference between the grayscale mean values ​​of the suspected focus areas. The grayscale differences between pixels in the suspected focus area are large, and the overall grayscale distribution is uneven. The bigger it is, the the bigger it is; For the The degree of disorder of the second grayscale distribution of the suspected focus accumulation area, The larger the The grayscale distribution inside the suspected focus area is quite diverse, reflecting that the surface of this area has complex texture features and a strong contrast with the normal area. The bigger it is, the the bigger it is; The larger the The grayscale differences between pixels in the suspected focus area are large, and the overall grayscale distribution is uneven. At the same time, the surface of the area has complex texture features, which contrasts strongly with the normal area, making it more likely to be a focus area.

[0044] (2) Obtain the degree of accumulation characteristics of any suspected focal accumulation area.

[0045] Specifically, the abscissas of all pixels in any suspected focus area are combined into a set of abscissas, and the ordinates of all pixels in any suspected focus area are combined into a set of ordinates. The means of the abscissas and ordinates are respectively obtained to form mean coordinates. The pixel corresponding to the mean coordinates is recorded as the center pixel, and the eight neighborhoods of the center pixel are taken as the center area. The Canny edge detection algorithm is used to obtain the edge pixel points of any suspected focus accumulation area. The Canny edge detection algorithm belongs to the existing technology and will not be described in detail here. The grayscale value mean of the pixel points in the central area is obtained, which is recorded as the comparison mean. The absolute value of the difference between the grayscale value of each edge pixel point of any suspected focus accumulation area and the comparison mean is calculated respectively, and the corresponding absolute value accumulation value is obtained. The absolute value accumulation value of the difference is normalized to obtain the degree of accumulation characteristics of any suspected focus accumulation area.

[0046] In one embodiment, the For example, the suspected focal area The calculation formula for the accumulation characteristic degree of a suspected coke accumulation area is:

[0047] in, For the The degree of accumulation characteristics of the suspected focal area; is the comparison mean; For the The grayscale value of the zth edge pixel in the suspected focus area; t is the The number of edge pixels in the suspected focus area; is the normalization function; is the absolute value symbol.

[0048] It should be noted that The larger the The more significant the grayscale change from the center to the edge of a suspected focus area is, the more obvious the gradient change characteristics of the area are, and the stronger the overall accumulation hierarchy is. The bigger, The more likely a suspected focus accumulation area is, the more likely it is to be a focus accumulation area.

[0049] (3) Obtaining the focus confidence level of any suspected focus area based on the grayscale change level, the accumulation feature level, and the multi-angle difference index.

[0050] Specifically, normalizing the inverse of the multi-angle difference index to obtain a first focus accumulation confidence level of any suspected focus accumulation area; Obtaining a mean value between the grayscale variation degree and the accumulation feature degree to obtain a second focus accumulation confidence level of any suspected focus accumulation area; Obtain an average of the first focus accumulation confidence level and the second focus accumulation confidence level to obtain the focus accumulation confidence level of any suspected focus accumulation area.

[0051] In one embodiment, the For example, the suspected focal area The calculation formula for the focus accumulation confidence of a suspected focus accumulation area is:

[0052] in, For the Focus confidence level of each suspected focus area; For the Multi-angle difference indicators of suspected focus areas; For the The degree of grayscale change in the suspected focus accumulation area; For the The degree of accumulation characteristics of the suspected focal area; is the normalization function.

[0053] It should be noted that For the The first focus confidence level of the suspected focus area, The larger the The suspected focus area has large differences in images taken from multiple angles. This area is more likely to be an oxidation spot area. The smaller it is, the The smaller it is; For the The second focus confidence level of the suspected focus area, The larger the In the suspected focus area, the grayscale difference between each pixel is large, the overall grayscale distribution is uneven, and the surface of the area has complex texture features, which is in sharp contrast with the normal area. It is more likely to be a focus area. The larger the The more significant the grayscale change from the center to the edge of the suspected focus area is, the more obvious the gradient change is. The more likely a suspected focus area is to be a focus area, The bigger it is, the The bigger it is.

[0054] At this point, the focus accumulation confidence level of any suspected focus accumulation area is obtained.

[0055] Step S104: Obtain a focus confidence level for each suspected focus area in the fused image, obtain a total weighted frequency of each grayscale value in the fused image based on the focus confidence level of each suspected focus area, use the total weighted frequency of each grayscale value in the fused image as the frequency of the grayscale value in a histogram equalization process, and perform enhancement processing on the fused image using histogram equalization to obtain an equalized image of any part to be detected, for use in performing focus detection on any part to be detected.

[0056] According to any of the above-mentioned methods for obtaining the focus confidence level of suspected focus areas, the focus confidence level of each suspected focus area in the fused image is obtained, and then based on the focus confidence level of each suspected focus area in the fused image, it is determined whether each suspected focus area is a focus area.

[0057] The method for determining whether each suspected focus area is a focus area according to the focus confidence level of each suspected focus area in the fused image is as follows: The focus confidence threshold is set to 0.7. There is no restriction here. It can be set according to the specific implementation scenario. The suspected focus area with a focus confidence greater than 0.7 is recorded as the focus area, and the suspected focus area with a focus confidence less than or equal to 0.7 is recorded as the interference area.

[0058] Furthermore, the total weighted frequency of each grayscale value in the fused image is obtained according to the focus confidence level of each suspected focus area, so that when the fused image is enhanced using histogram equalization, the focus area can dominate the cumulative histogram calculation of the fused image, allowing the equalization process to focus on the real focus area, making the features of the focus area clearer, the contrast improvement more significant, and providing a higher quality equalized image, thereby significantly improving the accuracy and stability of focus identification in UAV inspections.

[0059] The method for obtaining the total weighted frequency of each gray value in the fused image according to the focus confidence level of each suspected focus area is as follows: (1) Obtain the weighted frequency of each grayscale value in each focus area according to the focus confidence level of each focus area.

[0060] For any focus accumulation area, the frequency of each grayscale value in the any focus accumulation area is counted; for any grayscale value, the sum of a constant 1 and the focus confidence level of the any focus accumulation area is obtained to obtain a focus weight coefficient of the frequency of the any grayscale value; the product of the frequency of the any grayscale value and its focus weight coefficient is rounded up to obtain a weighted frequency of the any grayscale value.

[0061] In one embodiment, taking the cth gray value in the bth focus accumulation area as an example, the calculation formula for the weighted frequency of the cth gray value in the bth focus accumulation area is:

[0062] in, is the weighted frequency of the cth gray value in the bth focal accumulation area; is the frequency of the cth gray value in the bth focal accumulation area; is the confidence level of focus accumulation in the bth focus accumulation area; The symbol for rounding up.

[0063] It should be noted that is the focus weight coefficient of the gray value frequency in the bth focus area, The larger the value is, the more likely the bth focal accumulation area is to be a focal accumulation area, and the more enhancement processing is needed. The bigger it is, the The bigger it is.

[0064] (2) Obtain the weighted frequency of each grayscale value in each interference area according to the confidence level of the focus accumulation in each interference area.

[0065] For any interference area, count the frequency of each grayscale value in the any interference area, for any grayscale value, obtain the difference between the constant 1 and the focus confidence level of the any interference area, obtain the interference weight coefficient of the frequency of the any grayscale value, round down the product of the frequency of the any grayscale value and its interference weight coefficient, and obtain the weighted frequency of the any grayscale value.

[0066] In one embodiment, taking the e-th grayscale value in the d-th interference area as an example, the calculation formula for the weighted frequency of the e-th grayscale value in the d-th interference area is:

[0067] in, is the weighted frequency of the e-th gray value in the d-th interference area; is the frequency of the e-th gray value in the d-th interference area; is the confidence level of the focus accumulation in the d-th interference area; is the floor symbol.

[0068] It should be noted that is the focal weight coefficient of the gray value frequency in the dth interference area, The larger the value is, the more likely it is that the dth interference area belongs to the oxidation spot area, and the less enhancement processing is needed. The smaller it is, the The smaller it is.

[0069] (3) Obtain the frequency of each gray value in the normal area, and obtain the total weighted frequency of each gray value in the fused image based on the weighted frequency of each gray value in each focal area, the weighted frequency of each gray value in each interference area, and the frequency of each gray value in the normal area.

[0070] Specifically, the normal area, the focused area and the interference area in the fused image are all recorded as the areas to be counted. For any grayscale value in the fused image, the sum of the weighted frequencies of any grayscale value in each area to be counted is obtained to obtain the total weighted frequency of any grayscale value in the fused image.

[0071] Similarly, the total weighted frequency of each gray value in the fused image is obtained.

[0072] After obtaining the total weighted frequency of each grayscale value in the fused image, the total weighted frequency of each grayscale value in the fused image is used as the frequency of the grayscale value in the histogram equalization process. The fused image is enhanced using histogram equalization. That is, a weighted grayscale histogram is obtained based on the total weighted frequency of each grayscale value in the fused image, a cumulative weighted histogram (CDF) is calculated, and a grayscale mapping table is obtained based on the cumulative weighted histogram. The fused image is mapped using the grayscale mapping table so that the grayscale structure of the focused area dominates the mapping function, enhancing the contrast of the focused area, making features such as focused textures and edges clearer and easier to distinguish in the image. At the same time, the interference of normal areas (i.e., background parts) and oxidation spots is effectively suppressed, allowing the equalization process to focus on the focused area, making the features of the focused area clearer and the contrast more significantly improved. A higher-quality equalized image of any part to be inspected is obtained, which is used for focused detection of any part to be inspected, thereby significantly improving the accuracy and stability of focused detection in UAV inspections.

[0073] The main purpose of the present invention is to obtain the total weighted frequency of each pixel in the fused image, so that the key features of the focus accumulation can be retained while suppressing the interference of oxidation spots during the image enhancement process using the total weighted frequency of each pixel in the fused image. Among them, the use of histogram equalization to enhance the fused image belongs to the existing technology and will not be repeated here.

[0074] In summary, the embodiment of the present invention uses a drone to obtain initial images of any part to be detected inside the boiler at at least two shooting angles, performs correction processing on each initial image to obtain a corrected image, and performs fusion processing on all the corrected images to obtain a fused image; if at least one suspected focus area is detected in the fused image, then for any suspected focus area, a reference area corresponding to the any suspected focus area is obtained in each corrected image, and a multi-angle difference index of the any suspected focus area is obtained according to the grayscale value difference of the pixel points in the any suspected focus area and each reference area; according to any The method comprises the following steps: obtaining the grayscale value change characteristics and grayscale value differences of pixels in a suspected focus area, and multi-angle difference indicators of any suspected focus area, to obtain the focus confidence level of any suspected focus area; obtaining the focus confidence level of each suspected focus area in the fused image, obtaining the total weighted frequency of each grayscale value in the fused image according to the focus confidence level of each suspected focus area, using the total weighted frequency of each grayscale value in the fused image as the frequency of the grayscale value in the histogram equalization processing, and using the histogram equalization to enhance the fused image to obtain the equalized image of any part to be detected, so as to perform focus detection on any part to be detected. Among them, the focus confidence level of any suspected focus area is obtained according to the grayscale value difference of the pixels in any suspected focus area and each reference area, as well as the grayscale value change characteristics and grayscale value difference of the pixels in any suspected focus area, so as to distinguish the focus area and other interference areas in the fused image, and then the total weighted frequency of each grayscale value in the fused image is obtained according to the focus confidence level of each suspected focus area, so that when the fused image is enhanced by histogram equalization, the focus area can dominate the cumulative histogram calculation of the fused image, allowing the equalization process to focus on the real focus area, making the features of the focus area clearer, the contrast improvement more significant, and providing a higher quality equalized image, thereby significantly improving the accuracy and stability of focus identification in UAV inspections.

[0075] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A boiler coke accumulation detection system based on drone inspection, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the following steps are implemented: For any part to be inspected inside the boiler, using a drone to obtain initial images of the part to be inspected at at least two shooting angles, performing correction processing on each initial image to obtain a corrected image, and performing fusion processing on all the corrected images to obtain a fused image; If at least one suspected out-of-focus area is detected in the fused image, for any suspected out-of-focus area, a reference area corresponding to the suspected out-of-focus area is obtained in each corrected image, and a multi-angle difference index of the suspected out-of-focus area is obtained based on the grayscale value difference between the pixel points in the suspected out-of-focus area and each reference area; Obtaining a focus confidence level for any suspected focus area based on grayscale value variation characteristics and grayscale value differences of pixels in any suspected focus area, and multi-angle difference indicators of any suspected focus area; Obtain a focus confidence level for each suspected focus area in the fused image, obtain a total weighted frequency of each grayscale value in the fused image based on the focus confidence level of each suspected focus area, use the total weighted frequency of each grayscale value in the fused image as the frequency of the grayscale value in a histogram equalization process, perform enhancement processing on the fused image using histogram equalization, and obtain an equalized image of any part to be detected, for use in performing focus detection on any part to be detected.

2. A boiler coke accumulation detection system based on drone inspection according to claim 1, characterized in that: The step of obtaining a multi-angle difference index of any suspected focus area according to a difference in grayscale values ​​of pixels in any suspected focus area and each reference area includes: A two-dimensional rectangular coordinate system is constructed with the lower left corner of each corrected image as the origin, the horizontal direction as the abscissa, and the vertical direction as the ordinate. One corrected image is selected from all corrected images as a reference image. For any pixel point in the reference area of ​​the reference image, the grayscale value of each pixel point with the same coordinates as the pixel point in all corrected images is obtained to form a grayscale value sequence for the pixel point. Obtain a grayscale value sequence for each pixel in the reference area of ​​the baseline image, and obtain a multi-angle difference index for any suspected focus area based on the data fluctuation characteristics of the grayscale value sequence for each pixel in the reference area of ​​the baseline image and the distance difference between each reference area and any suspected focus area.

3. A boiler coke accumulation detection system based on drone inspection according to claim 2, characterized in that: The obtaining of a multi-angle difference index of any suspected focus area based on a data fluctuation feature of a grayscale value sequence of each pixel in a reference area of ​​the benchmark image and a distance difference between each reference area and any suspected focus area includes: Obtaining the standard deviation of each grayscale value sequence respectively, obtaining a corresponding standard deviation cumulative value, and normalizing the standard deviation cumulative value to obtain a grayscale difference value of any suspected focus accumulation area; A two-dimensional rectangular coordinate system is constructed with the lower left corner of the fused image as the origin, the horizontal direction as the abscissa, and the vertical direction as the ordinate. Based on the coordinates of the pixel points in each reference area and the coordinates of the pixel points in any suspected focus area, the Hausdorff distance between each reference area and the any suspected focus area is obtained, and the corresponding Hausdorff distance accumulation value is obtained. The Hausdorff distance accumulation value is normalized to obtain the distance difference value of the any suspected focus area. An average value between the grayscale difference value and the distance difference value is obtained to obtain a multi-angle difference index of any suspected focus accumulation area.

4. The boiler coke accumulation detection system based on drone inspection according to claim 1 is characterized in that: The obtaining of the focus confidence level of any suspected focus area according to the grayscale value variation characteristics and grayscale value differences of the pixels in any suspected focus area and the multi-angle difference index of the any suspected focus area includes: Eliminating all suspected focus areas in the fused image to obtain normal areas, and obtaining a grayscale change degree of any suspected focus area based on grayscale value change characteristics of pixels in any suspected focus area and grayscale distribution characteristics of any suspected focus area and the normal area; Obtaining a central area in any of the suspected focus accumulation areas, obtaining edge pixels of any of the suspected focus accumulation areas using a Canny edge detection algorithm, and obtaining a degree of accumulation characteristics of any of the suspected focus accumulation areas based on a difference in grayscale values ​​between the edge pixels of any of the suspected focus accumulation areas and the pixels in the central area; The focus confidence level of any suspected focus accumulation area is obtained according to the grayscale change level, the accumulation feature level and the multi-angle difference index.

5. The boiler coke accumulation detection system based on drone inspection according to claim 4 is characterized in that: The obtaining, based on the grayscale value variation characteristics of the pixels in the any suspected focus accumulation area and the grayscale distribution characteristics of the any suspected focus accumulation area and the normal area, of the grayscale variation degree of the any suspected focus accumulation area includes: Obtaining a grayscale value mean of pixels in any suspected focus accumulation area, respectively obtaining the absolute value of the difference between the grayscale value of each pixel in any suspected focus accumulation area and the grayscale value mean, obtaining a corresponding mean of the absolute values ​​of the differences, and normalizing the mean of the absolute values ​​of the differences to obtain a first grayscale distribution disorder degree of any suspected focus accumulation area; Obtaining the number of gray levels in any suspected focus accumulation area, obtaining the number of gray levels in the normal area, calculating the difference between the number of gray levels in any suspected focus accumulation area and the number of gray levels in the normal area to obtain a gray level number difference, substituting the inverse of the gray level number difference into an exponential function with a natural constant as the base to obtain a second gray level distribution disorder degree of any suspected focus accumulation area; The average of the first grayscale distribution disorder degree and the second grayscale distribution disorder degree is obtained to obtain the grayscale change degree of any suspected focus accumulation area.

6. The boiler coke accumulation detection system based on drone inspection according to claim 4 is characterized in that: The obtaining, based on the grayscale value difference between the edge pixels of the suspected focus accumulation area and the pixels in the central area, of the suspected focus accumulation area includes: The grayscale value mean of the pixel points in the central area is obtained and recorded as the comparison mean. The absolute value of the difference between the grayscale value of each edge pixel point of any suspected focus accumulation area and the comparison mean is calculated respectively, and the corresponding absolute value accumulation value of the difference is obtained. The absolute value accumulation value of the difference is normalized to obtain the degree of accumulation characteristics of any suspected focus accumulation area.

7. The boiler coke accumulation detection system based on drone inspection according to claim 4 is characterized in that: The obtaining, according to the grayscale variation degree, the accumulation feature degree, and the multi-angle difference index, a focus accumulation confidence level of any suspected focus accumulation area includes: Normalizing the inverse of the multi-angle difference index to obtain a first focus accumulation confidence level of any suspected focus accumulation area; Obtaining a mean value between the grayscale change degree and the accumulation feature degree to obtain a second focus accumulation confidence level of any suspected focus accumulation area; Obtain an average of the first focus accumulation confidence level and the second focus accumulation confidence level to obtain the focus accumulation confidence level of any suspected focus accumulation area.

8. The boiler coke accumulation detection system based on drone inspection according to claim 4 is characterized in that: The acquiring of a central area in any of the suspected focal accumulation areas comprises: The horizontal coordinates of all pixel points in any suspected focus area are combined into a horizontal coordinate set, and the vertical coordinates of all pixel points in any suspected focus area are combined into a vertical coordinate set. The means of the horizontal coordinate set and the vertical coordinate set are respectively obtained to form mean coordinates. The pixel point corresponding to the mean coordinate is recorded as the center pixel point, and the eight neighborhoods of the center pixel point are taken as the center area.

9. The boiler coke accumulation detection system based on drone inspection according to claim 4 is characterized in that: The obtaining, according to the focus confidence level of each suspected focus area, the total weighted frequency of each grayscale value in the fused image includes: Setting a focus accumulation confidence threshold, recording a suspected focus accumulation area whose focus accumulation confidence level is greater than the focus accumulation confidence threshold as a focus accumulation area, and recording a suspected focus accumulation area whose focus accumulation confidence level is less than or equal to the focus accumulation confidence threshold as an interference area; Obtaining a weighted frequency of each grayscale value in each focus area according to the focus confidence level of each focus area, obtaining a weighted frequency of each grayscale value in each interference area according to the focus confidence level of each interference area, and obtaining the frequency of each grayscale value in the normal area; The total weighted frequency of each gray value in the fused image is obtained according to the weighted frequency of each gray value in each focused area, the weighted frequency of each gray value in each interference area, and the frequency of each gray value in the normal area.

10. The boiler coke accumulation detection system based on drone inspection according to claim 9, characterized in that: The step of obtaining a weighted frequency of each grayscale value in each focus accumulation area according to the focus accumulation confidence level of each focus accumulation area includes: For any focus accumulation area, the frequency of each grayscale value in the any focus accumulation area is counted; for any grayscale value, the sum of a constant 1 and the focus confidence level of the any focus accumulation area is obtained to obtain a focus weight coefficient of the frequency of the any grayscale value; the product of the frequency of the any grayscale value and its focus weight coefficient is rounded up to obtain a weighted frequency of the any grayscale value.

11. The boiler coke accumulation detection system based on drone inspection according to claim 9, characterized in that: The step of obtaining the weighted frequency of each grayscale value in each interference area according to the focus confidence level of each interference area includes: For any interference area, count the frequency of each grayscale value in the any interference area, for any grayscale value, obtain the difference between the constant 1 and the focus confidence level of the any interference area, obtain the interference weight coefficient of the frequency of the any grayscale value, round down the product of the frequency of the any grayscale value and its interference weight coefficient, and obtain the weighted frequency of the any grayscale value.

12. The boiler coke accumulation detection system based on drone inspection according to claim 9, characterized in that: The obtaining, according to the weighted frequency of each gray value in each focal area, the weighted frequency of each gray value in each interference area, and the frequency of each gray value in the normal area, of the total weighted frequency of each gray value in the fused image includes: The normal area, the focused area, and the interference area in the fused image are all recorded as the areas to be counted. For any grayscale value in the fused image, the sum of the weighted frequencies of any grayscale value in each area to be counted is obtained to obtain the total weighted frequency of any grayscale value in the fused image.

13. The boiler coke accumulation detection system based on drone inspection according to claim 1, characterized in that: The step of obtaining, for each suspected focus area, a reference area corresponding to the suspected focus area in each corrected image includes: After each initial image is corrected, a spatial coordinate system is obtained, and the spatial coordinates of each pixel point in any suspected focus area are obtained in the spatial coordinate system. Based on the spatial coordinates of each pixel point in any suspected focus area, pixel points with the same spatial coordinates are obtained in each corrected image to form a reference area in each corrected image.

Images