A boiler coke accumulation detection system based on unmanned aerial vehicle inspection

Abstract

The present application relates to the technical field of image data processing, and especially relates to a boiler coke accumulation detection system based on unmanned aerial vehicle inspection, which comprises a processor and a memory, and the processor executes the computer program of the memory to realize the following steps: acquiring at least two correction images and a fusion image of any to-be-detected part inside a boiler, if a suspected coke accumulation area is detected in the fusion image, acquiring a corresponding reference area of the suspected coke accumulation area in each correction image, acquiring the coke accumulation confidence degree of each suspected coke accumulation area in the fusion image according to the gray value difference between the suspected coke accumulation area and each reference area, the gray value change feature and the gray value difference of the pixel points in the suspected coke accumulation area, acquiring the total weighted frequency of each gray value in the fusion image according to the coke accumulation confidence degree of each suspected coke accumulation area, and performing enhancement processing on the fusion image by using histogram equalization to obtain an equalized image, thereby improving the accuracy of coke accumulation detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, and particularly relates to a boiler coke accumulation detection system based on unmanned aerial vehicle inspection. BACKGROUND

[0002] The coke accumulation phenomenon is caused by incomplete combustion of fuel, uneven flue gas flow or structural dead angle in the boiler. Once the coke is deposited in the heat transfer pipeline, the convection heating surface or the backflow area, it not only seriously reduces the heat exchange efficiency, but also may cause local overheating, equipment fatigue and even fire hazards. Therefore, the detection and evaluation of the coke accumulation state in the boiler is a key means to ensure the safe operation of the equipment and prolong the service life.

[0003] In the prior art, an unmanned aerial vehicle is usually used to continuously shoot multiple different parts in the boiler according to a preset path to obtain a covering sequence of static images, and then the gray levels of the images are redistributed through image enhancement technology (such as histogram equalization) to improve the global contrast, so as to provide a higher quality data basis for subsequent image segmentation, edge detection and feature recognition, and to improve the system recognition accuracy and reduce misjudgment and omission.

[0004] However, in the process of image enhancement using histogram equalization, the coke accumulation phenomenon usually occurs in parts such as pipeline bends, the lower part of the convection heating surface and the corners of the furnace, etc. Due to the long-term high-temperature operation, the local metal area is subjected to the continuous action of heat stress, humidity and chemical components such as oxygen and sulfide in the flue gas, and then an oxidation reaction occurs, forming oxidation deposits (oxidation spots) with dark color, loose structure and irregular texture on the surface. In the gray image, the oxidation spots show similar gray level and texture characteristics as the coke. Since histogram equalization gives equal weight to all pixel points in the process of image enhancement, and the cumulative distribution function is calculated by counting the frequency to map the original gray value, the oxidation spots and the coke are enhanced to the same degree, the gray level and texture characteristics of the oxidation spots are amplified, which interferes with the accurate identification of the coke, and misjudgment may occur in the subsequent identification, reducing the accuracy and reliability of the boiler coke detection.

[0005] Therefore, how to preserve the key features of coke accumulation while suppressing the interference of oxidation spots in the image enhancement process has become a problem to be solved. SUMMARY

[0006] Therefore, the embodiments of the present application provide a boiler coke accumulation detection system based on unmanned aerial vehicle inspection to solve the problem of how to preserve the key features of coke accumulation while suppressing the interference of oxidation spots in the image enhancement process.

[0007] The embodiment of the present application provides a boiler coke accumulation detection system based on unmanned aerial vehicle inspection, which comprises a memory, a processor and a computer program stored in the memory and running on the processor, and the processor realizes the following steps when executing the computer program.

[0008] For any to-be-detected part in the boiler, the unmanned aerial vehicle is used to obtain initial images of the any to-be-detected part at at least two shooting angles, each initial image is corrected to obtain a corrected image, and all corrected images are fused to obtain a fused image.

[0009] If it is detected that there is at least one suspected coke accumulation area in the fused image, for any suspected coke accumulation area, the corresponding reference area of the any suspected coke accumulation area is obtained in each corrected image, the multi-angle difference index of the any suspected coke accumulation area is obtained according to the gray value difference of the pixels in the any suspected coke accumulation area and each reference area.

[0010] According to the gray value change characteristics and the gray value difference of the pixels in the any suspected coke accumulation area and the multi-angle difference index of the any suspected coke accumulation area, the coke accumulation confidence degree of the any suspected coke accumulation area is obtained.

[0011] The coke accumulation confidence degree of each suspected coke accumulation area in the fused image is obtained, the total weighted frequency of each gray value in the fused image is obtained according to the coke accumulation confidence degree of each suspected coke accumulation area, the total weighted frequency of each gray value in the fused image is used as the frequency of the gray value in the histogram equalization processing, the fused image is enhanced by using the histogram equalization to obtain an equalized image of the any to-be-detected part, and the coke accumulation detection of the any to-be-detected part is performed.

[0012] Compared with the prior art, the embodiment of the present application has the beneficial effects that:

[0013] The present application is directed to any to-be-detected part inside a boiler, and the initial image of the any to-be-detected part at at least two shooting angles is obtained by using a UAV, each initial image is corrected to obtain a corrected image, and all corrected images are fused to obtain a fused image; if at least one suspected coking area is detected in the fused image, for any suspected coking area, the corresponding reference area of the any suspected coking area is obtained in each corrected image, the multi-angle difference index of the any suspected coking area is obtained according to the gray value difference of the pixel points in the any suspected coking area and each reference area, the coking confidence degree of the any suspected coking area is obtained according to the gray value variation feature and the gray value difference of the pixel points in the any suspected coking area and the multi-angle difference index of the any suspected coking area, the coking confidence degree of each suspected coking area in the fused image is obtained, the total weighted frequency of each gray value in the fused image is obtained according to the coking confidence degree of each suspected coking area, the total weighted frequency of each gray value in the fused image is used as the frequency of the gray value in the histogram equalization processing, the fused image is enhanced by using histogram equalization to obtain an equalized image of the any to-be-detected part, which is used for coking detection of the any to-be-detected part. Wherein, the coking confidence degree of any suspected coking area is obtained according to the gray value difference of the pixel points in any suspected coking area and each reference area, and the gray value variation feature and the gray value difference of the pixel points in any suspected coking area, so as to distinguish the coking area and other interference areas in the fused image, and then the total weighted frequency of each gray value in the fused image is obtained according to the coking confidence degree of each suspected coking area, so that in the enhancement processing of the fused image by using histogram equalization, the coking area can dominate the cumulative histogram calculation of the fused image, the equalization process focuses on the real coking area, the characteristics of the coking area are clearer, the contrast is more significant, and the equalized image with higher quality is provided, thereby significantly improving the accuracy and stability of coking identification in UAV inspection. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0015] Figure 1 is a flow chart of a boiler coking detection method based on UAV inspection provided by the first embodiment of the present application. DETAILED DESCRIPTION

[0016] Embodiments of the present disclosure are described in detail below with reference to the attached drawing figures, wherein the embodiments of the present disclosure given by way of example are intended to explain the present disclosure, and cannot be understood as limiting the present disclosure.

[0017] It should be noted that the terms "first", "second" and the like in the description of the present disclosure and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the present disclosure.

[0018] In order to illustrate the technical solutions of the present application, the following will be described by specific embodiments.

[0019] The embodiment of the present application provides a kind of boiler coke detection system based on unmanned aerial vehicle inspection, including processor and memory, the computer program of the processor executes the memory to realize a kind of boiler coke detection method based on unmanned aerial vehicle inspection, as shown in Figure 1 The method comprises the following steps:

[0020] Step S101, for any to-be-detected part inside the boiler, an initial image of the any to-be-detected part is obtained at least two shooting angles by using the unmanned aerial vehicle, each initial image is corrected to obtain a corrected image, and all corrected images are fused to obtain a fused image.

[0021] In the long-term operation process of the boiler, the coke phenomenon will be generated in the internal structure of the boiler due to incomplete combustion of fuel, uneven flue gas flow or structural dead angle and other factors. Once the coke is deposited in the heat transfer pipeline, the convection heating surface or the backflow area, not only the heat exchange efficiency is seriously reduced, but also the local overheating, equipment fatigue and even fire hazard may be caused. Therefore, the detection and evaluation of the internal coke state of the boiler is a key means to ensure the safe operation of the equipment and prolong the service life.

[0022] In the prior art, an unmanned aerial vehicle is usually used to continuously shoot at multiple different parts inside the boiler according to a preset path to obtain a covering static image sequence, so as to fully reflect the structural features and details of the image, ensure the complete capture of the coke details, and then redistribute the gray levels of the image through image enhancement technology (such as histogram equalization), so that the image brightness level is more rich and the local contrast is more significant, to provide a higher quality data basis for subsequent image segmentation, edge detection and feature recognition, and to improve the system recognition accuracy and reduce misjudgment and omission.

[0023] Since there are multiple to-be-detected positions (i.e. positions needing to be detected for coke accumulation, such as the dense tube bundle position below the convection heating surface and the back of the furnace water wall tube row) in the boiler, in the embodiment, any to-be-detected position is taken as an example for subsequent analysis of coke accumulation detection.

[0024] Due to the complex lighting conditions inside the boiler, there are shadows, reflections and local dark corners, and a single view angle may not be able to completely capture the shape and position of the coke accumulation. In order to prevent the problem of occlusion and dead angle and reduce the interference of reflections and shadows, in the embodiment, the camera angle of the unmanned aerial vehicle is adjusted around the center of the to-be-detected position by slightly rotating the left and right shooting angles (such as ±5° to 10°) and slightly adjusting the flight height (such as ±10cm to 20cm), which is not limited here and can be set according to the specific implementation scene to obtain an initial image of any to-be-detected position at at least two shooting angles, ensuring that each image has high consistency. One initial image is obtained at each shooting angle. Due to the limited light source inside the boiler, in order to reduce image noise in low light environments, the initial image is denoised by bilateral filtering to avoid interference with subsequent gray scale distribution analysis and enhancement processing. Bilateral filtering is prior art and will not be described here.

[0025] Because the internal environment of the boiler is mainly single-tone and low-light, coke accumulation detection focuses on structural light-dark differences, texture contrast, etc. rather than color information, therefore, a gray scale image can directly reflect the brightness characteristics and is more conducive to subsequent analysis of the contrast difference between the coke accumulation area and the background. Therefore, if the unmanned aerial vehicle uses a color camera for shooting, the initial image needs to be processed to obtain a gray scale image. If an industrial black-and-white camera is used, the initial image does not need to be processed to obtain a gray scale image. Gray scale processing is prior art and will not be described here.

[0026] Due to the limitation of narrow space, airflow interference and structure obstruction when the UAV flies inside the boiler, the initial images are prone to slight blur and jitter. In order to retain the respective clear texture details in each initial image and improve the definition of the image, the initial images need to be fused to obtain a fused image. The method for obtaining the fused image is: (1) extracting the key points with stable texture in each initial image through the SIFT algorithm; (2) matching the key points in each initial image to establish a corresponding relationship, calculating the homography matrix through the matching points, and determining the transformation relationship of one initial image to another initial image; (3) using affine transformation to correct the space of each initial image, so that the pixel points in each initial image are registered to the same coordinate system to obtain a corrected image; (4) using a weighted image pyramid fusion algorithm to fuse all the corrected images (the weighted image pyramid fusion algorithm can fully utilize the high-quality features of different regions in each image, balance the brightness, strengthen the details, and support the suppression of blurred frames or abnormal brightness regions, so the weighted image pyramid fusion algorithm is used to fuse all the corrected images in this embodiment, which is not limited here and can be set according to the specific implementation scene); the SIFT algorithm, key point matching, using affine transformation to correct the space and the weighted image pyramid fusion algorithm belong to the prior art, which will not be described here.

[0027] At this point, N initial images, N corrected images and a fused image of any to-be-detected part are obtained.

[0028] In the process of using histogram equalization for image enhancement in the prior art, the phenomenon of accretion usually occurs in parts of the boiler where heat exchange is uneven, airflow disturbance is strong or cleaning blind area, such as pipe elbow, below the convection heating surface, corners of the furnace, etc. Due to the continuous action of heat stress, humidity and chemical components such as oxygen and sulfide in the flue gas on the local metal area under long-term high-temperature operating conditions, oxidation reaction occurs, and then dark-colored, loose-structured and irregular-textured oxidation deposits (oxidation spots) are formed on the surface. In the gray-scale image, the oxidation spots show similar gray scale and texture characteristics as the accretion. Since histogram equalization gives equal weight to all pixel points in the process of image enhancement, and the cumulative distribution function is calculated based on the frequency to map the original gray value, the oxidation spots and the accretion are enhanced to the same degree, which magnifies the gray scale and texture characteristics of the oxidation spots, and interferes with the accurate identification of the accretion, which may lead to misjudgment in the subsequent identification, reducing the accuracy and reliability of the boiler accretion detection.

[0029] Therefore, the embodiment detects the suspected coking area in the fusion image, acquires the reference area corresponding to the suspected coking area in each correction image, acquires the coking confidence degree of each suspected coking area in the fusion image according to the gray value difference between the suspected coking area and the pixel points in each reference area, the gray value change characteristics and the gray value difference of the pixel points in the suspected coking area, and then acquires the total weighted frequency of each gray value in the fusion image according to the coking confidence degree of each suspected coking area, performs enhancement processing on the fusion image by using histogram equalization to obtain an equalized image, and improves the accuracy of coking detection on any to-be-detected part.

[0030] Since the coking and the oxidation spot usually have the image characteristics of gray value mutation and clear structural boundary, obvious edge contour is presented in the image, and there is a certain gray value gradient difference with the background area. Therefore, in the embodiment, edge detection is first performed on the fusion image by using a Canny edge detection algorithm. The output of the Canny edge detection algorithm is a binary image, white represents an edge, and black represents a background. Then, connected domain analysis is used to extract a connected domain, and the connected white pixel region is marked as a suspected coking area. The Canny edge detection algorithm and the connected domain analysis belong to the prior art, and will not be described here.

[0031] In step S102, if it is detected that there is at least one suspected coking area in the fusion image, for any suspected coking area, the reference area corresponding to the suspected coking area is acquired in each correction image, and the multi-angle difference index of the suspected coking area is acquired according to the gray value difference between the suspected coking area and the pixel points in each reference area.

[0032] If it is detected that there is a suspected coking area in the fusion image, it indicates that the to-be-detected part may have a coking phenomenon or an oxidation spot phenomenon. In order to reduce the interference of the oxidation spot on the coking detection and recognition, improve the accuracy and reliability of the boiler coking detection, and analyze the suspected coking area in the fusion image, it is necessary to judge whether the suspected coking area is a coking area.

[0033] Since the accumulated coke usually exists in the form of accumulation, hardening, and thick protrusion, the surface is rough and porous, and the specular reflection of visible light is weak. Therefore, when the UAV changes the angle of view, the gray value of each pixel point at the same position in each corrected image remains high consistency, and the overall morphology fluctuates low. The oxide spots are mainly composed of oxide layers (such as iron oxide) formed by the oxidation reaction of the metal surface and high-temperature gas and steam. The structure is relatively shallow, and the oxide layer still has a certain specular reflection. Therefore, the oxide spots are more dependent on the visual performance of the light angle, and the gray value of each pixel point at the same position in each corrected image fluctuates greatly. At the same time, the overall morphology shows greater difference than the accumulated coke due to the different incident light of the shooting angle.

[0034] Therefore, for any suspected accumulated coke area, the corresponding reference area of any suspected accumulated coke area is obtained in each corrected image. According to the gray value difference of each pixel point in any suspected accumulated coke area and each reference area, the multi-angle difference index of any suspected accumulated coke area is obtained.

[0035] The method for obtaining the reference area corresponding to any suspected accumulated coke area in each corrected image is as follows:

[0036] After each initial image is corrected, a spatial coordinate system is obtained. The spatial coordinates of each pixel point in the any suspected accumulated coke area are obtained in the spatial coordinate system. The pixel points with the same spatial coordinates in each corrected image are obtained according to the spatial coordinates of each pixel point in the any suspected accumulated coke area, to form the reference area in each corrected image.

[0037] Further, the method for obtaining the multi-angle difference index of any suspected accumulated coke area according to the gray value difference of each pixel point in any suspected accumulated coke area and each reference area is as follows:

[0038] A two-dimensional rectangular coordinate system is constructed with the lower left corner of each corrected image as the origin, the horizontal direction as the abscissa, and the vertical direction as the ordinate. In all corrected images, any one corrected image is selected as a reference image. In this embodiment, the corrected image corresponding to the main view angle (i.e. the corrected image corresponding to the first shooting of the detection part) is selected as the reference image. This is not limited here, and can be set according to the specific implementation scene. For any pixel point in the reference area of the reference image, the gray value of each pixel point with the same coordinate as the any pixel point is obtained in all corrected images to form the gray value sequence of the any pixel point.

[0039] Obtain the gray value sequence of each pixel in the reference area of ​​the reference image, obtain the standard deviation of each gray value sequence, obtain the cumulative standard deviation value, and normalize the cumulative standard deviation value to obtain the gray difference value of any suspected in-focus area.

[0040] A two-dimensional rectangular coordinate system is constructed with the lower left corner of the fused image as the origin, the horizontal direction as the x-coordinate, and the vertical direction as the y-coordinate. Based on the coordinates of the pixels in each reference region and the coordinates of the pixels in any suspected in-focus region, the Hausdorf distance between each reference region and any suspected in-focus region is obtained, and the corresponding Hausdorf distance accumulation value is obtained. The Hausdorf distance accumulation value is normalized to obtain the distance difference value of any suspected in-focus region. The Hausdorf distance is an existing technology and will not be described in detail here.

[0041] The mean between the grayscale difference value and the distance difference value is obtained to obtain the multi-angle difference index of any suspected coking area.

[0042] In one implementation, the first Taking a suspected coking area as an example, the first The formula for calculating the multi-angle difference index of a suspected coking area is as follows:

[0043]

[0044] in, For the first Multiple-angle difference indicators for suspected coking areas; Let be the standard deviation of the i-th grayscale value sequence; n is the number of grayscale value sequences. For the j-th reference region and the j-th reference region The distance from Hausdorf to a suspected area of ​​coke buildup; For the first The j-th reference area of ​​a suspected coking area; To fuse the image in the first There are 1 suspected in-focus areas; N is the number of reference areas (i.e., the number of corrected images); This is the normalization function.

[0045] It should be noted that, For the first Gray-scale difference values ​​of suspected coking areas The larger the value, the higher the value. The greater the fluctuation in grayscale values ​​of pixels in a suspected out-of-focus area across multiple angles, the greater the difference in pixel values ​​in that area under different shooting angles. The larger it is, the more... The larger it is; For the first a distance difference value of the suspected coking area, The greater the distance difference value, the greater the difference in shape and position of the suspected coking area in images taken at different angles. The greater the distance difference value, the greater the difference in shape and position of the suspected coking area in images taken at different angles. The greater the distance difference value, the greater the difference in shape and position of the suspected coking area in images taken at different angles. The greater the distance difference value, the greater the difference in shape and position of the suspected coking area in images taken at different angles. The greater the distance difference value, the greater the difference in shape and position of the suspected coking area in images taken at different angles, and the more likely the suspected coking area is an oxidation spot area.

[0046] Thus, the multi-angle difference index of any suspected coking area is obtained.

[0047] In step S103, the coking confidence degree of the suspected coking area is obtained according to the gray value variation feature and the gray value difference of the pixel points in the suspected coking area and the multi-angle difference index of the suspected coking area.

[0048] Because coking is a coke-like substance formed on the heating surface of the boiler due to incomplete combustion of fuel, accumulation of impurities or obstruction of heat exchange, etc., containing various particulate impurities or cracks, the surface of the coking is uneven and the thickness is uneven, resulting in extremely uneven absorption and reflection of light during shooting. Oxidation spots are a relatively uniform oxide film or oxidation product formed by oxidation of metal materials in a high-temperature environment, usually attached to the metal surface. Therefore, in the final fused image, the gray value of the pixel points in the coking area changes more sharply than that in the oxidation spot area. At the same time, because the inside of the boiler is usually metal or high-temperature layer, the illumination is more uniform under stable light source, so the gray values of the pixel points in the background part of the fused image are more uniform, while the gray values of the pixel points in the coking area are more complex, i.e., the number of gray levels contained in the coking area is significantly higher than that in the oxidation spot area.

[0049] Because the physical process of coking formation and accumulation causes coking particles to accumulate over time, the center part has a longer accumulation time and has a pyramid-like structure, so the gray value of the center part of the coking area is relatively higher than that of the edge area, and there is a gray value change from the center to the edge. The formation of oxidation spots mainly depends on local chemical reaction and temperature conditions, and is less affected by airflow and particle deposition, and does not have such obvious three-dimensional structure as the coking area, and generally has no obvious hierarchical structure as a whole.

[0050] Therefore, the coking confidence degree of any suspected coking area can be obtained according to the gray value variation feature and the gray value difference of the pixel points in the suspected coking area and the multi-angle difference index of the suspected coking area, which is used to determine whether the suspected coking area is a coking area.

[0051] Wherein, according to the gray value variation characteristics and the gray value difference of the pixel points in any suspected coking area, and the multi-angle difference index of any suspected coking area, the method for obtaining the coking confidence degree of any suspected coking area is as follows:

[0052] (1) In the fusion image, all suspected coking areas are removed to obtain a normal area, and according to the gray value variation characteristics of the pixel points in any suspected coking area and the gray distribution characteristics in the normal area, the gray variation degree of any suspected coking area is obtained.

[0053] Specifically, the mean value of the gray values of the pixel points in the any suspected coking area is obtained, the absolute value of the difference between the gray value of each pixel point in the any suspected coking area and the mean value of the gray values is obtained, the mean value of the absolute values of the differences is obtained, the mean value of the absolute values of the differences is normalized to obtain the first gray distribution disorder degree of the any suspected coking area.

[0054] The number of gray levels in the any suspected coking area is obtained, the number of gray levels in the normal area is obtained, the difference between the number of gray levels in the any suspected coking area and the number of gray levels in the normal area is calculated to obtain the gray level difference, and the reciprocal of the gray level difference is substituted into the exponential function with the natural constant as the base to obtain the second gray distribution disorder degree of the any suspected coking area.

[0055] The mean value between the first gray distribution disorder degree and the second gray distribution disorder degree is obtained to obtain the gray variation degree of the any suspected coking area.

[0056] In an embodiment, the first suspected coking area is taken as an example, and the calculation formula of the gray variation degree of the first suspected coking area is as follows:

[0057]

[0058] Wherein, is the gray variation degree of the first suspected coking area; is the gray value of the kth pixel point in the first suspected coking area; m is the number of pixel points in the first suspected coking area; is the number of gray levels in the first suspected coking area; is the number of gray levels in the normal area; is a normalization function; is an exponential function with the natural constant as the base, which is used for inverse proportional normalization. is an absolute value symbol.

[0059] It should be noted that, is the first gray scale distribution disorder degree of the th suspected coking area, the greater, the greater the difference between the gray scale values of each pixel point in the th suspected coking area and the average gray scale value of the th suspected coking area, the greater the gray scale difference between each pixel point in the th suspected coking area, and the more uneven the overall gray scale distribution, the greater, and thus the greater. is the second gray scale distribution disorder degree of the th suspected coking area, the greater, the more diverse the gray scale distribution within the th suspected coking area, reflecting that the surface of the region has complex texture characteristics, and the contrast with the normal region is strong, the greater, and thus the greater. the greater, the greater the gray scale difference between each pixel point in the th suspected coking area, the more uneven the overall gray scale distribution, and the surface of the region has complex texture characteristics, and the contrast with the normal region is strong, and it is more likely to belong to the coking area.

[0060] (2) Obtain the accumulation feature degree of any suspected coking area.

[0061] Specifically, the horizontal coordinates of all pixel points in the any suspected coking area are grouped into a horizontal coordinate set, and the vertical coordinates of all pixel points in the any suspected coking area are grouped into a vertical coordinate set. The mean values of the horizontal coordinate set and the vertical coordinate set are obtained respectively to form a mean coordinate, and the pixel point corresponding to the mean coordinate is recorded as a center pixel point. The eight neighborhoods of the center pixel point are taken as a center region.

[0062] The edge pixel points of the any suspected coking area are obtained by using the Canny edge detection algorithm. The Canny edge detection algorithm is prior art and will not be described here. The average gray scale value of the pixel points in the center region is obtained and recorded as a contrast average. The absolute value of the difference between the gray scale value of each edge pixel point of the any suspected coking area and the contrast average is calculated respectively to obtain a difference absolute value accumulation value. The difference absolute value accumulation value is normalized to obtain the accumulation feature degree of the any suspected coking area.

[0063] In an embodiment, taking the th suspected coking area as an example, the The formula for calculating the degree of accumulation characteristics of a suspected coking area is as follows:

[0064]

[0065] in, For the first The degree of accumulation characteristics in the suspected coking area; To compare the mean; For the first The grayscale value of the z-th edge pixel in the suspected coherent region; t is the grayscale value of the z-th edge pixel; The number of edge pixels in a suspected area of ​​in focus; This is the normalization function; It is the absolute value symbol.

[0066] It should be noted that, The larger the value, the higher the value. The more significant the grayscale change from the center to the edge of a suspected coking area, the more pronounced the gradient change characteristics of that area, and the stronger the overall stacking hierarchy structure. The larger the number, the more... The more likely a suspected area of ​​scorch buildup is to actually be a scorch buildup area.

[0067] (3) Based on the degree of grayscale change, the degree of accumulation characteristics and the multi-angle difference index, obtain the degree of confidence in the coking of any suspected coking area.

[0068] Specifically, the reciprocal of the multi-angle difference index is normalized to obtain the first confidence level of coking in any suspected coking area;

[0069] The average value between the grayscale change level and the stacking feature level is obtained to obtain the second coking confidence level of any suspected coking region.

[0070] The average value between the first coking confidence level and the second coking confidence level is obtained to determine the coking confidence level of any suspected coking region.

[0071] In one implementation, the first Taking a suspected coking area as an example, the first The formula for calculating the confidence level of coking in a suspected coking area is as follows:

[0072]

[0073] in, For the first Confidence level of coking in a suspected coking area; For the first Multiple-angle difference indicators for suspected coking areas; is the first accumulation confidence degree of the first suspected accumulation region, is the gray level change degree of the first suspected accumulation region, is the accumulation feature degree of the first suspected accumulation region, is the second accumulation confidence degree of the first suspected accumulation region, is the normalized function.

[0074] It should be noted that, is the first accumulation confidence degree of the first suspected accumulation region, is the first accumulation confidence degree of the first suspected accumulation region, is the first accumulation confidence degree of the first suspected accumulation region, is the first accumulation confidence degree of the first suspected accumulation region, is the first accumulation confidence degree of the first suspected accumulation region, is the first accumulation confidence degree of the first suspected accumulation region, is the second accumulation confidence degree of the first suspected accumulation region, is the second accumulation confidence degree of the first suspected accumulation region, is the second accumulation confidence degree of the first suspected accumulation region, is the second accumulation confidence degree of the first suspected accumulation region, is the second accumulation confidence degree of the first suspected accumulation region, is the second accumulation confidence degree of the first suspected accumulation region, is the second accumulation confidence degree of the first suspected accumulation region, is the second accumulation confidence degree of the first suspected accumulation region, is the second accumulation confidence degree of the first suspected accumulation region.

[0075] Thus, the accumulation confidence degree of any suspected accumulation region is obtained.

[0076] In step S104, the accumulation confidence degree of each suspected accumulation region in the fusion image is obtained, the total weighted frequency of each gray value in the fusion image is obtained according to the accumulation confidence degree of each suspected accumulation region, the total weighted frequency of each gray value in the fusion image is taken as the frequency of the gray value in the histogram equalization processing, the histogram equalization is used to perform the enhancement processing on the fusion image, and the equalization image of any to-be-detected part is obtained, which is used for the accumulation detection of any to-be-detected part.

[0077] According to the above-mentioned method for obtaining the accumulation confidence degree of any suspected accumulation region, the accumulation confidence degree of each suspected accumulation region in the fusion image is obtained, and then whether each suspected accumulation region is an accumulation region is judged according to the accumulation confidence degree of each suspected accumulation region in the fusion image.

[0078] The method for judging whether each suspected accumulation region is an accumulation region according to the accumulation confidence degree of each suspected accumulation region in the fusion image is as follows:

[0079] The accumulation focus confidence degree threshold is set to 0.7, which is not limited here, and can be set according to the specific implementation scene. The suspected accumulation focus area with the accumulation focus confidence degree greater than 0.7 is recorded as an accumulation focus area, and the suspected accumulation focus area with the accumulation focus confidence degree less than or equal to 0.7 is recorded as an interference area.

[0080] Further, the total weighted frequency of each gray value in the fusion image is obtained according to the accumulation focus confidence degree of each suspected accumulation focus area, so that in the enhancement processing of the fusion image by using histogram equalization, the accumulation focus area can dominate the cumulative histogram calculation of the fusion image, the equalization process is focused on the real accumulation focus area, the features of the accumulation focus area are clearer, the contrast is more significantly improved, and a higher-quality equalization image is provided, thereby significantly improving the accuracy and stability of the accumulation focus recognition in the unmanned aerial vehicle inspection.

[0081] The method for obtaining the total weighted frequency of each gray value in the fusion image according to the accumulation focus confidence degree of each suspected accumulation focus area is as follows:

[0082] (1) The weighted frequency of each gray value in each accumulation focus area is obtained according to the accumulation focus confidence degree of each accumulation focus area.

[0083] For any accumulation focus area, the frequency of each gray value in the accumulation focus area is counted. For any gray value, the addition result of the constant 1 and the accumulation focus confidence degree of the accumulation focus area is obtained, to obtain the accumulation focus weight coefficient of the frequency of the gray value. The product of the frequency of the gray value and the accumulation focus weight coefficient is rounded up to obtain the weighted frequency of the gray value.

[0084] In an embodiment, taking the cth gray value in the bth accumulation focus area as an example, the calculation formula of the weighted frequency of the cth gray value in the bth accumulation focus area is as follows:

[0085]

[0086] wherein, is the weighted frequency of the cth gray value in the bth accumulation focus area; is the frequency of the cth gray value in the bth accumulation focus area; is the accumulation focus confidence degree of the bth accumulation focus area; is the rounding up symbol.

[0087] It should be noted that, is the accumulation focus weight coefficient of the gray value frequency in the bth accumulation focus area, the greater the value is, the greater the possibility that the bth accumulation focus area belongs to the accumulation focus area is, and the more enhancement processing is needed, the greater the value is, and further The greater the more.

[0088] (2) According to the product focus confidence degree of each interference region, the weighted frequency of each gray value in each interference region is obtained.

[0089] For any interference region, the frequency of each gray value in the interference region is counted, for any gray value, the difference between the constant 1 and the product focus confidence degree of the interference region is obtained, to obtain the interference weight coefficient of the frequency of the gray value, and the product of the frequency of the gray value and its interference weight coefficient is rounded down to obtain the weighted frequency of the gray value.

[0090] In an embodiment, taking the e-th gray value in the d-th interference region as an example, the calculation formula of the weighted frequency of the e-th gray value in the d-th interference region is:

[0091]

[0092] Wherein, is the weighted frequency of the e-th gray value in the d-th interference region; is the frequency of the e-th gray value in the d-th interference region; is the product focus confidence degree of the d-th interference region; is the down rounding symbol.

[0093] It should be noted that, is the product focus weight coefficient of the gray value frequency in the d-th interference region, The greater the greater the possibility that the d-th interference region belongs to the oxidation spot region, the less the need for enhancement processing, The smaller the more The smaller the more.

[0094] (3) The frequency of each gray value in the normal region is obtained, and according to the weighted frequency of each gray value in each product focus region, the weighted frequency of each gray value in each interference region and the frequency of each gray value in the normal region, the total weighted frequency of each gray value in the fusion image is obtained.

[0095] Specifically, the normal region, the product focus region and the interference region in the fusion image are all recorded as the region to be counted, and for any gray value in the fusion image, the sum of the weighted frequency of the gray value in each region to be counted is obtained, to obtain the total weighted frequency of any gray value in the fusion image.

[0096] Similarly, the total weighted frequency of each gray value in the fusion image is obtained.

[0097] After obtaining the total weighted frequency of each gray value in the fusion image, the total weighted frequency of each gray value in the fusion image is taken as the frequency of the gray value in the histogram equalization processing, and the fusion image is enhanced by using the histogram equalization, that is, a weighted gray histogram is obtained according to the total weighted frequency of each gray value in the fusion image, a cumulative weighted histogram (CDF) is calculated, and a gray mapping table is obtained according to the cumulative weighted histogram, the fusion image is mapped through the gray mapping table, the gray structure of the focus area is dominant in the mapping function, the contrast of the focus area is enhanced, the focus texture, edge and other features are more clear and easy to identify in the image, the interference of the normal area (i.e. the background part) and the oxidation spot is effectively suppressed, the equalization process is focused on the focus area, the features of the focus area are clearer, the contrast is more significantly improved, and the equalization image of any to-be-detected part with higher quality is obtained, which is used for focus detection of any to-be-detected part, so that the accuracy and stability of the focus recognition in the unmanned aerial vehicle inspection are significantly improved.

[0098] The main purpose of the present application is to obtain the total weighted frequency of each pixel point in the fusion image, so that the key features of the focus are retained and the interference of the oxidation spot is suppressed in the image enhancement process using the total weighted frequency of each pixel point in the fusion image, wherein the enhancement of the fusion image by using the histogram equalization belongs to the prior art, and will not be described here.

[0099] In summary, the embodiment of the present application is aimed at any to-be-detected part in a boiler, and an initial image of the any to-be-detected part is acquired by using a UAV at at least two shooting angles, each initial image is corrected to obtain a corrected image, and all corrected images are fused to obtain a fused image; if at least one suspected coking area is detected in the fused image, for any suspected coking area, a reference area corresponding to the any suspected coking area is acquired in each corrected image, a multi-angle difference index of the any suspected coking area is acquired according to the gray value difference of the pixels in the any suspected coking area and each reference area, the coking confidence degree of the any suspected coking area is acquired according to the gray value change feature and the gray value difference of the pixels in the any suspected coking area and the multi-angle difference index of the any suspected coking area, the coking confidence degree of each suspected coking area in the fused image is acquired, the total weighted frequency of each gray value in the fused image is acquired according to the coking confidence degree of each suspected coking area, the total weighted frequency of each gray value in the fused image is used as the frequency of the gray value in the histogram equalization processing, the fused image is enhanced by using the histogram equalization to obtain an equalized image of the any to-be-detected part, and the equalized image is used for coking detection of the any to-be-detected part. Wherein, the coking confidence degree of the any suspected coking area is acquired according to the gray value difference of the pixels in the any suspected coking area and each reference area, and the gray value change feature and the gray value difference of the pixels in the any suspected coking area, so as to distinguish the coking area and other interference areas in the fused image, then the total weighted frequency of each gray value in the fused image is acquired according to the coking confidence degree of each suspected coking area, so that in the enhancement processing of the fused image by using the histogram equalization, the coking area can dominate the cumulative histogram calculation of the fused image, the equalization process focuses on the real coking area, the characteristics of the coking area are clearer, the contrast is more significant, and the equalized image with higher quality is provided, thereby the accuracy and stability of coking identification in UAV inspection are significantly improved.

[0100] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A boiler coke accumulation detection system based on UAV inspection, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it performs the following steps: For any part inside the boiler to be inspected, an initial image of the part to be inspected is acquired using a drone from at least two shooting angles. Each initial image is then corrected to obtain a corrected image. All corrected images are then fused to obtain a fused image. If at least one suspected in-focus area is detected in the fused image, then for any suspected in-focus area, a reference area corresponding to the suspected in-focus area is obtained in each corrected image, and a multi-angle difference index of the suspected in-focus area is obtained based on the difference in gray value of the pixels in the suspected in-focus area and each reference area. Based on the gray value change characteristics and gray value differences of pixels in any suspected in-focus area, as well as the multi-angle difference index of any suspected in-focus area, the confidence level of in-focus in any suspected in-focus area is obtained. The confidence level of focus in each suspected focus region in the fused image is obtained. Based on the confidence level of focus in each suspected focus region, the total weighted frequency of each gray value in the fused image is obtained. The total weighted frequency of each gray value in the fused image is used as the frequency of gray values ​​in histogram equalization. Histogram equalization is used to enhance the fused image to obtain an equalized image of any region to be detected, which is used to detect focus in any region to be detected.

2. The boiler coke accumulation detection system based on UAV inspection according to claim 1, characterized in that, The step of obtaining a multi-angle difference index for any suspected in-focus area based on the difference in grayscale values ​​of pixels in any suspected in-focus area and each reference area includes: A two-dimensional rectangular coordinate system is constructed with the lower left corner of each corrected image as the origin, the horizontal direction as the x-coordinate, and the vertical direction as the y-coordinate. One corrected image is randomly selected from all corrected images as the reference image. For any pixel in the reference area of ​​the reference image, the gray value of each pixel with the same coordinates as the pixel is obtained in all corrected images, forming a gray value sequence of the pixel. Obtain the grayscale value sequence of each pixel in the reference region of the benchmark image. Based on the data fluctuation characteristics of the grayscale value sequence of each pixel in the reference region of the benchmark image, and the distance difference between each reference region and any suspected in-focus region, obtain the multi-angle difference index of any suspected in-focus region.

3. The boiler coke accumulation detection system based on UAV inspection according to claim 2, characterized in that, The step of obtaining a multi-angle difference index for any suspected in-focus region based on the data fluctuation characteristics of the gray value sequence of each pixel in the reference region of the benchmark image, and the distance difference between each reference region and any suspected in-focus region, includes: The standard deviation of each gray value sequence is obtained, and the cumulative standard deviation value is obtained. The cumulative standard deviation value is normalized to obtain the gray difference value of any suspected coking area. A two-dimensional rectangular coordinate system is constructed with the lower left corner of the fused image as the origin, the horizontal direction as the abscissa, and the vertical direction as the ordinate. Based on the coordinates of the pixels in each reference region and the coordinates of the pixels in any suspected in-focus region, the Hausdorf distance between each reference region and any suspected in-focus region is obtained, and the Hausdorf distance accumulation value is obtained accordingly. The Hausdorf distance accumulation value is normalized to obtain the distance difference value of any suspected in-focus region. The mean between the grayscale difference value and the distance difference value is obtained to obtain the multi-angle difference index of any suspected coking area.

4. The boiler coke accumulation detection system based on UAV inspection according to claim 1, characterized in that, The step of obtaining the confidence level of focus in any suspected focus region based on the grayscale value change characteristics and grayscale value differences of pixels in any suspected focus region, as well as the multi-angle difference index of any suspected focus region, includes: In the fused image, all suspected in-focus areas are removed to obtain normal areas. Based on the gray value change characteristics of pixels in any suspected in-focus area and the gray value distribution characteristics of any suspected in-focus area and the normal area, the degree of gray value change of any suspected in-focus area is obtained. In any suspected coagulation region, a central region is obtained, and edge pixels of the suspected coagulation region are obtained using the Canny edge detection algorithm. Based on the difference in grayscale values ​​between the edge pixels of the suspected coagulation region and the pixels in the central region, the degree of stacking features of the suspected coagulation region is obtained. Based on the degree of grayscale change, the degree of accumulation characteristics, and the multi-angle difference index, the confidence level of coking in any suspected coking area is obtained.

5. A boiler coke accumulation detection system based on UAV inspection according to claim 4, characterized in that, The step of obtaining the degree of grayscale change in any suspected in-focus region based on the grayscale value change characteristics of pixels in any suspected in-focus region and the grayscale distribution characteristics of the suspected in-focus region and the normal region includes: The average grayscale value of the pixels in any suspected coma region is obtained. The absolute value of the difference between the grayscale value of each pixel in any suspected coma region and the average grayscale value is obtained respectively. The average absolute value of the difference is obtained accordingly. The average absolute value of the difference is normalized to obtain the first grayscale distribution disorder of any suspected coma region. Obtain the number of gray levels in any suspected coking area, obtain the number of gray levels in the normal area, calculate the difference between the number of gray levels in any suspected coking area and the number of gray levels in the normal area, obtain the gray level difference value, substitute the negative of the gray level difference value into an exponential function with the natural constant as the base, and obtain the second gray level distribution disorder degree of any suspected coking area. The mean between the first degree of grayscale distribution disorder and the second degree of grayscale distribution disorder is obtained to determine the degree of grayscale change in any suspected coking area.

6. A boiler coke accumulation detection system based on UAV inspection according to claim 4, characterized in that, The step of obtaining the degree of accumulation feature of any suspected coking region based on the difference in grayscale values ​​between the edge pixels of any suspected coking region and the pixels in the central region includes: The average grayscale value of the pixels in the central region is obtained and recorded as the comparison mean. The absolute value of the difference between the grayscale value of each edge pixel in any suspected coking region and the comparison mean is calculated. The corresponding cumulative value of the absolute value of the difference is obtained. The cumulative value of the absolute value of the difference is normalized to obtain the degree of accumulation feature of any suspected coking region.

7. A boiler coke accumulation detection system based on UAV inspection according to claim 4, characterized in that, The step of obtaining the confidence level of coking in any suspected coking area based on the degree of grayscale change, the degree of accumulation characteristics, and the multi-angle difference index includes: The reciprocal of the multi-angle difference index is normalized to obtain the first coking confidence level of any suspected coking area; The average value between the grayscale change level and the stacking feature level is obtained to obtain the second coking confidence level of any suspected coking region. The average value between the first coking confidence level and the second coking confidence level is obtained to determine the coking confidence level of any suspected coking region.

8. A boiler coke accumulation detection system based on UAV inspection according to claim 4, characterized in that, The step of obtaining the central region in any of the suspected coking areas includes: The x-coordinates of all pixels in any suspected focal region are combined into an x-coordinate set, and the y-coordinates of all pixels in any suspected focal region are combined into a y-coordinate set. The mean values ​​of the x-coordinate set and the y-coordinate set are obtained respectively to form a mean coordinate. The pixel corresponding to the mean coordinate is recorded as the center pixel. The eight neighbors of the center pixel are taken as the center region.

9. A boiler coke accumulation detection system based on UAV inspection according to claim 4, characterized in that, The step of obtaining the total weighted frequency of each grayscale value in the fused image based on the confidence level of the focus in each suspected focus region includes: Set a confidence threshold for focus accumulation. Areas with a focus accumulation confidence level greater than the confidence threshold are recorded as focus accumulation areas, and areas with a focus accumulation confidence level less than or equal to the confidence threshold are recorded as interference areas. The weighted frequency of each gray value in each coking region is obtained based on the confidence level of coking in each coking region. The weighted frequency of each gray value in each interfering region is obtained based on the confidence level of coking in each interfering region. The frequency of each gray value in the normal region is then obtained. The total weighted frequency of each gray value in the fused image is obtained based on the weighted frequency of each gray value in each in-focus region, the weighted frequency of each gray value in each interference region, and the frequency of each gray value in the normal region.

10. A boiler coke accumulation detection system based on UAV inspection according to claim 9, characterized in that, The step of obtaining the weighted frequency of each grayscale value in each forked region based on the forked confidence level of each forked region includes: For any forked region, the frequency of each gray value in the forked region is counted. For any gray value, the sum of the constant 1 and the forked confidence level of the forked region is obtained to get the forked weight coefficient of the frequency of the gray value. The product of the frequency of the gray value and the forked weight coefficient is rounded up to get the weighted frequency of the gray value.

11. A boiler coke accumulation detection system based on UAV inspection according to claim 9, characterized in that, The step of obtaining the weighted frequency of each grayscale value in each interference region based on the confidence level of the focus in each interference region includes: For any interference region, the frequency of each gray value in the interference region is counted. For any gray value, the difference between the constant 1 and the focus confidence level of the interference region is obtained to obtain the interference weight coefficient of the frequency of the gray value. The product of the frequency of the gray value and its interference weight coefficient is rounded down to obtain the weighted frequency of the gray value.

12. A boiler coke accumulation detection system based on UAV inspection according to claim 9, characterized in that, The step of obtaining the total weighted frequency of each gray value in the fused image based on the weighted frequency of each gray value in each focused region, the weighted frequency of each gray value in each interference region, and the frequency of each gray value in the normal region includes: The normal region, the in-focus region, and the interference region in the fused image are all recorded as regions to be counted. For any gray value in the fused image, the sum of the weighted frequencies of any gray value in each region to be counted is obtained to obtain the total weighted frequency of any gray value in the fused image.

13. A boiler coke accumulation detection system based on UAV inspection according to claim 1, characterized in that, For any suspected area of ​​incongruity, the step of obtaining the reference region corresponding to that suspected area of ​​incongruity in each corrected image includes: After each initial image is corrected, a spatial coordinate system is obtained. The spatial coordinates of each pixel in any suspected in-focus area are obtained in the spatial coordinate system. Based on the spatial coordinates of each pixel in any suspected in-focus area, pixels with the same spatial coordinates are obtained in each corrected image to form a reference area in each corrected image.

Images