An automated detection method and system for bridge diseases and defects

Abstract

The present invention relates to the technical field of image data processing, and provides a method and system for automatic detection of bridge disease defects, including: acquiring a bridge image; obtaining a bridge disease defect index according to the characteristics of the disease defects in the bridge image; obtaining a defect fusion gain index according to the bridge disease defect index; obtaining a fusion weight according to the defect fusion gain index; obtaining a fusion pixel value according to the fusion weight; obtaining a bridge fusion image according to the fusion pixel value; and obtaining a bridge disease defect detection result according to the bridge fusion image. The present invention uses the method of image fusion to highlight the bridge disease defect area and improves the accuracy of bridge disease defect detection.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and particularly to an automatic detection method and system for bridge disease defects. Background Art

[0002] Bridges are an important part of transportation infrastructure, responsible for bearing the weight of vehicles and pedestrians. As the operation time of bridges increases, honeycombing, pitting, damage and exposed reinforcement, waterproof layer damage, water seepage, cracks and other damages are likely to appear on the surface of their structures. If not treated in time, it will affect the service life and bearing capacity of the bridges. Therefore, it is necessary to regularly detect the disease defects of bridges, timely discover potential safety hazards, and then take measures to reinforce or repair the bridges to ensure their safety.

[0003] With the continuous development of computer vision, the bridge defect detection technology based on computer vision has gradually matured. Due to the characteristics of non-contact and high efficiency of computer vision, it has gradually become an important method for bridge disease defect detection. Traditional algorithms for detecting bridge disease defects based on computer vision, such as the weighted average fusion algorithm, are simple and easy to implement, suitable for fast fusion tasks, but the calculation of weights lacks certain adaptability, and may not be able to well fuse the edge detail information of images, resulting in low accuracy when detecting bridge disease defects. Summary of the Invention

[0004] The present invention provides an automatic detection method and system for bridge disease defects to solve the problem of low accuracy in bridge disease defect detection. The specific technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present invention provides an automatic detection method for bridge disease defects, and the method includes the following steps:

[0006] Obtain three bridge images, where the three bridge images include a bridge RGB image, a bridge infrared image, and a bridge grayscale image;

[0007] Obtain an RGB disease observation window, a grayscale disease observation window, and an infrared disease observation window for each pixel point according to the three bridge images. Obtain the entropy of all pixel points in four directions in the grayscale disease observation window of each pixel point and each jump sub-sequence of the grayscale disease observation window of each pixel point according to the grayscale disease observation window of each pixel point. Obtain the bridge disease defect index of each pixel point according to the entropy of all pixel points in four directions in the grayscale disease observation window of each pixel point and all jump sub-sequences of the grayscale disease observation window of each pixel point. Obtain the defect fusion gain index of each pixel point in the bridge RGB image, the bridge infrared image, and the bridge grayscale image respectively according to the bridge disease defect index of each pixel point.

[0008] Obtain the fusion weight of each pixel in the bridge RGB image, bridge infrared image, and bridge grayscale image according to the defect fusion gain index of each pixel in the bridge RGB image, bridge infrared image, and bridge grayscale image; obtain the fusion pixel value of each pixel according to the fusion weight of each pixel in the bridge RGB image, bridge infrared image, and bridge grayscale image;

[0009] Obtain the bridge fusion image according to the fusion pixel values of all pixels, and use the bridge fusion image to obtain the bridge disease defect detection result.

[0010] Preferably, the method for obtaining the RGB disease observation window, grayscale disease observation window, and infrared disease observation window of each pixel according to the three bridge images, and obtaining the entropy of all pixels in the grayscale disease observation window of each pixel in four directions and each jump subsequence of the grayscale disease observation window of each pixel is as follows:

[0011] For each pixel in each bridge image, use the window with a preset size centered on the pixel as the disease observation window of the pixel; use the disease observation windows of each pixel in the bridge RGB image, bridge grayscale image, and bridge infrared image as the RGB disease observation window, grayscale disease observation window, and infrared disease observation window of each pixel respectively;

[0012] For the grayscale disease observation window of each pixel, use the grayscale matrix composed of all grayscale values in the grayscale disease observation window as the input of the gray-level co-occurrence matrix algorithm, and use the output of the gray-level co-occurrence matrix algorithm as the entropy of all pixels in the grayscale disease observation window in four directions, and the four directions include 0 degrees, 45 degrees, 90 degrees, and 135 degrees;

[0013] Use the pixel at the center of the grayscale disease observation window as the starting point, and use the sequence formed by sorting the grayscale values of all pixels in the grayscale disease observation window according to the preset rule starting from the starting point as the one-dimensional expanded grayscale sequence of the grayscale disease observation window, and the preset rule is the clockwise direction and the order of layer-by-layer arrangement;

[0014] For each element value in the one-dimensional expanded grayscale sequence of the grayscale disease observation window, if the element value is greater than or equal to the grayscale value of the pixel at the center of the grayscale disease observation window, update the element value to 1, and if the element value is less than the grayscale value of the pixel at the center of the grayscale disease observation window, update the element value to 0;

[0015] Use the result after updating all element values in the one-dimensional expanded grayscale sequence as the one-dimensional expanded binary sequence of the grayscale disease observation window; take the first element value in the one-dimensional expanded binary sequence as the first jump point, take each element value in the one-dimensional expanded binary sequence except the first element value as the target element value, and if the target element value is different from the previous element value, take the target element value as the jump point;

[0016] Take the sequence composed of the elements between every two adjacent jump points in the one-dimensional expanded binary sequence as each jump subsequence of the grayscale disease observation window.

[0017] Preferably, the method for obtaining the bridge disease defect index of each pixel point according to the entropy of all pixel points in the four directions in the grayscale disease observation window of each pixel point and all jump subsequences of the grayscale disease observation window of each pixel point is as follows:

[0018]

[0019]

[0020]

[0021] In the formula, I ac represents the color variation index of the c-th pixel point in the RGB disease observation window of pixel point a, L represents the number of channels in the color space of the bridge RGB image A, Sig() represents the Sigmoid function, represents the value of the c-th pixel point in the d-th channel of the RGB disease observation window of pixel point a, represents the mean value of all pixel points in the d-th channel of the RGB disease observation window of pixel point a; R a represents the temperature variation index of the infrared disease observation window of pixel point a, P a represents the peak signal-to-noise ratio of all pixel values of the infrared disease observation window of pixel point a, m 2 represents the number of pixel points in the infrared disease observation window and the grayscale disease observation window, r 1c r 2c respectively represent the number of pixel points whose pixel values in the eight-neighborhood of the c-th pixel point in the infrared disease observation window of pixel point a are less than and greater than or equal to the pixel value of pixel point a; D a represents the bridge disease defect index of pixel point a, represents the minimum value of the entropy of all pixel points in the four directions in the grayscale disease observation window of pixel point a, f c represents the sequence length of the jump subsequence where the c-th element in the one-dimensional expanded binary sequence of the grayscale disease observation window of pixel point a is located.

[0022] Preferably, the method for obtaining the defect fusion gain index of each pixel in the bridge RGB image, bridge infrared image, and bridge grayscale image according to the bridge disease defect index of each pixel is as follows:

[0023] Obtain the defect fusion gain index of each pixel in the bridge RGB image according to the bridge disease defect index of each pixel;

[0024] Obtain the defect fusion gain index of each pixel in the bridge infrared image according to the bridge disease defect index of each pixel;

[0025] For each pixel in the bridge grayscale image, calculate the absolute value of the difference between the grayscale value of the pixel and the grayscale values of each pixel in the eight-neighborhood of the pixel, and take the product of the sum of the absolute values accumulated over the eight-neighborhood of the pixel and the bridge disease defect index of the pixel as the defect fusion gain index of the pixel.

[0026] Preferably, the method for obtaining the defect fusion gain index of each pixel in the bridge RGB image according to the bridge disease defect index of each pixel is as follows:

[0027] For each pixel in the bridge RGB image, calculate the absolute value of the difference between the value of each channel of the pixel and the values of each channel of each pixel in the eight-neighborhood of the pixel, calculate the mean value of the sum of the absolute values accumulated over all channels in the color space, and take the product of the sum of the mean values accumulated over the eight-neighborhood of the pixel and the bridge disease defect index of the pixel as the defect fusion gain index of the pixel.

[0028] Preferably, the method for obtaining the defect fusion gain index of each pixel in the bridge infrared image according to the bridge disease defect index of each pixel is as follows:

[0029] For each pixel in the bridge infrared image, calculate the absolute value of the difference between the pixel value of the pixel and the pixel values of each pixel in the eight-neighborhood of the pixel, and take the product of the sum of the absolute values accumulated over the eight-neighborhood of the pixel and the bridge disease defect index of the pixel as the defect fusion gain index of the pixel.

[0030] Preferably, the method for obtaining the fusion weight of each pixel in the bridge RGB image, bridge infrared image, and bridge grayscale image according to the defect fusion gain index of each pixel in the bridge RGB image, bridge infrared image, and bridge grayscale image is as follows:

[0031] For each pixel in each bridge image of the bridge RGB image, bridge infrared image, and bridge grayscale image, take the product of the information entropy of the pixel values of all pixels in the eight-neighborhood of the pixel and the defect fusion gain index of the pixel as the fusion weight of the pixel;

[0032] The fusion weights of the pixel points include the fusion weights of each pixel point in the bridge RGB image, the fusion weights of each pixel point in the bridge infrared image, and the fusion weights of each pixel point in the bridge grayscale image.

[0033] Preferably, the method for obtaining the fusion pixel value of each pixel point according to the fusion weights of each pixel point in the bridge RGB image, the bridge infrared image, and the bridge grayscale image is as follows:

[0034] Perform normalization processing on the fusion weights of each pixel point in the bridge RGB image, the bridge infrared image, and the bridge grayscale image respectively to obtain the normalized fusion weights of each pixel point in the bridge RGB image, the bridge infrared image, and the bridge grayscale image;

[0035] For each pixel point in the bridge RGB image, take the product of the mean value of the pixel point on the three channels and the normalized fusion weight of the pixel point as the first component factor of the pixel point;

[0036] For each pixel point in the bridge grayscale image, take the product of the grayscale value of the pixel point and the normalized fusion weight of the pixel point as the second component factor of the pixel point;

[0037] For each pixel point in the bridge infrared image, take the product of the pixel value of the pixel point and the normalized fusion weight of the pixel point as the third component factor of the pixel point;

[0038] Take the product of the first component factor, the second component factor, and the third component factor of the pixel point as the fusion pixel value of the pixel point.

[0039] Preferably, the specific method for obtaining the bridge fusion image according to the fusion pixel values of all pixel points and obtaining the bridge disease defect detection result by using the bridge fusion image is as follows:

[0040] Replace the grayscale value of each pixel point in the bridge grayscale image with the fusion pixel value of each pixel point, and take the result after replacing all pixel points in the bridge grayscale image as the bridge fusion image;

[0041] Take the bridge fusion image as the input of the edge detection algorithm, take the output of the edge detection algorithm as the edge image of the bridge fusion image, take the area with irregular edge shapes in the edge image as the bridge disease defect area, and take the bridge disease defect area as the bridge disease defect detection result.

[0042] In a second aspect, an embodiment of the present invention further provides a bridge disease defect automatic detection system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned bridge disease defect automatic detection methods are implemented.

[0043] The beneficial effects of the present invention are as follows: By analyzing the color and texture change characteristics of visible light images and infrared images when bridge diseases and defects occur, and combining them to construct a bridge disease and defect index, the pixel value of the pixel point at the same position in the bridge image reflects the degree of the disease and defect characteristics of the pixel point; Based on the bridge disease and defect index and combined with the obviousness of the edge features of each image, a defect fusion gain index is constructed to reflect the degree of the disease and defect or edge features of each pixel point in each image; Based on the defect fusion gain index, the fusion weight of each pixel point in each image is constructed and normalized. The fusion weight reflects the contribution degree of each pixel point in each image to the richness of the edge or disease and defect. Therefore, taking it as the weight of the weighted average fusion algorithm can improve the fusion accuracy of the weighted average fusion algorithm. When fusing the bridge RGB image, the bridge grayscale image, and the bridge infrared image, the edges and disease and defect parts in the fused image are clearer, and then the edge disease and defect parts can be extracted more accurately, and the bridge diseases can be automatically detected accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0045] Figure 1 It is a schematic flowchart of a method for automatically detecting bridge diseases and defects provided by an embodiment of the present invention;

[0046] Figure 2 It is a schematic diagram of the one-dimensional expansion gray sequence sorting provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0048] Please refer to Figure 1 , which shows a flowchart of a method for automatically detecting bridge diseases and defects provided by an embodiment of the present invention. The method includes the following steps:

[0049] Step S001: Obtain three bridge images and preprocess the bridge images.

[0050] Collect bridge images through a CMOS high-definition camera installed on a drone. The obtained image type is an RGB space image. Convert the obtained RGB space image into a grayscale image through the grayscale averaging method. Collect the infrared image of the bridge through an infrared sensor installed on the drone. The grayscale averaging method is a well-known technology, and the specific process will not be elaborated here. Since noise may be generated during the image acquisition process due to environmental interference and other situations, affecting the image quality and subsequent analysis results, the present invention performs denoising processing on the obtained images. Common denoising methods include Gaussian filtering denoising, bilateral filtering denoising, mean filtering denoising, etc. To retain as much edge detail information in the image as possible, the present invention uses the bilateral filtering denoising technology to perform denoising processing on the image. The bilateral filtering denoising is a well-known technology, and the specific process will not be elaborated here.

[0051] So far, three preprocessed bridge images are obtained, which are respectively denoted as bridge RGB image A, bridge infrared image C, and bridge grayscale image H.

[0052] Step S002: Obtain the bridge disease defect index according to the characteristics of the disease defects in the bridge image, and obtain the defect fusion gain index according to the bridge disease defect index.

[0053] When the bridge has disease defects, such as water seepage, waterproof layer damage, steel bar exposure, etc., it will cause irregular changes in the surface color, texture, etc. of the bridge, and this change can be observed through the bridge RGB image A and the bridge grayscale image H; at the same time, when the bridge has disease defects, such as structural deformation and material damage in the bearing structure, support or expansion joint of the bridge, it is easy to cause friction and generate frictional heat, and this phenomenon can be observed through the bridge infrared image C.

[0054] Through the above analysis, it can be seen that different types of disease defects in the bridge are more obvious in different images. Based on this, a bridge disease defect index can be constructed based on the distribution of pixel values of each pixel point in the bridge RGB image A, the bridge grayscale image H, and the bridge infrared image C, comprehensively reflecting the possibility that the pixel point belongs to a defect pixel point. The construction process is as follows:

[0055] Specifically, taking the pixel point a in the bridge RGB image A as an example, the bridge grayscale image H and the bridge infrared image C are processed in the same way. Denote the window with a size of m×m centered on the pixel point a as the disease observation window of the pixel point a. Denote the disease observation windows of the pixel point in the bridge RGB image A, the bridge grayscale image H, and the bridge infrared image C as the RGB disease observation window, the grayscale disease observation window, and the infrared disease observation window respectively. The empirical value of m is 11.

[0056] For the grayscale disease observation window of pixel point a, the grayscale matrix composed of all grayscale values within the grayscale disease observation window is used as the input of the gray-level co-occurrence matrix algorithm, and the output of the gray-level co-occurrence matrix algorithm is used as the entropy of all pixel points within the grayscale disease observation window in four directions, including 0 degrees, 45 degrees, 90 degrees, and 135 degrees. The gray-level co-occurrence matrix algorithm is a well-known technology, and the specific process will not be elaborated here.

[0057] Further, in the grayscale disease observation window of pixel point a, starting from pixel point a (i.e., the central pixel point of the grayscale disease observation window), sort in the order of clockwise and layer-by-layer arrangement, and record the sequence composed of the grayscale values of each sorted pixel point as the one-dimensional expanded grayscale sequence. For example, taking a 5×5 window as an example, the schematic diagram of the one-dimensional expanded grayscale sequence sorting is as Figure 2 shown.

[0058] Compare each element value in the one-dimensional expanded grayscale sequence with the grayscale value of pixel point a (i.e., the value of the first element in the grayscale sequence). If the element value is greater than or equal to the grayscale value of pixel point a, update the element value to 1; otherwise, if the element value is less than the grayscale value of pixel point a, update the element value to 0. Update each element value in the one-dimensional expanded grayscale sequence, and record the result after updating all element values in the one-dimensional expanded grayscale sequence as the one-dimensional expanded binary sequence.

[0059] Record the position where the element value in the one-dimensional expanded binary sequence is different from the previous element value as the jump point. In particular, define the first position in the sequence as the jump point. Starting from each jump point and ending at the next jump point, record the sequence composed of the intermediate elements as each jump subsequence.

[0060] Calculate the bridge disease defect index of each pixel point:

[0061]

[0062]

[0063]

[0064] In the formula, Iac represents the color variation index of the c-th pixel point in the RGB disease observation window of pixel point a, L represents the number of channels in the color space of the bridge RGB image A. In the present invention, the empirical value 3 is taken, d taking 1, 2, 3 respectively represents the R, G, B channels, Sig() represents the Sigmoid function, represents the value of the c-th pixel point in the RGB disease observation window of pixel point a in the d-th channel, represents the mean value of all pixel points in the RGB disease observation window of pixel point a in the d-th channel; Ra represents the temperature variation index of the infrared disease observation window of pixel point a, Pa represents the peak signal-to-noise ratio of all pixel points in the infrared disease observation window of pixel point a, and m 2 respectively represent the number of pixel points in the infrared disease observation window and the grayscale disease observation window. r1c and r2c respectively represent the number of pixel points whose pixel values in the eight-neighborhood of the c-th pixel point in the infrared disease observation window of pixel point a are less than, greater than, or equal to the pixel value of pixel point a. The calculation of the peak signal-to-noise ratio is a well-known technology, and the specific process will not be elaborated here; Da represents the bridge disease defect index of pixel point a, represents the minimum value of the entropy of all pixel points in the grayscale disease observation window of pixel point a in four directions. fc represents the sequence length of the jump subsequence where the c-th element in the one-dimensional expanded binary sequence of the grayscale disease observation window of pixel point a is located.

[0065] When the difference between the c-th pixel point in the RGB disease observation window of pixel point a and the channel values of the pixel points in this window is larger, that is the farther away from 0, the more obvious the contrast between the c-th pixel point and the remaining pixel points in the window. Since when there are disease defects such as cracks in the bridge, the color at this position is quite different from the color of the surrounding normal defect-free area, so this pixel point is more likely to be a defective pixel point. The calculated color variation index of this pixel point is larger, and through the Sigmoid function, it is mapped to the range of (0,1) without affecting its size relationship to avoid the final calculation result from being too large. At the same time, the larger the peak signal-to-noise ratio of all pixel points in the infrared disease observation window, the clearer the feature of local temperature anomaly, that is, the larger the temperature variation index. At the same time, the smaller the length of the jump subsequence in the one-dimensional expanded binary sequence obtained after processing the grayscale disease observation window of pixel point a, that is, the smaller fc, the more frequent the alternation of grayscale values in this window, indicating that there are more likely to be disease defects such as wear and honeycombing on the bridge surface. At the same time, when the minimum value of the grayscale co-occurrence matrix entropy in four directions in the grayscale disease observation window is larger, that is larger, the more chaotic the distribution of pixel grayscale values in this window, indicating that there are more likely to be disease defects with irregular textures such as rust and waterproof layer damage on the bridge surface, and pixel point a is more likely to belong to defective pixel points. Therefore, the calculated bridge disease defect index of this pixel point is larger.

[0066] Furthermore, when there are disease defects such as deformation of the bridge structure and holes in the concrete, when heat passes through the bridge building materials, it may cause local heat conduction obstacles, resulting in certain changes in the temperature of the area around the disease defects, which can then be observed through infrared images. However, due to the problems of low resolution, low contrast, and low signal-to-noise ratio in infrared images, when using infrared images to detect bridge disease defects, the detection effect of relatively fine surface cracks in the bridge is poor. At the same time, since infrared images judge bridge disease defects based on the thermal radiation intensity of different temperature regions, the detection effect of disease defects of non-thermally sensitive materials in the bridge through infrared images may be poor; the pixel points in RGB images contain color information of three channels, red, green, and blue, which can provide richer visual information. For obvious color features such as water seepage and waterproof layer peeling in the bridge, such defects can be better identified through the bridge RGB images, but they are more sensitive to changes in illumination and may have certain misjudgment phenomena, resulting in misjudgment of defects of regional types; in grayscale images, there is only one channel, which can capture detailed information on surface deformation, such as deformation and small cracks on the bridge surface, but some color and texture information may be lost.

[0067] Based on the above analysis, since the bridge disease defect index of each pixel point obtained in the above steps comprehensively reflects the degree to which the pixel point belongs to a defect in combination with the characteristics of each image, the defects of the bridge RGB image A, the bridge grayscale image H, and the bridge infrared image C can be enhanced in contrast based on the bridge disease defect index of each pixel point, making the defect features in each image more obvious, and thus achieving a better fusion effect when fusing each image subsequently.

[0068] Furthermore, calculate the defect fusion gain index of each pixel point in the bridge RGB image A, the bridge grayscale image H, and the bridge infrared image C respectively:

[0069]

[0070]

[0071]

[0072] In the formula, J a represents the defect fusion gain index of pixel point a in the bridge RGB image A, D a represents the bridge disease defect index of pixel point a, L represents the number of channels in the color space of the bridge RGB image A, and the empirical value 3 is taken in the present invention. h takes 1, 2, and 3 respectively to represent the R, G, and B channels. n represents the number of neighboring pixel points of pixel point a, and the empirical value 8 is taken in the present invention. A ah represents the value of the h-th channel of pixel point a in the bridge RGB image. Denote the value of the g-th pixel in the eight-neighborhood of pixel point a in the bridge RGB image at the h-th channel; K a Denote the defect fusion gain index of pixel point a in the bridge grayscale image H, H a Denote the grayscale value of pixel point a in the bridge grayscale image H Denote the grayscale value of the g-th pixel in the eight-neighborhood of pixel point a in the bridge grayscale image H; M a Denote the defect fusion gain index of pixel point a in the bridge infrared image C, C a Denote the pixel value of pixel point a in the bridge infrared image C Denote the pixel value of the g-th pixel in the eight-neighborhood of pixel point a in the bridge infrared image C

[0073] In the bridge RGB image A, the bridge grayscale image H, and the bridge infrared image C, if the difference in pixel values between pixel point a and its neighborhood pixels is greater, that is and are greater, it indicates that the pixel point at this position in each image has more obvious edge features. To make the edge details in the fused bridge image clearer and facilitate the extraction of edges for defect recognition, it is necessary to enhance the contrast of pixel point a in each image. Therefore, the larger the calculated defect fusion gain index of each image, and at the same time, if the bridge disease defect index of pixel point a is larger, that is D a is greater, it indicates that the pixel point at this position in each image has disease defect features such as water seepage, waterproof layer rupture, cracks, etc. To increase the difference between the defect part and the rest of the fused bridge image, the pixel value of this pixel point in the fused bridge image is required. Therefore, the larger the calculated defect fusion gain index of each image

[0074] According to the above steps, the defect fusion gain index of each pixel point in the bridge RGB image A, the bridge grayscale image H, and the bridge infrared image C can be calculated

[0075] So far, the defect fusion gain index of each pixel point in the bridge RGB image A, the bridge grayscale image H, and the bridge infrared image C has been obtained respectively

[0076] Step S003, obtain the fusion weight according to the defect fusion gain index, obtain the fusion pixel value according to the fusion weight, and obtain the bridge fusion image according to the fusion pixel value

[0077] The defect fusion gain index of each pixel point in each image obtained through the above steps reflects the importance of each pixel point when fusing each pixel point in each image. Based on this, the fusion weight of each image when fusing the images can be calculated, and its calculation formula is as follows

[0078] w ia = SE ia ×Oia

[0079] In the formula, w ia represents the fusion weight of pixel point a in the i-th image. When i takes 1, 2, and 3 respectively, it represents the bridge RGB image A, the bridge grayscale image H, and the bridge infrared image C. SE ia represents the information entropy of all pixel values within the eight-neighborhood of pixel point a in the i-th image. O ia represents the defect fusion gain index of pixel point a in the i-th type of image.

[0080] Among the bridge RGB image A, the bridge grayscale image H, and the bridge infrared image C, the more obvious the edge and defect features of pixel point a are in one of the images, that is, O ia is larger. At the same time, if the information entropy of the pixel values in the local area of the image is larger, that is, SE ia is larger, it indicates that the overall detailed complex information in this image is more, and there are more likely to be more irregular edges, etc. That is, the bridge is more likely to have more edges and defects. Then, the fusion pixel value of this pixel point in the bridge fusion image F should be as close as possible to the value of this pixel point in the image with obvious edge features. Therefore, its corresponding fusion weight is larger, that is, w ia is larger.

[0081] Furthermore, according to the above steps, the fusion weights of each pixel point in each image can be calculated. For the convenience of subsequent fusion, it is necessary to normalize the fusion weights. In the present invention, the Z-score normalization method is used to normalize the fusion weights of the pixel points at the same position in each image, and the obtained result is denoted as the normalized fusion weight W ia , which represents the normalized fusion weight of pixel point a in the i-th image. When i takes 1, 2, and 3 respectively, it represents the bridge RGB image A, the bridge grayscale image H, and the bridge infrared image C. For example, if the fusion weights of pixel point a in each image are w 1a , w 2a , w 3a , then the obtained normalized fusion weights are W 1a , W 2a , W 3a . The Z-score normalization method is a well-known technology, and the specific process is not elaborated in the present invention.

[0082] The normalized fusion weights of each pixel point in the bridge RGB image A, the bridge grayscale image H, and the bridge infrared image C obtained through the above steps reflect the contribution degree of each pixel point in each image to the bridge disease defect features. Then, the weighted average fusion algorithm can be used to fuse the bridge RGB image A, the bridge grayscale image H, and the bridge infrared image C to obtain the bridge fusion image F. The calculation formula for the fusion pixel value of each pixel point in the bridge fusion image is as follows:

[0083]

[0084] In the formula, W 1a 、W 2a 、W 3a respectively represent the normalized fusion weights of the pixel point a in the bridge RGB image A, the bridge grayscale image H, and the bridge infrared image C. F a represents the fusion pixel value of the pixel point a in the bridge fusion image F. L represents the number of channels in the color space of the bridge RGB image A. In the present invention, the empirical value 3 is taken. Q ab represents the value of the pixel point a in the bridge RGB image A in the b-th channel. When b takes 1, 2, and 3, they respectively represent the R, G, and B channels. H a represents the grayscale value of the pixel point a in the bridge grayscale image H. C a represents the pixel value of the pixel point a in the bridge infrared image C.

[0085] Calculate each pixel point according to the above steps, replace the grayscale value of each pixel point in the bridge grayscale image with the fusion pixel value of each pixel point, and use the result after replacing all pixel points in the bridge grayscale image as the bridge fusion image F.

[0086] Step S004, obtain the bridge disease and defect detection result according to the bridge fusion image.

[0087] The bridge fusion image F obtained through the above steps comprehensively reflects the bridge disease and defect characteristics in the bridge RGB image A, the bridge grayscale image H, and the bridge infrared image C, that is, the bridge disease characteristics in the bridge fusion image F are more obvious. Since bridge diseases and defects usually have irregular shapes, such as cracks and exposed steel bars, there are obvious differences in geometric features from the areas without diseases and defects. Based on this, the present invention takes the bridge fusion image as the input of the canny edge detection algorithm, takes the output of the canny edge detection algorithm as the edge image of the bridge fusion image, takes the area with irregular edge shapes in the edge image as the bridge disease and defect area, and takes the bridge disease and defect area as the bridge disease and defect detection result. The canny edge detection algorithm is a well-known technology, and the specific process will not be elaborated here.

[0088] Upload the bridge disease and defect detection result to the bridge disease and defect automatic detection system, evaluate the level of the bridge disaster and defect through the system, and then judge the urgency and priority of bridge repair, and take relevant measures in time to avoid major accidents.

[0089] Based on the same inventive concept as the above method, an embodiment of the present invention further provides a bridge disease defect automatic detection system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above methods for automatically detecting bridge disease defects are implemented.

[0090] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. The key point of each embodiment is to illustrate the differences from other embodiments. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An automated detection method for bridge disease defects, characterized in that, The method includes the following steps: Obtain three bridge images, where the three bridge images include a bridge RGB image, a bridge infrared image, and a bridge grayscale image; Obtain an RGB disease observation window, a grayscale disease observation window, and an infrared disease observation window for each pixel point according to the three bridge images. Obtain the entropy of the gray-level co-occurrence matrix determined by all pixel points in the grayscale disease observation window of each pixel point in four directions, where the four directions include 0 degrees, 45 degrees, 90 degrees, and 135 degrees. Use the pixel point at the center of the grayscale disease observation window as the starting point, and starting from the starting point, arrange the gray-level values of all pixel points in the grayscale disease observation window in a sequence according to a preset rule, and the sequence formed is used as the one-dimensional unfolded gray-level sequence of the grayscale disease observation window. The preset rule is the clockwise direction and the order of layer-by-layer arrangement. For each element value in the one-dimensional unfolded gray-level sequence of the grayscale disease observation window, if the element value is greater than or equal to the gray-level value of the pixel point at the center of the grayscale disease observation window, update the element value to 1; if the element value is less than the gray-level value of the pixel point at the center of the grayscale disease observation window, update the element value to 0. Use the result after updating all element values in the one-dimensional unfolded gray-level sequence as the one-dimensional unfolded binary sequence of the grayscale disease observation window. Use the first element value in the one-dimensional unfolded binary sequence as the first jump point, and use each element value in the one-dimensional unfolded binary sequence except the first element value as the target element value. If the target element value is different from the previous element value, use the target element value as the jump point. Use the sequence formed by the elements between every two adjacent jump points in the one-dimensional unfolded binary sequence as each jump subsequence of the grayscale disease observation window. Obtain the bridge disease defect index of each pixel point according to the entropy of the gray-level co-occurrence matrix determined by all pixel points in the grayscale disease observation window of each pixel point in four directions and all jump subsequences of the grayscale disease observation window of each pixel point. For each pixel point in the bridge RGB image, calculate the absolute value of the difference between the value of each channel of the pixel point and the value of each channel of each pixel point in the eight-neighborhood of the pixel point, calculate the mean value of the sum of the absolute values accumulated on all channels in the color space, and use the product of the sum of the mean values accumulated on the eight-neighborhood of the pixel point and the bridge disease defect index of the pixel point as the defect fusion gain index of the pixel point. For each pixel point in the bridge infrared image, calculate the absolute value of the difference between the pixel value of the pixel point and the pixel value of each pixel point in the eight-neighborhood of the pixel point, and use the product of the sum of the absolute values accumulated on the eight-neighborhood of the pixel point and the bridge disease defect index of the pixel point as the defect fusion gain index of the pixel point. For each pixel point in the bridge grayscale image, calculate the absolute value of the difference between the gray-level value of the pixel point and the gray-level value of each pixel point in the eight-neighborhood of the pixel point, and use the product of the sum of the absolute values accumulated on the eight-neighborhood of the pixel point and the bridge disease defect index of the pixel point as the defect fusion gain index of the pixel point. Obtain the fusion weights of each pixel in the bridge RGB image, bridge infrared image, and bridge gray image according to the defect fusion gain index of each pixel in the bridge RGB image, bridge infrared image, and bridge gray image; obtain the fusion pixel value of each pixel according to the fusion weights of each pixel in the bridge RGB image, bridge infrared image, and bridge gray image; Obtain the bridge fusion image according to the fusion pixel values of all pixels, and use the bridge fusion image to obtain the bridge disease defect detection result.

2. The automated detection method for bridge disease defects according to claim 1, characterized in that, The method for obtaining the entropy of the gray-level co-occurrence matrix determined in four directions for all pixels in the gray disease observation window of each pixel according to the RGB disease observation window, gray disease observation window, and infrared disease observation window of each pixel from the three bridge images is as follows: For each pixel in each bridge image, use the window with a preset size centered on the pixel as the disease observation window of the pixel; use the disease observation windows of each pixel in the bridge RGB image, bridge gray image, and bridge infrared image as the RGB disease observation window, gray disease observation window, and infrared disease observation window of each pixel respectively; For the gray disease observation window of each pixel, use the gray matrix composed of all gray values in the gray disease observation window as the input of the gray-level co-occurrence matrix algorithm, and use the output of the gray-level co-occurrence matrix algorithm as the entropy of the gray-level co-occurrence matrix determined in four directions for all pixels in the gray disease observation window.

3. The automated detection method for bridge disease defects according to claim 1, characterized in that, The method for obtaining the bridge disease defect index of each pixel according to the entropy of all pixels in four directions in the gray disease observation window of each pixel and all jump subsequences of the gray disease observation window of each pixel is as follows: In the formula, I ac represents the color variation index of the c-th pixel in the RGB disease observation window of pixel point a. L represents the number of channels in the color space of the bridge RGB image A. Sig() represents the Sigmoid function, represents the value of the c-th pixel in the RGB disease observation window of pixel point a in the d-th channel, represents the mean value of all pixels in the RGB disease observation window of pixel point a in the d-th channel; R a represents the temperature variation index of the infrared disease observation window of pixel point a. P a represents the peak signal-to-noise ratio of all pixel values in the infrared disease observation window of pixel point a. m 2 represents the number of pixel points in the infrared disease observation window and the gray disease observation window. r 1c 、r 2c respectively represent the number of pixel points in the eight-neighborhood of the c-th pixel in the infrared disease observation window of pixel point a whose pixel values are less than and greater than or equal to the pixel value of pixel point a; D a represents the bridge disease defect index of pixel point a. E a min represents the minimum value of the entropy in four directions of all pixel points in the gray disease observation window of pixel point a. f c represents the sequence length of the jump subsequence where the c-th element in the one-dimensional expanded binary sequence of the gray disease observation window of pixel point a is located.

4. The automated detection method for bridge disease defects according to claim 1, characterized in that, The method for obtaining the fusion weights of each pixel in the bridge RGB image, bridge infrared image, and bridge gray image according to the defect fusion gain index of each pixel in the bridge RGB image, bridge infrared image, and bridge gray image is as follows: For each pixel in each of the bridge RGB image, bridge infrared image, and bridge gray image, use the product of the information entropy of the pixel values of all pixels in the eight-neighborhood of the pixel and the defect fusion gain index of the pixel as the fusion weight of the pixel; The fusion weights of the pixels include the fusion weights of each pixel in the bridge RGB image, the fusion weights of each pixel in the bridge infrared image, and the fusion weights of each pixel in the bridge gray image.

5. The automated detection method for bridge disease defects according to claim 1, characterized in that, The method for obtaining the fusion pixel value of each pixel according to the fusion weights of each pixel in the bridge RGB image, bridge infrared image, and bridge gray image is as follows: Perform normalization processing on the fusion weights of each pixel in the bridge RGB image, bridge infrared image, and bridge gray image to obtain the normalized fusion weights of each pixel in the bridge RGB image, bridge infrared image, and bridge gray image respectively; For each pixel in the bridge RGB image, use the product of the mean value of the pixel on the three channels and the normalized fusion weight of the pixel as the first component factor of the pixel; For each pixel point in the grayscale image of the bridge, the product of the grayscale value of the pixel point and the normalized fusion weight of the pixel point is used as the second component factor of the pixel point; For each pixel point in the infrared image of the bridge, the product of the pixel value of the pixel point and the normalized fusion weight of the pixel point is used as the third component factor of the pixel point; The sum of the first component factor, the second component factor, and the third component factor of the pixel point is used as the fusion pixel value of the pixel point.

6. The automated detection method for bridge disease defects according to claim 1, characterized in that, The specific method for obtaining the bridge fusion image based on the fusion pixel values of all pixel points and obtaining the bridge disease defect detection result using the bridge fusion image is as follows: Replace the grayscale value of each pixel point in the grayscale image of the bridge with the fusion pixel value of each pixel point, and the result after replacing all pixel points in the grayscale image of the bridge is used as the bridge fusion image; Use the bridge fusion image as the input of the edge detection algorithm, use the output of the edge detection algorithm as the edge image of the bridge fusion image, use the area with irregular edge shapes in the edge image as the bridge disease defect area, and use the bridge disease defect area as the bridge disease defect detection result.

7. An automated detection system for bridge disease defects, including a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for automatically detecting bridge disease defects according to any one of claims 1-6.

Images