Crack detection method and device based on image enhancement and texture extraction and medium

Abstract

The application relates to a crack detection method, device and medium based on image enhancement and texture extraction. Before identification, an image enhancement network is introduced to enhance the crack identification image, a texture feature extraction network is introduced to extract a texture feature image, and then the texture feature image is input into an improved UNet network to perform crack identification. The improved UNet network replaces the standard convolution in the existing UNet network coding stage with deformable convolution; the improved UNet network replaces the last feature extraction module of the existing UNet network with a feature fusion module, the feature fusion module adopts an SE-Block unit to extract and weight the channel information of the texture feature image, adopts an ASPP unit to obtain multi-scale global context information through setting an expansion convolution with multiple expansion rates; the feature fusion module realizes the enhancement of the texture feature and improves the reusability of the texture feature, and finally performs layer-by-layer up-sampling on the extracted information to obtain a final identification result.

Description

Technical Field

[0001] This invention relates to the field of crack detection, and more particularly to crack detection methods, apparatus and media based on image enhancement and texture extraction. Background Technology

[0002] Dams are important projects for the national economy and people's livelihood, playing a vital role in power generation, flood control, and irrigation. However, due to external forces, cracks of varying degrees may appear in the dam body. Efficient and accurate crack detection and early warning are of great significance to ensuring its stability.

[0003] Traditional dam crack identification relies primarily on manual labor, which is time-consuming and costly. With the development of computer vision, crack detection technology based on digital image processing has seen significant advancements. Currently, research on crack identification based on computer vision can be broadly divided into two directions: one is based on digital image processing methods. This method involves manually identifying features, designing feature recognition conditions to constrain the identification process, and using various transformations such as frequency, grayscale, and texture to enhance crack features and eliminate interference noise. Crack segmentation is achieved through edge detection, thresholding, and region growing, and finally, morphological operations are performed on the segmentation results to obtain the final detection result. This method does not require a large amount of image data; it only needs to utilize target features and set appropriate discrimination conditions. However, this method has low robustness, cannot completely remove various complex noise interferences, is not applicable to all scenarios, and has unstable recognition results. The second type is crack detection methods based on deep learning, which mainly include crack classification, detection and segmentation methods based on deep learning. These methods use networks to automatically learn features, allowing the network to continuously adjust itself according to certain rules to complete the recognition. However, the quality of the image, which is the upper limit of these methods, will have a significant impact on the detection results. Low-quality images often cause the model to have low recognition accuracy. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, the present invention provides a crack detection method, apparatus and medium based on image enhancement and texture extraction.

[0005] In a first aspect, the present invention provides a crack detection method based on image enhancement and texture extraction, comprising:

[0006] The acquired crack recognition image is processed by an image enhancement network to obtain an enhanced RGB image of the crack recognition image; the enhanced RGB image of the crack recognition image is processed by a texture feature extraction network to obtain a texture feature image.

[0007] The texture feature image is input into the improved UNet network to obtain a binarized crack image. The improved UNet network replaces the standard convolution in the encoding stage of the existing UNet network with deformable convolution. The improved UNet network replaces the last feature extraction module of the existing UNet network with a feature fusion module. The feature fusion module is divided into two paths for feature sampling and fusion. One path uses the SE-Block unit to extract and weight the channel information of the texture feature image, and the other path uses the ASPP unit to obtain multi-scale global context information by setting multiple dilation rates of dilated convolution.

[0008] Morphological operations are performed on the cracks identified in the crack image to remove false cracks, repair the fractures in the crack detection, and output the processed identification results.

[0009] Furthermore, multiple differentiable filters are configured in the image enhancement network;

[0010] Each branch of the differentiable filter is configured with a first parameter prediction network for the first filter parameters. The first parameter prediction network includes: 3 convolutional layers, 3 pooling layers, 4 activation functions and 2 fully connected layers. The input of the first parameter prediction network is the brightness image of a single channel in the YCbCr color space, and the output is the first filter parameters of the differentiable filter.

[0011] The image enhancement network weighted and integrated the brightness enhancement sub-images obtained from each differentiable filter into an enhanced brightness image;

[0012] The enhanced luminance image is superimposed with the chromaticity component image in the YCbCr color space to obtain the enhanced RGB image.

[0013] Furthermore, the differentiable filter includes: a least-squares filter, a guided filter, and a Gaussian filter.

[0014] Furthermore, the crack images in the training dataset are converted into the YCbCr color space to obtain the luminance and chrominance component images of the crack images; different differentiable filters process the luminance images based on empirical parameters to obtain corresponding target images, with each differentiable filter corresponding to one target image; the luminance images are input into an image enhancement network, and the differentiable filters in the image enhancement network process the luminance images into corresponding luminance enhancement sub-images; the mean square error between the target image and the luminance enhancement sub-image corresponding to each differentiable filter is calculated; the first parameter prediction network weights are trained with the goal of minimizing the sum of the mean square errors of the training dataset to update the first filter parameters;

[0015] The proportion of the mean square error of each differentiable filter to the total mean square error is used as the selection weight of each differentiable filter. The brightness enhancement sub-image output by each differentiable filter is multiplied by the corresponding selection weight and superimposed to obtain the enhanced brightness image. The selection weight corresponding to the optimal first parameter prediction network is used as the final selection weight of the image enhancement network.

[0016] Furthermore, multiple texture extraction filters are configured in the texture extraction network;

[0017] A second parameter prediction network is configured for each texture extraction filter. The second parameter prediction network is used to predict the second filter parameters of the corresponding texture extraction filter. Each second parameter prediction network includes 5 convolutional layers, 5 pooling layers, 6 activation functions and 2 fully connected layers.

[0018] Furthermore, the texture extraction filter adopts a Gabor filter, and the texture extraction network is configured with 5 Gabor filters, with the filter kernel sizes of the 5 Gabor filters being 7, 9, 11, 13, and 15, respectively.

[0019] Furthermore, the ASPP unit is configured with three parallel dilated convolutions, with a kernel size of 3×3 and dilation rates of 3, 5, and 9, respectively.

[0020] Furthermore, crack label images and enhanced RGB images of crack images in the training dataset are obtained. The enhanced RGB image is input into a texture extraction network, and the output of the texture extraction network and the crack label image are input into an improved UNet network. The binarized crack image is obtained through the improved UNet network. During this process, the parameters of the texture feature extraction network and the improved UNet network are updated by calculating the cross-entropy loss between the crack label image and the binarized crack image. The trained texture feature extraction network and the improved UNet network are obtained when the cross-entropy loss is minimized.

[0021] Secondly, the present invention provides a crack detection device based on image enhancement and texture extraction, comprising: an image acquisition unit, a storage unit, a processing unit, and a bus unit, wherein the bus unit connects the image acquisition unit, the storage unit, and the processing unit, the storage unit stores a computer program, and the computer program, when executed by the processing unit, implements the crack detection method based on image enhancement and texture extraction.

[0022] The third invention provides a storage medium for implementing a crack detection method based on image enhancement and texture extraction, wherein the storage medium stores a computer program that, when executed by a processor, implements the crack detection method based on image enhancement and texture extraction.

[0023] The technical solutions provided in the embodiments of the present invention have the following advantages compared with the prior art:

[0024] This invention constructs an image enhancement network for image enhancement. Taking into account the diversity of image noise when enhancing images, it constructs an image enhancement network containing multiple differentiable filters and adopts an adaptive filter weight configuration strategy for image enhancement, resulting in good enhancement effect.

[0025] The enhanced RGB image is used for crack identification through a texture feature extraction network and an improved UNet network. The improved UNet network replaces the standard convolution in the encoding stage of the existing UNet network with deformable convolution to enhance the texture features of the texture image. The improved UNet network replaces the last feature extraction module of the existing UNet network with a feature fusion module. The feature fusion module is divided into two paths for feature sampling and fusion. One path uses SE-Block units to extract and weight the channel information of the texture image, and the other path uses ASPP units to obtain multi-scale global context information by setting multiple dilation rates of dilated convolutions. The feature fusion module realizes the fusion of features in both the channel and spatial parts, fully extracts and reuses texture features, and ensures the recognition effect of crack identification. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart of a crack detection method based on image enhancement and texture extraction provided in an embodiment of the present invention;

[0029] Figure 2 A schematic diagram of the network structure for implementing the crack detection method based on image enhancement and texture extraction, provided in an embodiment of the present invention;

[0030] Figure 3 A schematic diagram of an image enhancement network provided for an embodiment of the invention;

[0031] Figure 4 A schematic diagram of a texture feature extraction network and an improved UNet network provided for embodiments of the invention;

[0032] Figure 5A crack image provided in an embodiment of the present invention for forming a training dataset;

[0033] Figure 6 A crack label image provided for a crack image in an embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0036] Example 1

[0037] This invention provides a crack detection method based on image enhancement and texture extraction, implemented through the combined use of a trained image enhancement network, a texture extraction network, and an improved UNet network. (See reference...) Figure 2 As shown, the image enhancement network, texture extraction network, and improved UNet network are combined to form the network structure that implements the crack detection method based on image enhancement and texture extraction.

[0038] Training image enhancement networks, texture extraction networks, and improved UNet networks requires training datasets. The process of creating training datasets includes: acquiring crack images for training network models, such as... Figure 5 As shown. Crack regions in each crack image are labeled. In this embodiment, image labeling is done using Photoshop software. The lasso tool is used to obtain the crack regions in the crack images, and the pixel values ​​of the crack regions are set to 1, while the pixel values ​​of the background regions (excluding the crack regions) are set to 0 to obtain the corresponding crack label images, as shown. Figure 6 As shown.

[0039] like Figure 1 As shown, this invention provides a crack detection method based on image enhancement and texture extraction, comprising the following steps:

[0040] S100: The acquired crack identification image is processed by an image enhancement network to obtain an enhanced RGB image of the crack identification image. Specifically, when the crack identification image is acquired, it is converted into the YCbCr color space to obtain the luminance image and chrominance component image of the crack identification image. The luminance image is input into the image enhancement network to obtain several first filter parameters for adjusting differentiable filters. The luminance image is then processed by the differentiable filters defined by the first filter parameters to output a luminance enhancement sub-image. The luminance enhancement sub-images are weighted and integrated to obtain the enhanced luminance image. The enhanced luminance image is then superimposed with the chrominance component image to obtain the enhanced RGB image of the crack identification image.

[0041] The image enhancement network required to construct a crack detection method based on image enhancement and texture extraction:

[0042] See Figure 3 As shown, the image enhancement network is configured with several differentiable filters, including a weighted least squares filter, a guided filter, and a Gaussian filter. Each branch of the differentiable filter is configured with a first parameter prediction network, which is used by the image enhancement network to predict the first filter parameters of the differentiable filter. In a preferred embodiment, the first parameter prediction network includes three convolutional layers, three pooling layers, four activation functions, and two fully connected layers. The input to the first parameter prediction network is a single-channel luminance image in the YCbCr color space, and the output is the first filter parameters of the differentiable filter. The first filter parameters to be trained for the weighted least squares filter include: smoothness lambda and edge-preserving parameter alpha; the first filter parameters to be trained for the guided filter include: variance harmonic parameter eps of the guided image; the first filter parameters to be trained for the Gaussian filter include: standard deviation of the Gaussian kernel sigmaX in the X direction and standard deviation of the Gaussian kernel sigmaY in the Y direction. The kernel size of both the guided filter and the Gaussian filter is set to 13.

[0043] The constructed image enhancement network is trained using a training dataset, including: converting crack images in the training dataset into the YCbCr color space to obtain luminance and chrominance component images of the crack images; different differentiable filters process the luminance images based on empirical parameters to obtain corresponding target images, with each differentiable filter corresponding to one target image; inputting the luminance images into the image enhancement network, where the differentiable filters process the luminance images into corresponding luminance enhancement sub-images; calculating the mean square error between the target image and the luminance enhancement sub-image for each differentiable filter; and training the first parameter prediction network weights to update the first filter parameters with the goal of minimizing the sum of the mean square errors in the training dataset.

[0044] The total mean square error is obtained by summing the mean square errors of the brightness enhancement sub-images from different differentiable filters. The proportion of the mean square error of each differentiable filter to the total mean square error is used as the selection weight Wi for each differentiable filter. The enhanced brightness image is obtained by multiplying the brightness-enhanced sub-images output by each differentiable filter with their corresponding selection weights and then summing them. The selection weights corresponding to the optimal first parameter prediction network are used as the final selection weights for the image enhancement network.

[0045] S200: The enhanced RGB image of the crack identification image is processed by a texture feature extraction network to obtain a texture feature image. The texture feature image is then input into an improved UNet network to obtain a binarized crack image, where pixel value 0 represents the background and pixel value 1 represents the crack. Specifically, the enhanced RGB image is input into the texture feature extraction network, and then input into an image texture extraction filter to extract the texture feature image.

[0046] The texture extraction network required to construct a crack detection method based on image enhancement and texture extraction:

[0047] See Figure 4 As shown, multiple texture extraction filters are configured in the texture extraction network, and a second parameter prediction network is configured for each texture extraction filter. The second parameter prediction network is used to predict the second filter parameters of the corresponding texture extraction filter. Each second parameter prediction network includes 5 convolutional layers, 5 pooling layers, 6 activation functions, and 2 fully connected layers. In the embodiment provided by this invention, the texture extraction filters are Gabor filters, and the texture extraction network is configured with 5 Gabor filters. The kernel sizes of the 5 Gabor filters are 7, 9, 11, 13, and 15, respectively. The Gabor filter is a windowed short-time Fourier transform, which is a special case of the short-time Fourier transform when the window function is a Gaussian function. The 5 Gabor filters output 5 texture feature images. The second filter parameters required by the Gabor filters include the filtering wavelength, the tilt angle of the Gabor kernel function image, the phase offset, the standard deviation of the Gaussian function, and the ellipticity.

[0048] The improved UNet network required to build a crack detection method based on image enhancement and texture extraction:

[0049] The improved UNet network replaces the standard convolutions in the encoding stage of the existing UNet network with deformable convolutions to enhance the texture features of texture feature images. To better reuse texture information in texture feature images, a feature fusion module is designed to replace the last feature extraction module in the existing UNet network. This feature fusion module performs feature sampling and fusion in two paths: one path uses an SE-Block unit to extract and weight the channel features of the texture feature image. Specifically, for the input feature map, the SE-Block unit performs spatial feature compression through global average pooling in the spatial dimension. For the spatially compressed feature map, channel features are learned through fully connected layer operations, and finally, the channel features and the input feature map are weighted channel-wise. The other path uses an ASPP unit to obtain multi-scale global context information through dilated convolutions with multiple dilation rates. This feature fusion module achieves feature fusion of both channel and spatial components. In this embodiment, the ASPP unit sets three parallel dilated convolutions with a kernel size of 3×3 and dilation rates of 3, 5, and 9.

[0050] The texture extraction network and the improved UNet network are trained as follows:

[0051] Obtain the crack label images of the crack images in the training dataset.

[0052] The RGB image of the adaptively enhanced crack image is input into the texture extraction network. The output of the texture extraction network and the crack label image are input into the improved UNet network. The binarized crack image is obtained through the improved UNet network. During this process, the parameters of the texture feature extraction network and the improved UNet network are updated by calculating the cross-entropy loss between the crack label image and the binarized crack image. The trained texture feature extraction network and the improved UNet network are obtained when the cross-entropy loss is minimized.

[0053] S300: Perform morphological operations on the cracks identified in the crack image to remove false cracks, repair the fractures in the crack detection, and output the processed identification results.

[0054] In the embodiments provided by this invention, it should be understood that the disclosed structures and methods can be implemented in other ways. For example, the structural embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, structures, or units, and may be electrical, mechanical, or other forms.

[0055] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0056] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0057] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A crack detection method based on image enhancement and texture extraction, characterized in that, include: The acquired crack identification images are processed by an image enhancement network to obtain an enhanced RGB image of the crack identification images; When a crack identification image is acquired, the crack identification image is converted into the YCbCr color space to obtain the brightness image and chromaticity component image of the crack identification image; The brightness image is input into an image enhancement network to obtain several first filter parameters that can be adjusted for differentiable filters. The brightness image is input into an image enhancement network and processed by a differentiable filter defined by the first filter parameters to output a brightness enhancement sub-image. The proportion of the mean square error of each differentiable filter to the total mean square error is used as the selection weight of each differentiable filter. The brightness enhancement sub-images output by each differentiable filter are multiplied by their corresponding selection weights and then superimposed to obtain the enhanced brightness image. Multiple differentiable filters are configured in the image enhancement network. Each branch of a differentiable filter is configured with a first parameter prediction network for the first filter parameters. The first parameter prediction network includes 3 convolutional layers, 3 pooling layers, 4 activation functions, and 2 fully connected layers. The input of the first parameter prediction network is a single-channel brightness image in the YCbCr color space, and the output is the first filter parameters of the differentiable filter. The image enhancement network weights and integrates the brightness enhancement sub-images obtained from each differentiable filter to obtain the enhanced brightness image. The enhanced brightness image is superimposed with the chrominance component image in the YCbCr color space and inversely transformed to obtain an enhanced RGB image. The enhanced RGB image of the crack recognition image is processed by a texture feature extraction network to obtain a texture feature image; The texture feature image is input into the improved UNet network to obtain a binarized crack image. The improved UNet network replaces the standard convolution in the encoding stage of the existing UNet network with deformable convolution. The improved UNet network replaces the last feature extraction module of the existing UNet network with a feature fusion module. The feature fusion module is divided into two paths for feature sampling and fusion. One path uses the SE-Block unit to extract and weight the channel information of the texture feature image, and the other path uses the ASPP unit to obtain multi-scale global context information by setting multiple dilation rates of dilated convolution. Morphological operations are performed on the cracks identified in the crack image to remove false cracks, repair the fractures in the crack detection, and output the processed identification results.

2. The crack detection method based on image enhancement and texture extraction according to claim 1, characterized in that, The differentiable filters include: weighted least squares filters, guided filters, and Gaussian filters.

3. The crack detection method based on image enhancement and texture extraction according to claim 1, characterized in that, Multiple texture extraction filters are configured in the texture feature extraction network; A second parameter prediction network is configured for each texture extraction filter. The second parameter prediction network is used to predict the second filter parameters of the corresponding texture extraction filter. Each second parameter prediction network includes 5 convolutional layers, 5 pooling layers, 6 activation functions and 2 fully connected layers.

4. The crack detection method based on image enhancement and texture extraction according to claim 3, characterized in that, The texture extraction filter uses a Gabor filter, and the texture feature extraction network is configured with 5 Gabor filters, with filter kernel sizes of 7, 9, 11, 13, and 15, respectively.

5. The crack detection method based on image enhancement and texture extraction according to claim 1, characterized in that, The ASPP unit is configured with three parallel dilated convolutions, with a kernel size of 3.

3. The expansion rates are 3, 5, and 9, respectively.

6. The crack detection method based on image enhancement and texture extraction according to claim 1, characterized in that, Crack label images and enhanced RGB images of crack images from the training dataset are obtained. The enhanced RGB image is input into a texture feature extraction network. The output of the texture feature extraction network and the crack label image are input into an improved UNet network. The binarized crack image is obtained through the improved UNet network. During this process, the parameters of the texture feature extraction network and the improved UNet network are updated by calculating the cross-entropy loss between the crack label image and the binarized crack image. The trained texture feature extraction network and the improved UNet network are obtained when the cross-entropy loss is minimized.

7. A crack detection device based on image enhancement and texture extraction, characterized in that, include: The system includes an image acquisition unit, a storage unit, a processing unit, and a bus unit, wherein the bus unit connects the image acquisition unit, the storage unit, and the processing unit, the storage unit stores a computer program, and the computer program, when executed by the processing unit, implements the crack detection method based on image enhancement and texture extraction as described in any one of claims 1-6.

8. A storage medium for implementing a crack detection method based on image enhancement and texture extraction, the storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the crack detection method based on image enhancement and texture extraction as described in any one of claims 1-6.

Images