A multi-scene adaptive method for bridge tunnel apparent disease identification

Abstract

The application discloses a multi-scene adaptive method for bridge tunnel apparent disease identification, comprising the following steps: dividing a training image set and a target operation scene image set into a preset number of groups according to brightness; performing preset number of times of style transfer based on LAB space on the preset number of groups of data of the two groups of images in a one-to-one correspondence, combining the preset number of groups of training data after the preset number of times of migration into a new data set; performing frequency spectrum exchange based on Fourier transform on the new data set and the target scene image data set again, exchanging low-frequency information, and making the two groups of data more similar in appearance style; and sending the training images after the style transfer into a preset disease identification network for training. The method can be implemented in the case where a target scene data set label is lacking, and the apparent disease identification network can be trained by using the obtained labeled apparent disease data set, so that the self-adaptation to different target scenes can be achieved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and in particular relates to a multi-scenario adaptive method for identifying apparent defects in bridges and tunnels. Background Technology

[0002] Over time, bridges and tunnels develop surface defects such as cracks, uneven foundation settlement, peeling, potholes, and even spalling of the road surface. These problems significantly impact the safety of travel and transportation. As the cornerstone of the transportation system, bridge and tunnel defect detection is receiving increasing attention. More effort is needed in both the early construction and later maintenance phases. Traditional detection methods mostly rely on manual labor using scaffolding, ladders, bridge inspection vehicles, and climbing ropes. Due to the safety risks and excessive time and labor involved in manual inspection, deep learning-based bridge and tunnel surface defect detection has become a research hotspot in recent years. This method primarily uses CNN (Convolutional Neural Networks) to extract defect image features for defect identification. Various image processing and deep learning-based methods for detecting bridge and tunnel surface defects have emerged in recent years. Compared to manual inspection, automatic identification using image algorithms can more quickly and accurately detect and identify defects, precisely determining their location and type. As a key research area in computer vision, classic image algorithms for bridge and tunnel surface defect detection include methods based on grayscale thresholding, edge detection, and specific tools. Numerous scholars have proposed improved algorithms to address issues such as background interference, illumination occlusion, and image noise in classical algorithms. These algorithms have shown good performance in crack detection. However, with the advancement of segmentation models based on convolutional neural networks, deep learning-based detection of apparent defects in bridges and tunnels has become a major research hotspot in recent years. Early methods involved combining manual feature extraction with classifiers to extract texture features from preprocessed grayscale images, followed by classification using an SVM classifier. However, this method was limited in complex backgrounds. With the development of sensing technology, software, and hardware, deep learning-based methods for detecting apparent defects can now automatically acquire features. Furthermore, the increasing depth and number of parameters in convolutional neural networks have gradually improved the detection and recognition capabilities for apparent defects. One scholar proposed dividing a photograph into rectangular blocks, extracting features using an 11-layer convolutional neural network, and then training a softmax layer for classification to detect whether the rectangular blocks contain cracks. Other scholars have introduced semantic segmentation-based parallel networks into defect detection, enabling the identification of a wider range of defects. The aforementioned convolutional neural network-based defect identification method has a problem: deep learning methods require training data and its annotations. When faced with different bridge and tunnel operation scenarios, existing network models need to acquire annotated data for the target scene after the switch. However, image annotation is a time-consuming and labor-intensive task. Even if data from scene A is used for training, the accuracy often drops significantly when identifying defects in scene B because the trained network has poor adaptability. When the amount of image data for the target scene is large and annotation is difficult, the adaptability of the identification method becomes particularly important. To solve this problem, a multi-scene adaptive method for identifying apparent defects in bridges and tunnels is urgently needed. Summary of the Invention

[0003] To address the above technical problems, this invention provides a multi-scenario adaptive method for identifying apparent defects in bridges and tunnels.

[0004] The technical solution adopted by this invention to solve its technical problem is:

[0005] A multi-scenario adaptive method for identifying apparent defects in bridges and tunnels, comprising the following steps:

[0006] S100: Divide the training image dataset and the target scene image dataset into a preset number of groups according to brightness, and make them correspond one-to-one according to brightness;

[0007] S200: Perform style transfer on the corresponding group of training image datasets and target scene image datasets in the LAB color space, convert the style-transferred LAB images to RGB images, and merge the style-transferred RGB training image datasets of a preset number of groups into a new dataset.

[0008] S300: Perform a Fourier transform on the new dataset and the target image dataset, then swap their spectra to obtain a new dataset with swapped spectra. Perform an inverse Fourier transform on the new dataset with swapped spectra to obtain the processed training dataset.

[0009] S400: The processed training dataset is fed into a pre-set network for identifying bridge and tunnel defects for training.

[0010] Preferably, S100 includes:

[0011] S110: Obtain the training image dataset, and convert R... 3×H×W Training image data The entire image was converted to the LAB color space, i.e. Where L represents the overall brightness of the image, A represents the component from green to red, and B represents the component from blue to yellow.

[0012] S120: Obtain the target scene image dataset, and convert it to R 3×H×W Target scene image data The entire image was converted to the LAB color space, i.e. ;

[0013] S130: Obtain the converted image and The brightness matrix of the L channel is used to obtain the average brightness value of each training data image and the target scene data image according to the following formula:

[0014] ,

[0015] in, for The brightness matrix of the L channel in the three channels. for The brightness matrix of the L channel in the three channels;

[0016] S140: Based on the average brightness value of each training data image and the target scene data image obtained. and The images in the training image dataset and the target scene image dataset are arranged from low brightness to high brightness and divided into 5 groups with an average number of images each. The five groups in the training image dataset and the target scene image dataset are matched one-to-one according to brightness.

[0017] Preferably, in S200, the training image dataset and the target scene image dataset of the corresponding group are style transferred in the LAB color space, specifically as follows:

[0018]

[0019] in, For training set images converted to LAB space, This represents the training set LAB images whose style is similar to the target scene after using the average mean and average variance. This represents the mean of the i-th training set image in the current group. This represents the standard deviation of the i-th training image in the current group. This represents the mean value of the i-th target scene image in the corresponding group. This represents the standard deviation of the i-th target scene image in the corresponding group. This represents the number of images in the training scene within the current group. This represents the number of target scene images in the corresponding group.

[0020] Preferably, in S300, a Fourier transform is performed on the new dataset, specifically as follows:

[0021]

[0022] Where H represents the height of the training image, W represents the width of the training image, and m and n are the frequency components in the Fourier transform. Represents spatial domain image variables.

[0023] Preferably, in step S300, an inverse Fourier transform is performed on the new dataset after exchanging the spectrum to obtain the processed training dataset, specifically as follows:

[0024]

[0025] in,

[0026] in, , RH×W×3 → RH×W×3 is the Fourier transform of an RGB image. The amplitude and phase components, As a mask, its value is zero except in the central region β∈(0,1). It is a hyperparameter. For the target scene image dataset, This is the processed training dataset.

[0027] The aforementioned multi-scene adaptive method for identifying apparent defects in bridges and tunnels uses style transfer based on LAB color space grouping based on brightness. The style transfer effect is better in LAB color space with a larger color gamut. Moreover, the whole process is carried out offline, which enables the network to learn the information of the target scene when using target scene data with missing annotations without increasing network redundancy. This can effectively solve the problem of needing to reacquire the image annotation of the scene when switching target operation scenes.

[0028] Compared with existing methods for identifying apparent defects in bridges and tunnels, this invention achieves better defect identification results when the target scene is changed, without excessive reliance on target scene data or alteration of the network structure. The method proposed in this invention is applicable to most apparent defect identification networks, and can simply and intuitively improve the network's adaptability. Attached Figure Description

[0029] Figure 1 This is a flowchart of a multi-scenario adaptive method for identifying apparent defects in bridges and tunnels according to an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of the first process of a multi-scenario adaptive method for identifying apparent defects in bridges and tunnels according to an embodiment of the present invention;

[0031] Figure 3 This is a schematic diagram of the second process of a multi-scenario adaptive method for identifying apparent defects in bridges and tunnels according to an embodiment of the present invention;

[0032] Figure 4 This is a schematic diagram illustrating the grouping of the training image dataset and the target scene image dataset according to brightness in one embodiment of the present invention. Implementation

[0033] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0034] In one embodiment, such as Figure 1 , 2As shown in Figure 3, a multi-scenario adaptive method for identifying apparent defects in bridges and tunnels includes the following steps:

[0035] S100: Divide the training image dataset and the target scene image dataset into a preset number of groups according to brightness, and make them correspond one-to-one according to brightness.

[0036] Specifically, the training dataset uses the already acquired labeled training data, while the target scene image set consists of unlabeled target scene images.

[0037] In one embodiment, S100 includes:

[0038] S110: Obtain the training image dataset, and convert R... 3×H×W Training image data The entire image was converted to the LAB color space, i.e. Where L represents the overall brightness of the image, A represents the component from green to red, and B represents the component from blue to yellow.

[0039] S120: Obtain the target scene image dataset, and convert it to R 3×H×W Target scene image data The entire image was converted to the LAB color space, i.e. ;

[0040] S130: Obtain the converted image and The brightness matrix of the L channel is used to obtain the average brightness value of each training data image and the target scene data image according to the following formula:

[0041] ,

[0042] in, for The brightness matrix of the L channel in the three channels. for The brightness matrix of the L channel in the three channels;

[0043] S140: Based on the average brightness value of each training data image and the target scene data image obtained. and The images in the training image dataset and the target scene image dataset are arranged from low brightness to high brightness and divided into 5 groups with an average number of images each. The five groups in the training image dataset and the target scene image dataset are matched one-to-one according to brightness.

[0044] Specifically, source domain image The original three channels, R, G, B (red, green, blue), have been changed to L, A, B channels.

[0045] like Figure 4 As shown, the images in the training image dataset are arranged from low brightness to high brightness and divided into 5 groups with an average number of images each, such as the first brightness group to the fifth brightness group. The darkest group corresponds to the first brightness group, and the brightest group corresponds to the fifth brightness group. The target scene image dataset is also divided into the first brightness group to the fifth brightness group in the same way. The first brightness group in the training image dataset corresponds to the first brightness group in the target scene image dataset, and so on. The 5 groups correspond one-to-one according to brightness.

[0046] S200: Perform style transfer on the corresponding group of training image datasets and target scene image datasets in the LAB color space, convert the style-transferred LAB images to RGB images, and merge the style-transferred RGB training image datasets of a preset number of groups into a new dataset.

[0047] In one embodiment, in S200, style transfer is performed on the training image dataset and the target scene image dataset of the corresponding group in the LAB color space, specifically as follows:

[0048]

[0049] in, For training set images converted to LAB space, This represents the training set LAB images whose style is similar to the target scene after using the average mean and average variance. This represents the mean of the i-th training set image in the current group. This represents the standard deviation of the i-th training image in the current group. This represents the mean value of the i-th target scene image in the corresponding group. This represents the standard deviation of the i-th target scene image in the corresponding group. This represents the number of images in the training scene within the current group. This represents the number of target scene images in the corresponding group.

[0050] Specifically, style transfer is performed on the training image dataset and target scene image dataset of the first brightness group in the LAB color space. Similarly, style transfer is performed on the training image dataset and target scene image dataset of the second brightness group in the LAB color space, and so on, for all five corresponding groups of data. After style transfer of the five corresponding data groups, the transferred images are... Convert to RGB image .

[0051] S300: Perform a Fourier transform on the new dataset and the target image dataset, then swap their spectra to obtain a new dataset with swapped spectra. Perform an inverse Fourier transform on the new dataset with swapped spectra to obtain the processed training dataset.

[0052] In one embodiment, S300 performs a Fourier transform on the new dataset, specifically as follows:

[0053]

[0054] Where H represents the height of the training image, W represents the width of the training image, and m and n are the frequency components in the Fourier transform. Represents spatial domain image variables.

[0055] In one embodiment, in step S300, an inverse Fourier transform is performed on the new dataset after exchanging the spectrum to obtain the processed training dataset, specifically:

[0056]

[0057] in,

[0058] in, , RH×W×3 → RH×W×3 is the Fourier transform of an RGB image. The amplitude and phase components, As a mask, its value is zero except in the central region β∈(0,1). It is a hyperparameter. For the target scene image dataset, This is the processed training dataset.

[0059] Specifically, the Fourier transform and inverse transform mainly achieve the conversion of the low-frequency components of the new dataset image and the target scene image without changing the semantic content. The input and output of this step are both R. 3×H×W The style and pattern information of the transformed training set images is closer to that of the target scene image.

[0060] S400: The processed training dataset is fed into a pre-set network for identifying bridge and tunnel defects for training.

[0061] Specifically, any suitable apparent disease identification network can be selected as the basic network architecture for apparent disease identification. The method proposed in this invention does not require any changes to the network and does not depend on the performance of the identification network. R 3×H×W The training dataset images processed as described above are fed into the network for training. Even when training is conducted with missing or incomplete target scene annotations, the model can still learn the data information of the target scene.

[0062] This invention provides a multi-scene adaptive method for identifying apparent defects in bridges and tunnels. It uses style transfer based on the LAB color space with brightness grouping. The style transfer effect is better in the LAB color space with a larger color gamut. Moreover, the whole process is carried out offline. It can enable the network to learn the information of the target scene when using target scene data with missing annotations without increasing network redundancy. It can effectively solve the problem of needing to reacquire the image annotation of the scene when switching target operation scenes.

[0063] Compared with existing methods for identifying apparent defects in bridges and tunnels, this invention achieves better defect identification results when the target scene is changed, without excessive reliance on target scene data or alteration of the network structure. The method proposed in this invention is applicable to most apparent defect identification networks, and can simply and intuitively improve the network's adaptability.

[0064] The above provides a detailed description of a multi-scenario adaptive method for identifying apparent defects in bridges and tunnels, as provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the core ideas of this invention. It should be noted that those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this invention.

Claims

1. A multi-scenario adaptive method for identifying apparent defects in bridges and tunnels, characterized in that, The method includes the following steps: S100: Divide the training image dataset and the target scene image dataset into a preset number of groups according to brightness, and make them correspond one-to-one according to brightness; S200: Perform style transfer on the corresponding group of training image datasets and target scene image datasets in the LAB color space, convert the style-transferred LAB images to RGB images, and merge the style-transferred RGB training image datasets of a preset number of groups into a new dataset; Specifically, in S200, the style transfer on the corresponding group of training image datasets and target scene image datasets in the LAB color space is as follows: in, For training set images converted to LAB space, This represents the training set LAB images whose style is similar to the target scene after using the average mean and average variance. This represents the mean of the i-th training set image in the current group. This represents the standard deviation of the i-th training image in the current group. This represents the mean value of the i-th target scene image in the corresponding group. This represents the standard deviation of the i-th target scene image in the corresponding group. This represents the number of images in the training scene within the current group. This represents the number of target scene images in the corresponding group; S300: Perform a Fourier transform on the new dataset and the target scene image dataset, then swap their spectra to obtain a new dataset with swapped spectra. Perform an inverse Fourier transform on the new dataset with swapped spectra to obtain the processed training dataset. S400: The processed training dataset is fed into a preset bridge and tunnel performance defect identification network for training.

2. The method according to claim 1, characterized in that, S100 includes: S110: Obtain the training image dataset, and convert R... 3×H×W Training image data The entire image was converted to the LAB color space, i.e. Where L represents the overall brightness of the image, A represents the component from green to red, and B represents the component from blue to yellow. S120: Obtain the target scene image dataset, and convert it to R 3×H×W Target scene image data The entire image was converted to the LAB color space, i.e. ; S130: Obtain the converted image and The brightness matrix of the L channel is used to obtain the average brightness value of each training data image and the target scene data image according to the following formula: ， in, for The brightness matrix of the L channel in the three channels. for The brightness matrix of the L channel in the three channels; S140: Based on the average brightness value of each training data image and the target scene data image obtained. and The images in the training image dataset and the target scene image dataset are arranged from low brightness to high brightness, and each is divided into 5 groups with an average number of images. The five groups in the training image dataset and the target scene image dataset correspond one-to-one according to brightness.

3. The method according to claim 2, characterized in that, In S300, a Fourier transform is performed on the new dataset, specifically as follows: Where H represents the height of the training image, W represents the width of the training image, and m and n are the frequency components in the Fourier transform. Represents spatial domain image variables.

4. The method according to claim 3, characterized in that, In step S300, an inverse Fourier transform is performed on the new dataset after the spectrum exchange to obtain the processed training dataset, specifically: in, in, , It is the Fourier transform of an RGB image. The amplitude and phase components, As a mask, its value is zero except in the central region β∈(0,1). It is a hyperparameter. For the target scene image dataset, This is the processed training dataset.

Images