A method for automatic classification of ground penetrating radar images

Abstract

The disclosed embodiment provides an automatic classification of ground penetrating radar images, which belongs to the field of image processing technology, and specifically includes: scanning concrete by emitting radar waves through a ground penetrating radar, and the receiver in the radar captures multiple echo data to form a scanning echo data set; preprocessing the scanning echo data, removing the noise signal in the scanning echo data set and enhancing the echo signal of the deep target, and obtaining a ground penetrating radar grayscale image with clear text information; inputting the ground penetrating radar grayscale image into the optimized residual network model, and outputting the defect category corresponding to the image as the recognition result after calculation; and adding the recognition result to the data set to update the residual network model if it is accurate after actual verification in the later stage. Through the scheme disclosed in the present invention, the collected data is normalized and the convolutional neural network is improved to make it more suitable for the recognition of radar images, thereby improving the accuracy, efficiency and adaptability of the detection of internal defects in concrete.

Description

Technical Field

[0001] The disclosed embodiments relate to the field of image processing technology, and in particular to a method for automatically classifying ground penetrating radar images. Background Art

[0002] At present, with the progress of national infrastructure construction, the density of the national highway network continues to increase. By the end of 2020, the total mileage of national highways will reach more than 5 million kilometers; there will be more than 900,000 highway bridges, more than 60 million linear meters; there will be more than 20,000 highway tunnels, more than 20 million linear meters. As time goes by, various problems will gradually appear in the internal concrete structures of highways, bridges and tunnels, and various defects will also occur during the construction process. Therefore, the demand for detection of these safety hazards has increased sharply in recent years.

[0003] At present, according to the detection method, it can be divided into contact detection and non-contact detection. Contact detection can be surface contact or internal contact. Internal contact detection uses a probe to detect the internal non-compactness of concrete during pouring. However, this method has low detection efficiency and requires the probe to be inserted into the concrete. The types and depths of detection defects are limited and it is heavily dependent on the experience of the detection personnel. Distributed optical fiber is also an effective method for internal detection of concrete, but the detection position of this method is fixed, the sensor cannot be replaced and repaired, and when the detection area is too large, the number of buried sensors increases dramatically, resulting in uncontrollable costs. Surface contact detection schemes include the use of impact elastic waves, (ultrasound) waves, and CT. The application of elastic waves requires placing an exciter at different marked positions of the concrete to output elastic waves, and setting a receiver at the corresponding position on the opposite side to receive them. The single detection range is not large, and the measurement position cannot be quickly changed, resulting in low detection efficiency. Ultrasonic waves can detect continuous defects inside concrete through tomography, but this method is prone to diffraction, so that the size of the detected defects cannot be too small and the detection depth is shallow.

[0004] The most widely used non-contact detection method is the ground penetrating radar method. Ground penetrating radar has a fast detection speed, high resolution and accuracy, and can detect without contacting the detection target and will not cause damage or destruction to the detection target. The rapid growth of roads, tunnels, bridges, etc. built with concrete structures driven by urban development has posed a greater challenge to the efficiency of detection methods. Ground penetrating radar can achieve high-speed detection tasks through rapid scanning, so it is widely used. Although ground penetrating radar is efficient in collecting data, its analysis task for massive amounts of collected data still relies on manual work, which is not only inefficient and cannot be carried out for a long time, but also the analysis process of related manual experts relies on personal subjective feelings, and the recognition accuracy is not high in the face of the complex environment inside the concrete. In addition, the detection of deep defects by this method is greatly affected by the attenuation of reflected waves, and an effective unified standard for the preprocessing of collected data has yet to be formed.

[0005] For the above reasons, there are many methods for detecting internal defects in concrete. Non-destructive and non-contact ground penetrating radar has a wide range of applications. However, the interpretation of the collected radar data relies heavily on the subjectivity of human experts, and the accuracy and efficiency are low and lack automation. The pre-processing methods are too different, resulting in diverse manifestations of the same defect and the inability to unify them. The echo of deep defects in ground penetrating radar is easily attenuated and affected by noise.

[0006] It can be seen that there is an urgent need for an automatic classification method for ground penetrating radar images with high detection accuracy, efficiency and adaptability. Summary of the invention

[0007] In view of this, an embodiment of the present disclosure provides a method for automatic classification of ground penetrating radar images, which at least partially solves the problems of poor detection efficiency, accuracy and adaptability in the prior art.

[0008] In a first aspect, an embodiment of the present disclosure provides a method for automatically classifying ground penetrating radar images, comprising:

[0009] Step 1: Scanning the concrete by transmitting radar waves through a ground penetrating radar, and a receiver in the radar captures multiple echo data to form a scanning echo data set;

[0010] Step 2, preprocessing the scanning echo data, removing noise signals in the scanning echo data set and enhancing echo signals of deep targets to obtain a ground penetrating radar grayscale image with clear textural information;

[0011] Step 3, inputting the ground penetrating radar grayscale image into the optimized residual network model, and outputting the defect category corresponding to the image as the recognition result after calculation;

[0012] Step 4: If the recognition result is actually verified to be accurate in the later stage, the recognition result is added to the data set to update the residual network model.

[0013] According to a specific implementation of the embodiment of the present disclosure, step 1 specifically includes:

[0014] The ground penetrating radar transmitter continuously emits radar waves of a preset fixed center frequency as the radar moves, and the receiver receives multiple channels of single-column echo data and splices the multiple channels of the single-column echo data to form the scanning echo data.

[0015] According to a specific implementation of the embodiment of the present disclosure, step 2 specifically includes:

[0016] Step 2.1, removing DC drift from the scan echo data;

[0017] Step 2.2, remove the horizontal signal in the data after DC drift removal, and cut the data by the depth corresponding to the peak value of the echo waveform after static correction processing;

[0018] Step 2.3, the data after the horizontal signal noise is removed is subjected to gain processing, and the amplitude of the deep signal is amplified by the gain to enhance the texture features in the grayscale image;

[0019] Step 2.4, bandpass filtering is performed on the amplified data to remove low-frequency and high-frequency noise signals that are far from the center frequency of the radar transmitting antenna;

[0020] Step 2.5, removing background noise from the grayscale image after data conversion to further increase the effective signal;

[0021] Step 2.6, perform sliding average filtering on the grayscale image to remove irregular burst noise and burr noise;

[0022] Step 2.7, classifying and marking the ground penetrating radar grayscale image;

[0023] Step 2.8, scale all the ground penetrating radar grayscale images to a uniform resolution, and normalize the pixel values ​​to [0, 1].

[0024] According to a specific implementation of the embodiment of the present disclosure, the residual network model includes a plurality of building blocks stacked in series, wherein the building blocks include a convolutional layer, an activation layer, a Batch Normalization layer, and two propagation paths;

[0025] The bypass of the building block is a direct connection structure, which transmits the input directly to the end of the building block and outputs it after merging with the main path.

[0026] According to a specific implementation of the embodiment of the present disclosure, before step 3, the method further includes:

[0027] For the convolution grouping in the original residual network model, the multiple convolution kernels that originally extracted features with a step size of n are divided into multiple groups, which are used to re-extract features in the skipped areas on the feature map, and then the results of each group of convolutions are stacked into a new feature map, where n is a positive integer greater than 2, and the output calculation formula is as follows:

[0028]

[0029] In the formula, represents the nth group of the first convolution layer in the building block, represents the nth group of the bypass first convolution layer in the building block, W2 represents the second convolution layer of the main path in the building block, and W s2 represents the bypassed second convolution layer in the building block;

[0030] Adding a convolutional layer to the branch of the dimension reduction building block increases the dimension of the feature map of the branch and makes it the same as the feature map of the main path;

[0031] In the dimensionality reduction building block, bypass dimensionality increase and group convolution are combined to construct a new building block - regional dimension matching structure to obtain the optimized residual network model.

[0032] According to a specific implementation of the embodiment of the present disclosure, step 3 specifically includes:

[0033] Step 3.1, inputting the ground penetrating radar grayscale image into the optimized residual network model, and enhancing the richness of the ground penetrating radar grayscale image by rotating, flipping, translating, adding noise, scaling, random occlusion and brightness adjustment operations or composite operations on the radar grayscale image samples;

[0034] Step 3.2, input the enhanced ground penetrating radar grayscale image into multiple groups of building blocks in sequence, and output a feature map;

[0035] Step 3.3, inputting the feature map into the global pooling layer;

[0036] Step 3.4, input the output of the global pooling layer into a fully connected layer containing 4 neurons to obtain the recognition result.

[0037] The automatic classification scheme of ground penetrating radar images in the embodiment of the present disclosure includes: step 1, scanning concrete by transmitting radar waves by ground penetrating radar, and the receiver in the radar captures multiple echo data to form a scanning echo data set; step 2, preprocessing the scanning echo data, eliminating noise signals in the scanning echo data set and enhancing the echo signals of deep targets, so as to obtain a ground penetrating radar grayscale image with clear textual information; step 3, inputting the ground penetrating radar grayscale image into the optimized residual network model, and outputting the defect category corresponding to the image as the recognition result after calculation; step 4, adding the recognition result to the data set and updating the residual network model if it is accurate after actual verification in the later stage.

[0038] The beneficial effects of the embodiments of the present disclosure are as follows: through the scheme of the present disclosure, the collected data is normalized and the convolutional neural network is improved to make it more suitable for radar image recognition, thereby improving the accuracy and efficiency of concrete internal defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 A schematic diagram of a flow chart of a method for automatic classification of ground penetrating radar images provided in an embodiment of the present disclosure;

[0041] Figure 2 A diagram of the original building block structure of a residual network provided in an embodiment of the present disclosure;

[0042] Figure 3 A regional dimension matching structure diagram provided in an embodiment of the present disclosure;

[0043] Figure 4 A flow chart of detecting internal defects of concrete using a residual network based on regional dimension matching provided in an embodiment of the present disclosure;

[0044] Figure 5 A process diagram of an example of a preprocessing process provided by an embodiment of the present disclosure;

[0045] Figure 6 An example diagram of sample data enhancement provided by an embodiment of the present disclosure;

[0046] Figure 7 An example diagram of a disease sample provided by an embodiment of the present disclosure;

[0047] Figure 8 A diagram of the training process of a residual network detection model based on regional dimension matching provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0048] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0049] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0050] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0051] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0052] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0053] The disclosed embodiment provides a method for automatic classification of ground penetrating radar images, which can be applied to the process of detecting internal concrete structure defects in scenes such as roads, bridges, and tunnels.

[0054] See also Figure 1 , is a flow chart of a method for automatic classification of ground penetrating radar images provided by an embodiment of the present disclosure. Figure 1 As shown, the method mainly comprises the following steps:

[0055] Step 1: The ground penetrating radar transmits radar waves to scan the concrete, and the receiver in the radar captures multiple echo data to form a scanning echo data set;

[0056] Optionally, the step 1 specifically includes:

[0057] The ground penetrating radar transmitter continuously emits radar waves of a preset fixed center frequency as the radar moves, and the receiver receives multiple channels of single-column echo data and splices the multiple channels of the single-column echo data to form the scanning echo data.

[0058] During specific implementation, the model training of the recognition method needs to be carried out in the data set, so the data set must be established first. Considering that there are few public data sets and it is actually difficult to collect internal defects of concrete in batches, a variety of schemes are used to jointly build a data set. For example, the ground penetrating radar transmitter continuously emits radar waves with a preset fixed center frequency as the radar moves, and the receiver receives multiple single-column echo data, and splices the multiple single-column echo data to form the scanning echo data. At the same time, the source of the data set can also include on-site collection (highways, tunnels, etc. containing concrete components), test piece collection, forward simulation (such as Figure 5 As shown in the dotted box) and data enhancement processing is performed on the above data, such as Figure 6 As shown, Figure 6 Figure a is the original image, and the rest are enhanced data. The test piece is made by mixing cement and crushed sand and pouring them into a standard mold. During the production process, soil blocks, small piles of sandbags, empty plastic bottles, and plastic bottles filled with some water are added to the mold to cause artificial defects. If no additions are made, it is a normal test piece without defects. The radar grayscale image containing various defects can be obtained by scanning the test piece with ground penetrating radar. In addition, the forward simulation is realized by the open source software grrMax, and the simulation image is obtained by adding materials with different properties and shapes as defect samples in the forward model.

[0059] Step 2, preprocessing the scanning echo data, removing noise signals in the scanning echo data set and enhancing echo signals of deep targets to obtain a ground penetrating radar grayscale image with clear textural information;

[0060] Furthermore, the step 2 specifically includes:

[0061] Step 2.1, removing DC drift from the scan echo data;

[0062] Step 2.2, remove the horizontal signal in the data after DC drift removal, and cut the data by the depth corresponding to the peak value of the echo waveform after static correction processing;

[0063] Step 2.3, the data after the horizontal signal noise is removed is subjected to gain processing, and the amplitude of the deep signal is amplified by the gain to enhance the texture features in the grayscale image;

[0064] Step 2.4, bandpass filtering is performed on the amplified data to remove low-frequency and high-frequency noise signals that are far from the center frequency of the radar transmitting antenna;

[0065] Step 2.5, removing background noise from the grayscale image after data conversion to further increase the effective signal;

[0066] Step 2.6, perform sliding average filtering on the grayscale image to remove irregular burst noise and burr noise;

[0067] Step 2.7, classifying and marking the ground penetrating radar grayscale image;

[0068] Step 2.8, scale all the ground penetrating radar grayscale images to a uniform resolution, and normalize the pixel values ​​to [0, 1].

[0069] In the specific implementation, considering that based on the acquired samples, the samples are preprocessed first, the preprocessing order and the value of each step will affect the results. The specific processing steps can be as follows:

[0070] Y01: Remove the overall DC offset of the sample. This example uses the solution of subtracting the average of the entire echo data from each data. Figure 5 As shown in a, the result after removing DC drift is as follows Figure 5 As shown in b.

[0071] Y02: Remove horizontal texture from the data after drift removal. In this example, the horizontal signal band is removed by static correction method, and the depth corresponding to the second peak waveform after the start is selected for cutting, and the second half of the second peak is retained. Figure 5 As shown in c.

[0072] Y03: Perform gain processing after removing the horizontal signal noise. There are many gain processing methods, such as (AGC gain, energy attenuation gain, manual control gain, etc.). This example applies energy attenuation. The attenuation curve is calculated according to the waveform of the data after step P01, and the attenuation curve is reversely referenced to the entire echo, so that the amplitude of the reflected echo of the deep object is expanded to be equivalent to the amplitude of the reflected echo of the shallow object, while the amplitude of the shallow object does not change much. In order to prevent the amplitude value of some echoes from being too large and exceeding the range [0,255] after gain, it is necessary to set the scaling factor to control the overall waveform. The scaling factor is related to the maximum peak value in the echo, and the value range is [0,1]. In this example, the scaling factor is 0.41. The texture information in the image increases after gain, such as Figure 5 As shown in middle d.

[0073] Y04: The data after gain is further processed by bandpass filtering. In this example, the center frequency of the ground penetrating radar antenna is 300MHz. The data after gain is Fourier transformed to obtain a spectrum diagram. The upper and lower boundary frequencies are set according to the condition that the power is large and concentrated near 300MHz. The closer the upper boundary is set, the more severe the effective signal is filtered out. Conversely, the farther the setting is, the less early shots are filtered out. In this example, the lower boundary frequency is selected as 167.3203MHz and the upper boundary frequency is selected as 585.6208MHz for bandpass filtering. After filtering, Figure 5 As shown in figure e.

[0074] Y05: After the bandpass filter, the global background is filtered. The Arctic filter can further remove the subtle horizontal texture signal in the deep layer. The method used in this example is: first calculate the average echo data of the multi-channel echo data in the whole image; then subtract the average echo data from each echo data. The processing result is as follows: Figure 5 As shown in figure f.

[0075] Y06: Use sliding mean filtering on the image. After background filtering, the image still contains burst single point noise. Using sliding mean filtering can reduce the impact of such noise. A filter with too large a size is likely to blur the image. This example uses a filter with a size of 2 for filtering. The results are as follows: Figure 5 As shown in g.

[0076] Y07: After the above preprocessing steps are completed, different types of defect data are stored and marked separately, and all the ground penetrating radar grayscale images are scaled to a uniform resolution, and the pixel values ​​are normalized and converted to between [0, 1].

[0077] Step 3, inputting the ground penetrating radar grayscale image into the optimized residual network model, and outputting the defect category corresponding to the image as the recognition result after calculation;

[0078] Optionally, the residual network model includes a plurality of building blocks stacked in series, wherein the building blocks include a convolutional layer, an activation layer, a Batch Normalization layer, and two propagation paths;

[0079] The bypass of the building block is a direct connection structure, which transmits the input directly to the end of the building block and outputs it after merging with the main path.

[0080] Furthermore, before step 3, the method further includes:

[0081] For the convolution grouping in the original residual network model, the multiple convolution kernels that originally extracted features with a step size of n are divided into multiple groups, which are used to re-extract features in the skipped areas on the feature map, and then the results of each group of convolutions are stacked into a new feature map, where n is a positive integer greater than 2, and the output calculation formula is as follows:

[0082]

[0083] In the formula, represents the nth group of the first convolution layer in the building block, represents the nth group of the bypass first convolution layer in the building block, W2 represents the second convolution layer of the main path in the building block, and W s2 represents the bypassed second convolution layer in the building block;

[0084] Adding a convolutional layer to the branch of the dimension reduction building block increases the dimension of the feature map of the branch and makes it the same as the feature map of the main path;

[0085] In the dimensionality reduction building block, bypass dimensionality increase and group convolution are combined to construct a new building block - regional dimension matching structure to obtain the optimized residual network model.

[0086] Based on the above embodiment, step 3 specifically includes:

[0087] Step 3.1, inputting the ground penetrating radar grayscale image into the optimized residual network model, and enhancing the richness of the ground penetrating radar grayscale image by rotating, flipping, translating, adding noise, scaling, random occlusion and brightness adjustment operations or composite operations on the radar grayscale image samples;

[0088] Step 3.2, input the enhanced ground penetrating radar grayscale image into multiple groups of building blocks in sequence, and output a feature map;

[0089] Step 3.3, inputting the feature map into the global pooling layer;

[0090] Step 3.4, input the output of the global pooling layer into a fully connected layer containing 4 neurons to obtain the recognition result.

[0091] In the specific implementation, it is considered that the residual network is a type of convolutional neural network, which has the advantages of easy structural adjustment, not easy to overfit, and preventing the performance degradation of deep networks. Its characteristic is that it can be composed of multiple building blocks stacked in series. The building block consists of a convolutional layer, an activation layer, and a Batch Normalization (BN) layer, which contains two propagation paths. The building block bypass is a direct connection structure, which transmits the input directly to the end of the building block and outputs it after fusion with the main path. In the reduced-dimensional building block, a convolutional layer is also used on the bypass to match the size of the main path output. The structure diagram is shown in the figure. Figure 2 , the building block output is calculated as follows:

[0092] O i =f(H(x i ))=f(F(x i )+x i )

[0093] In the formula, x i and O i are the input and output of the ith building block, respectively, and F(x i ) and H(x i ) are the mappings before and after the fusion of the two paths, F(x i ) is also what the network needs to learn, which can be further expressed as follows:

[0094] O i =f(F(xi W i )+x i ),F=W2f(W1x i )

[0095] Where f is the activation function ReLu, W1 and W2 are the weight parameters of the first and second layers in the building block respectively.

[0096] As for the building blocks of dimensionality reduction, they are further expressed as follows:

[0097] O i =f(F(x i W i )+x i W s ),F=W2f(W1x i )

[0098] Where W s Used to adjust the size of the output feature maps of the two paths to be the same.

[0099] Each column of pixels in the grayscale image of the ground penetrating radar represents a complete echo received, carrying information about the target. Adjacent image columns are echoes after the ground penetrating radar moves a short distance, and have certain similarities. Multiple columns of pixels can reflect trend changes. A dimensionality reduction building block is inserted every certain number of building blocks in the residual network. The convolution kernel in the dimensionality reduction building block uses a step size of n for downsampling and dimensionality reduction, where n is generally 2, and can also be an integer of n>2 for greater dimensionality reduction. The dimensionality reduction process can reduce the overall amount of calculation and the occurrence of overfitting, but it will also cause the situation of skimming the entire column of pixels. The entire column of pixels is of great significance to the ground penetrating radar image, so it will reduce the accuracy of the model. In addition, the more convolutional layers the image passes through, the higher the dimension of its feature map, and the better the generalization performance of the extracted features. The number of bypass convolutional layers in the dimensionality reduction building block is less than the number of main convolutional layers, and the dimension of the feature map is different, so the feature enhancement is not obvious after fusion.

[0100] In view of the shortcomings of the above-mentioned original residual network in the application of ground penetrating radar images, the present invention will better solve the above problems by introducing convolution grouping and bypass dimensionality increase scheme.

[0101] Furthermore, for convolution grouping, the multiple convolution kernels that originally extracted features with a step size of 2 are divided into multiple groups, which are used to re-extract features from the skipped areas on the feature map, and then the results of each group of convolutions are stacked into a new feature map. The output calculation formula is as follows:

[0102]

[0103] In the formula, is the nth group of the first convolution layer in the building block, is the nth group of the bypass first convolution layer in the building block; W2 is the second convolution layer in the main path of the building block, W s2 is the bypassed second convolution layer in the building block.

[0104] Furthermore, in order to take advantage of the high dimensionality and the mismatch between the two-way dimensions of the dimensionality reduction building block, a convolutional layer is added to the branch of the dimensionality reduction building block so that the feature map dimension of the branch is increased and the feature map of the main path is the same. After that, the two feature maps are added and activated together so that the output features are strengthened. The output calculation formula is as follows:

[0105] O i =f(W2f(W1x i )+W s2 f(W s1 x i ))

[0106] Where W1 is the first convolution layer in the main path of the building block, and W s1 The first convolutional layer in the building block is bypassed.

[0107] Based on the above method, the present invention combines bypass dimension increase and group convolution in the dimension reduction building block to construct a new building block - regional dimension matching structure, such as Figure 3 As shown, the output calculation formula is as follows:

[0108]

[0109] At the same time, considering that the grayscale image samples of the concrete obtained by the above three schemes still lack balance and diversity, data enhancement can enrich the number of various defects and improve the generalization of the samples as a whole. The internal defects of concrete in this example are divided into four categories: voids, water voids, bulk soil, and bulk sandbags. Figure 7 As shown. Data enhancement can be done in the following ways:

[0110] H01: Rotate each defect sample image, and the rotation angle of each sample is randomly selected in the range of (0-360°)

[0111] H02: Perform flip processing on each defect sample image, and each sample is randomly flipped horizontally or vertically.

[0112] H03: Perform translation processing on each defect sample image, taking the upper right corner of each sample as the coordinate origin and randomly moving to the other coordinates.

[0113] H04: Scale each defect sample image, and each sample is randomly assigned a scaling factor α, α∈(0,1) indicates reduction, and α>1 indicates enlargement. Reduction is achieved through downsampling, and enlargement is achieved through upsampling experiments. In this example, bilinear interpolation is used. Too large or too small scaling factors will invalidate the image texture features. In this example, the scaling factor range is α∈(0.8,1.5).

[0114] H05: Perform random occlusion processing on each defect sample image, taking the upper right corner of each sample as the coordinate origin, randomly select the rectangular position in the image, and randomly refill the pixels in the rectangle. A rectangular frame that is too large will cover the original image and make the image invalid, and a rectangular frame that is too small will have limited occlusion effect. In this example, the side length of the rectangular frame ranges from 20 to 50 pixels.

[0115] H06: Noise processing is added to each defect sample image. The noise is randomly selected from Gaussian noise or salt and pepper noise. In this example, the Gaussian noise has a mean of 0 and a standard deviation range of [1,5]. The density of the salt and pepper noise is in the range of [0.1,0.5].

[0116] H07: Contrast adjustment is performed on each defect sample image. The contrast represents the difference between the brightest pixel concentration area and the darkest pixel concentration area in the image. The size of the difference reflects the size of the contrast. Too large or too small contrast will distort the image information. In this example, the contrast value range is [0.7,2].

[0117] H08: The brightness of each defect sample image is adjusted. Brightness is the brightness of the image, which is represented by the pixel grayscale value. A pixel value close to 0 indicates low brightness, and vice versa indicates high brightness. In this example, the grayscale value of the ground penetrating radar grayscale image is in the range of [0,255]. The brightness adjustment coefficient ranges from [0.7,2], and the bias is 10.

[0118] Among them, the brightness adjustment method is:

[0119] h(i,j)=k*f(i,j)+b

[0120] Among them, f(i,j) is the initial pixel value at (i,j), h(i,j) is the pixel value output after adjustment, k is the adjustment coefficient, and b is the bias, which can make all pixel values ​​change uniformly without changing the contrast.

[0121] H09: Randomly select a part of the image obtained after the operations H01-H08, and randomly perform one of the operations H01-H08 on each image. In this example, 50% of them are selected for secondary enhancement operations, such as Figure 6 Shown in the bottom dashed box.

[0122] The data after data enhancement forms a data set for training classification models. This example takes an 18-layer regional dimension matching residual network as an example. This network is optimized for ground penetrating radar image transformation based on an 18-layer residual network and is composed of 8 building blocks. Building blocks can be divided into dimensionality reduction building blocks and non-dimensionality reduction building blocks. The dimensionality reduction building blocks contain convolution layers with a convolution step of 2, and the convolution step of non-dimensionality reduction building blocks is 1. Every two building blocks can be divided into a group. Each group of building blocks has the same structure and uses the same number of convolution kernels. The main convolution kernel is 3*3 in size, and the bypass convolution kernel in the dimensionality reduction building block is 1*1 in size. After two building blocks, the number of convolutions doubles. The data goes through the following steps in this model:

[0123] T01: Image data is input into the network training in batches. In this example, 64 images are selected as a batch input. After input into the network, the preprocessing part of the network further processes the data accordingly to obtain standardized and unified training data. This includes scaling the image to a uniform size of M*N (in this example, M and N are both 128) and normalizing all pixels to between 0 and 1.

[0124] T02: The normalized image is firstly extracted by a set of 64 3*3 convolution kernels, and then activated and output by the ReLu function after passing through the normalization layer. The normalization layer here normalizes the same channel of a batch of images.

[0125] T03: Data is input into the first set of building blocks. The first building block receives the feature map after preliminary convolution. The building block consists of a direct path and a main path. The main path contains two convolution layers. The convolution layer uses 64 3*3 convolution kernels with a convolution step of one. Each convolution layer is followed by a BN layer. The first BN layer is followed by a ReLu activation layer. The output of the second BN layer is added and fused with the direct layer, and then passed through the second activation layer and output to the next set of building blocks. The second building block is the same as the first one, and the output feature image size is (B, M, N, C) (the size of this example is (64, 128, 128, 64)), where B represents the number of feature maps input in batches, M and N represent the size of the feature image, and C represents the depth of the feature map.

[0126] T04: The data enters the second group of building blocks. The third building block is the dimensionality reduction building block. The first convolution layer of the main path uses 128 3*3 convolution kernels, which are divided into 4 groups, each with 32 convolution kernels. The convolution starts at the intersection of the [1,3]th row and [1,3]th column, the [1,3]th row and [2,4]th column, the [2,4]th row and [1,3]th column, and the [2,4]th row and [2,4]th column of the input feature map (64,128,128,64). The convolution step is 2, as shown in the figure. Figure 3As shown in P1, P2, P3, and P4 in . The four sets of feature maps after feature extraction by four sets of convolution kernels have the same size of (64, 64, 64, 32). The new feature map size obtained by stacking the four sets of maps is (64, 64, 64, 128), denoted as F31. The output feature map size of F31 remains unchanged after passing through the BN layer, denoted as F31B. At the same time, the first layer of convolution in the bypass also uses 4 sets of 1*1 convolution kernels, totaling 128. The actual positions of each set of convolution kernels on the input feature map are the 1st row and 1st column, the 1st row and 2nd column, the 2nd row and 1st column, and the 2nd row and 2nd column, respectively. The stride of the convolution kernel is 2, as shown in Figure 3 As shown in Q1, Q2, Q3, and Q4. After bypassing one convolution, the size of the four groups of feature maps is (64, 64, 64, 32), and the size after stacking is (64, 64, 64, 128), denoted as M31. Then, the BN layer is used to normalize it to obtain M31B, and its size has not changed. The output of the first BN layer of the two paths is fused, that is, F31B is added to M31B, and the output size is (64, 64, 64, 128), denoted as F31S. The feature map after the initial fusion continues to propagate in the second convolution layer of the main path, and the features are extracted by 128 3*3 convolution kernels, and then continue to propagate into the second BN layer of the main path. The output is denoted as F32B, and its size has not changed. The bypassed second convolution layer extracts features by 128 1*1 convolution kernels, and the output of the second BN layer is denoted as M32B, and the size remains unchanged. From the above, the output of the third building block is: the output of the main path and the bypass after the second passing through the BN layer is added and fused and then activated by ReLu, that is, f(F32B+M32B).

[0127] The input data of the fourth building block is the output of the previous building block, that is, f(F32B+M32B). Its structure is the same as the first building block, except that the number of convolution kernels used is doubled to 128, and the output size is (64, 64, 64, 128).

[0128] T05: The data enters the third group of building blocks. The overall structure of this group of building blocks is the same as that of the second group. The number of convolution kernels used is increased to 256. The data is propagated internally with reference to the previous group of building blocks. The output feature map size is: (64, 32, 32, 256).

[0129] T06: The data enters the fourth group of building blocks, which is the last group of building blocks. The structure is the same as the second group of building blocks. The input feature map propagation process within it is the same as the second group of building blocks. The number of convolution kernels used in all convolutional layers reaches 512, and the output feature map size is (64, 16, 16, 256).

[0130] T07: The feature map propagates into the global pooling layer. The feature map is output from the last building block and then enters the global pooling layer. The global pooling layer takes the average value of the depth of each layer of the feature map to replace the original layer. Therefore, the feature map of size (16,16) is global pooled to size (1,1). In this example, the output data size after global pooling is (128,1,1,512).

[0131] T08: Finally, the recognition result is output through a fully connected layer containing 4 neurons.

[0132] Step 4: If the recognition result is actually verified to be accurate in the later stage, the recognition result is added to the data set to update the residual network model.

[0133] When implemented, the steps can be as follows:

[0134] T09: After obtaining the prediction result, compare the result with the label value and calculate its loss. In this example, the cross entropy loss function is used. After obtaining the cross entropy loss, the weight parameters of each layer of the regional dimension matching network are reversely updated through the Adam optimizer at a learning rate of 0.001.

[0135] T10: Update the parameters of the entire network. The next batch of training data continues to be input into the network from the beginning, and repeat steps T01-T09. In this example, the number of training cycles is 800. After training, a detection model is generated for actual classification of ground penetrating radar images of concrete defects. The training process is as follows: Figure 8 As shown, (a) is the accuracy graph of the training process, and (b) is the loss graph of the training process.

[0136] E01: The network model trained in this example is obtained through the above steps. In actual application, the B-Scan image is first formed by using ground penetrating radar to scan concrete structures such as roads, tunnels, and bridges to obtain echo data. The collected image goes through steps Y01-Y07 and is input into the model formed in T10. In the model, the image goes through steps T01-T08 to obtain the defect prediction classification of the ground penetrating radar image.

[0137] E02: After the images collected in actual applications are recognized by the above method, they can be further added to the database. After the database is updated, the model can be trained and updated continuously, such as Figure 4 The route is shown by the dashed arrow.

[0138] The automatic classification method for ground penetrating radar images provided in this embodiment improves the accuracy and efficiency of concrete internal defect detection by normalizing the collected data and improving the convolutional neural network to make it more suitable for radar image recognition.

[0139] It should be understood that various parts of the present disclosure may be implemented in hardware, software, firmware or a combination thereof.

[0140] The above is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present disclosure should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A method for automatic classification of ground penetrating radar images, characterized in that: include: Step 1: Scanning the concrete by transmitting radar waves through a ground penetrating radar, and a receiver in the radar captures multiple echo data to form a scanning echo data set; Step 2, preprocessing the scanning echo data, removing noise signals in the scanning echo data set and enhancing echo signals of deep targets to obtain a ground penetrating radar grayscale image with clear textural information; Step 3, inputting the ground penetrating radar grayscale image into the optimized residual network model, and outputting the defect category corresponding to the image as the recognition result after calculation, wherein the residual network model includes a plurality of building blocks stacked in series, and the building blocks include a convolution layer, an activation layer, and a Batch Normalization layer, which contain two propagation paths; The bypass of the building block is a direct connection structure, which transmits the input directly to the end of the building block and outputs it after merging with the main path; Before step 3, the method further includes: For the convolution grouping in the original residual network model, the multiple convolution kernels that originally extracted features with a step size of n are divided into multiple groups, which are used to re-extract features in the skipped areas on the feature map, and then the results of each group of convolutions are stacked into a new feature map, where n is a positive integer greater than 2, and the output calculation formula is as follows: In the formula, represents the nth group of the first convolution layer in the building block, represents the nth group of the bypass first convolution layer in the building block, W2 represents the second convolution layer of the main path in the building block, and W s2 represents the bypassed second convolution layer in the building block; Adding a convolutional layer to the branch of the dimension reduction building block increases the dimension of the feature map of the branch and makes it the same as the feature map of the main path; In the dimensionality reduction building block, bypass dimensionality increase and group convolution are combined to construct a new building block - regional dimension matching structure, and the optimized residual network model is obtained; Step 4: If the recognition result is actually verified to be accurate in the later stage, the recognition result is added to the data set to update the residual network model.

2. The method according to claim 1, characterized in that , the step 1 specifically includes: The ground penetrating radar transmitter continuously emits radar waves of a preset fixed center frequency as the radar moves, and the receiver receives multiple channels of single-column echo data and splices the multiple channels of the single-column echo data to form the scanning echo data.

3. The method according to claim 1, characterized in that , the step 2 specifically includes: Step 2.1, removing DC drift from the scan echo data; Step 2.2, remove the horizontal signal in the data after DC drift removal, and cut the data by the depth corresponding to the peak value of the echo waveform after static correction processing; Step 2.3, the data after the horizontal signal noise is removed is subjected to gain processing, and the amplitude of the deep signal is amplified by the gain to enhance the texture features in the grayscale image; Step 2.4, bandpass filtering is performed on the amplified data to remove low-frequency and high-frequency noise signals that are far from the center frequency of the radar transmitting antenna; Step 2.5, removing background noise from the grayscale image after data conversion to further increase the effective signal; Step 2.6, perform sliding average filtering on the grayscale image to remove irregular burst noise and burr noise; Step 2.7, classifying and marking the ground penetrating radar grayscale image; Step 2.8, scale all the ground penetrating radar grayscale images to a uniform resolution, and normalize the pixel values ​​to [0, 1].

4. The method according to claim 3, characterized in that , the step 3 specifically includes: Step 3.1, inputting the ground penetrating radar grayscale image into the optimized residual network model, and enhancing the richness of the ground penetrating radar grayscale image by rotating, flipping, translating, adding noise, scaling, random occlusion and brightness adjustment operations or composite operations on the radar grayscale image samples; Step 3.2, input the enhanced ground penetrating radar grayscale image into multiple groups of building blocks in sequence, and output a feature map; Step 3.3, inputting the feature map into the global pooling layer; Step 3.4, input the output of the global pooling layer into a fully connected layer containing 4 neurons to obtain the recognition result.

Images