A method and system for segmenting subsurface targets

Abstract

The underground target segmentation method and system can effectively detect various targets in a complex underground environment. The underground target segmentation method obtains more abundant target features by processing an image stream of B-Scan image processing and an original data stream of B-Scan original data processing through a multi-modal fusion module. Meanwhile, the B-Scan image is fused with the B-Scan original data as an auxiliary stream to guide the optimization of the original data stream. Then, multi-scale fusion and multi-level fusion are used to further strengthen the feature utilization of the B-Scan image and the B-Scan original data. The method combines the multi-stream attention mechanism of the two modes, and can effectively extract and fuse the original data and the preprocessed image information. The experimental results on multiple real images show that the segmentation method has better segmentation performance than other popular segmentation methods.

Description

Technical Field

[0001] This invention relates to ground-penetrating radar technology for detecting and identifying underground targets, and more particularly to a method and system for segmenting underground targets. Background Technology

[0002] The underground conditions of an airport runway have a significant impact on its health. Accurate detection of underground objects and defects (such as pipes, steel mesh, and voids) is a crucial aspect of routine airport maintenance. Ground-penetrating radar (GPR) is an effective method, and more importantly, it does not damage the runway surface. GPR actively emits electromagnetic signals into the ground; by analyzing and processing the reflected signals, it can non-destructively detect underground conditions.

[0003] GPR (Geometric Profiling) subsurface target detection methods based on preprocessed B-scan images typically involve preprocessing such as denoising, background removal, and gain adjustment before target detection is performed on the preprocessed image. With the development of deep learning, deep convolutional networks have achieved great success in computer vision and are increasingly being applied to preprocessed B-scan images. Preprocessed B-scan images, especially those with gain adjustment, can better represent the shape and texture of the target, but the relative intensity information in the depth direction is destroyed. Therefore, using preprocessed B-scan images alone cannot meet the detection requirements of complex subsurface targets. Summary of the Invention

[0004] The main objective of this invention is to propose a method for segmenting underground targets with better segmentation performance, aiming to solve the aforementioned problems existing in the prior art.

[0005] To achieve the above objectives, the present invention provides a method for segmenting underground targets, the method comprising:

[0006] Acquire raw B-Scan data and preprocessed B-Scan images;

[0007] The B-Scan image is processed to obtain a first multi-scale feature map;

[0008] The B-Scan image and the original B-Scan data are fused together. The positional information of the B-Scan image is used to guide the optimization of features in the original B-Scan data to obtain a second multi-scale feature map.

[0009] Multimodal fusion is performed on the first multi-scale feature map and the second multi-scale feature map to obtain a series of multimodal fused feature maps;

[0010] Multi-scale fusion is performed on the first multi-scale feature map, the second multi-scale feature map, and the multi-modal fusion series feature map to obtain a multi-scale fusion series feature map;

[0011] By aggregating the multi-scale fused feature maps, a higher-order feature map is obtained;

[0012] The high-order feature map, the first multi-scale feature map, and the second multi-scale feature map are fused at multiple levels to obtain a multi-level fusion output result. The multi-level fusion output result is then processed to obtain a segmentation result.

[0013] In some embodiments, the step of processing the B-Scan image to obtain a first multi-scale feature map includes:

[0014] The B-Scan image is downsampled by five convolutional layers, and each convolutional layer outputs feature maps g1, g2, g3, g4, and g5 in sequence. The feature maps g1, g2, g3, g4, and g5 constitute the first multi-scale feature map.

[0015] The step of fusing the B-Scan image and the original B-Scan data, and using the positional information of the B-Scan image to guide the optimization of parameters in the original B-Scan data to obtain the second multi-scale feature map includes:

[0016] The B-Scan image is subjected to five-layer pooling to obtain a multi-scale pooling feature map;

[0017] The B-Scan raw data is processed in five layers. The intermediate feature map after Vgg block processing is fused with the pooling feature map of the same layer and output as the input of the next layer. Each layer outputs feature maps s1, s2, s3, s4 and s5 in sequence. The feature maps s1, s2, s3, s4 and s5 constitute the second multi-scale feature map.

[0018] In some embodiments, the multimodal fusion series feature map is F n :

[0019] F n-1 =4-fusion(s) n ,s n+1 ,g n ,g n+1 ), n∈{2,3,4},

[0020] Here, 4-fusion means fusing four different feature map inputs.

[0021] In some embodiments, the multi-scale fused series feature map is D k k∈{1,2,3,4}:

[0022]

[0023] Here, 3-fusion means fusing three different feature map inputs.

[0024] In some embodiments, the step of performing multi-level fusion of the higher-order features, the first multi-scale feature map, and the second multi-scale feature map to obtain a multi-level fusion output result includes:

[0025] The higher-order feature is δ1, where δ1 = 4-fusion(D1,D2,D3,D4);

[0026]

[0027] G i =concat(3-fusion(g 3-i ,s 3-i ,k i ),k i ), i∈{1,2},

[0028] Among them, k3 is the multi-level fusion output result.

[0029] In some embodiments, in the step of fusing the intermediate feature map after Vgg block processing of the input of each layer with the pooled feature map of the same layer and outputting the feature map as the input of the next layer:

[0030] intermediate feature maps of at least some layers Pooling feature map of the same layer The shallow attention module fuses and outputs the feature map as the input for the next layer of processing, where C is the number of channels. The working process of the shallow attention module is as follows:

[0031] The intermediate feature map F is sequentially processed through pooling, a multilayer perceptron (MLP), and element-wise recombination to obtain the channel attention. Extended M c Multiply by F to get

[0032] F c (c,h,w)=F(c,h,w)×M c (c);

[0033] The pooling feature map G is processed by convolution to obtain the intermediate layer. The intermediate layer G m The spatial attention map is obtained after convolution processing. Extended M s Multiplying by F yields

[0034] F s(c,h,w)=F(c,h,w)×M s (h,w);

[0035] Output the final result F':

[0036] F'=aF s +(1-a)F c , where a is the weight parameter.

[0037] In some embodiments, the intermediate feature map F is sequentially processed through pooling, a multilayer perceptron (MLP), and element-wise recombination to obtain channel attention. The steps include:

[0038] The intermediate feature map F is obtained by performing max pooling and average pooling. avg and F max ,in Unified representation as

[0039] Using a multilayer perceptron (MLP) from F m Features are extracted from each element, and channel attention is obtained through element-wise recombination.

[0040] In some embodiments, in the step of fusing the intermediate feature map after Vgg block processing of the input of each layer with the pooled feature map of the same layer and outputting the feature map as the input of the next layer:

[0041] intermediate feature maps of at least some layers Pooling feature map of the same layer The deep attention module fuses and outputs feature maps, which are then used as input for the next layer of processing. The deep attention module works as follows:

[0042] The intermediate feature map F is expanded by convolution to... The pooling feature map G is expanded by convolution to...

[0043] F′ c With F′ c Multiplying by the transposes yields the channel relation matrix.

[0044] Obtain feature map

[0045]

[0046] Where α is initially 0 and can be trained;

[0047] G′ s transpose of G′s Multiplying them together yields the spatial relation matrix.

[0048] Obtain feature map

[0049]

[0050] Where β is initially 0 and can be trained;

[0051] To obtain the final output result F':

[0052] F'=λF+γG s +(1-λ-γ)F c

[0053] Where λ and γ are weighting parameters.

[0054] In some embodiments, the shallow attention module is used to fuse the intermediate feature maps of the first, second, and fifth layers with the corresponding pooling feature maps;

[0055] The deep attention module is used to fuse the intermediate feature maps of the 3rd and 4th layers with the corresponding pooled feature maps.

[0056] In addition, to achieve the above objectives, the present invention also provides an underground target segmentation system, the system comprising modules for implementing the methods described in any of the preceding claims.

[0057] The underground target segmentation method and system proposed in this invention obtains richer target features by using a multi-modal fusion module to fuse the image stream from B-Scan image processing and the raw data stream from B-Scan raw data processing. Simultaneously, the B-Scan image is fused with the raw B-Scan data as an auxiliary stream to guide feature optimization in the raw data stream. Then, multi-scale fusion and multi-level fusion are used to further enhance the feature utilization of the B-Scan image and raw B-Scan data, thereby improving segmentation performance.

[0058] Furthermore, this invention introduces corresponding multi-stream attention modules, namely shallow attention modules and deep attention modules, between the original data stream and the auxiliary stream to guide the original data stream in feature learning. This enables the underground target segmentation method and system to fully utilize the global features of the original data and the local texture features of the B-Scan image to accurately segment various underground targets. Attached Figure Description

[0059] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0060] The methods, systems, and / or procedures shown in the accompanying drawings will be further described with reference to exemplary embodiments. These exemplary embodiments will be described in detail with reference to the drawings. These exemplary embodiments are non-limiting exemplary embodiments, wherein reference numerals in the various views of the drawings represent similar mechanisms.

[0061] Figure 1 This is a flowchart illustrating an underground target segmentation method according to some embodiments of this application;

[0062] Figure 2 This is a diagram illustrating the architecture of an underground target segmentation system based on some embodiments of this application;

[0063] Figure 3 This is a schematic diagram of the shallow attention module architecture involved in some embodiments of this application;

[0064] Figure 4 This is a diagram illustrating the deep attention module architecture involved in some embodiments of this application;

[0065] Figure 5 This is a comparison of the raw data and preprocessed B-Scan images of three typical underground targets in this application.

[0066] Figure 6 This is a comparison chart of the detection results for different combinations of deep attention and shallow attention in this application.

[0067] Figure 7 The image shows the detection diagrams of the method in this application and currently popular segmentation networks.

[0068] Figure 8 This is a comparison chart of the detection results of the method in this application and currently popular segmentation networks. Detailed Implementation

[0069] To better understand the above technical solutions, the technical solutions of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solutions of this application, rather than limitations on the technical solutions of this application. In the absence of conflict, the embodiments of this application and the technical features in the embodiments can be combined with each other.

[0070] In the detailed description below, numerous specific details are illustrated with examples to provide a comprehensive understanding of the relevant guidance. However, it will be apparent to those skilled in the art that this application can be practiced without these details. In other instances, well-known methods, procedures, systems, components, and / or circuits have been described at a relatively high level without detail to avoid unnecessarily obscuring aspects of this application.

[0071] These and other characteristics, the functions disclosed in the current application, the methods of execution, the functions of related elements in the structure, the combination of components, and the economic efficiency of production, may become more apparent in consideration of the following description with reference to the accompanying drawings, all of which form part of this application. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this application. It should be understood that these drawings are not drawn to scale.

[0072] This application uses flowcharts to illustrate the execution process performed by a system according to embodiments of this application. It should be clearly understood that the execution processes in the flowcharts may not be executed sequentially. Instead, these execution processes may be executed in reverse order or simultaneously. Additionally, at least one other execution process may be added to the flowchart. One or more execution processes may be deleted from the flowchart.

[0073] This application utilizes a detection robot equipped with ground-penetrating radar to detect underground targets in the area to be detected. As the detection robot moves in a straight line to perform its detection task, the ground-penetrating radar's antenna transmitter radiates electromagnetic waves onto the ground at preset intervals. The receiver then collects signals reflected from underground objects or geological interfaces, obtaining two-dimensional B-Scan raw data. By performing signal processing operations such as filtering on the B-Scan raw data, a B-Scan image is obtained. In existing technologies, the underground target identification result is obtained by analyzing and processing the B-Scan image.

[0074] Please refer to Figure 1 , Figure 1 This is a flowchart of an underground target segmentation method according to some embodiments of this application, which may specifically include the following steps S100-S700.

[0075] Step S100: Obtain the raw B-Scan data and the preprocessed B-Scan image.

[0076] Step S200: Process the B-Scan image to obtain the first multi-scale feature map.

[0077] The detection robot acquires raw B-Scan data of the underground area to be detected and processes it to obtain B-Scan images. In this embodiment, after acquiring the B-Scan images, the images are input into a multi-layer neural network for feature extraction. Each layer of the neural network outputs feature maps of different scales to obtain the first multi-scale feature map. This path to obtain the first multi-scale feature map can be referred to as an image stream.

[0078] Step S300: The B-Scan image and the original B-Scan data are fused. The positional information of the B-Scan image is used to guide the optimization of features in the original B-Scan data to obtain a second multi-scale feature map.

[0079] Since foreign objects, reflected clutter, and noise in radar signals can greatly interfere with the target representation in the raw B-Scan data, this embodiment uses pre-processed B-Scan images to help filter out noise.

[0080] In this embodiment, the B-Scan image path used to guide the B-Scan raw data can be called the auxiliary stream, and the second multi-scale feature map path can be called the raw data stream. In the auxiliary stream, due to the "incorrect" global comparison information caused by preprocessing operations, the positional information in the auxiliary stream is more important. In the raw data stream, global information is reflected in the channels, while local texture information is not obvious enough. Therefore, the raw data stream needs to emphasize the importance of channel information and use the auxiliary stream for feature optimization. This embodiment uses the positional information of the B-Scan image to guide the optimization of features in the B-Scan raw data, so that the raw data stream gradually focuses on the target region during the feature learning process.

[0081] Step S400: Perform multimodal fusion on the first multi-scale feature map and the second multi-scale feature map to obtain a series of multimodal fused feature maps.

[0082] In this embodiment, the first multi-scale feature map of the image stream and the second multi-scale feature map of the original data stream belong to different modalities. Therefore, the fusion between the two is a multi-modal fusion. During the multi-modal fusion process, at least some feature maps in the first multi-scale feature map are fused with at least some feature maps in the second multi-scale feature map to obtain multiple feature maps, that is, a series of multi-modal fused feature maps are obtained.

[0083] Step S500: Perform multi-scale fusion on the first multi-scale feature map, the second multi-scale feature map, and the multi-modal fusion series feature map to obtain a multi-scale fusion series feature map.

[0084] In this embodiment, the first multi-scale feature map, the second multi-scale feature map, and the multi-modal fusion series feature map have multiple scales. Therefore, the fusion of the three is multi-scale fusion, and after fusion, multiple feature maps are obtained, that is, the multi-scale fusion series feature map is obtained.

[0085] Step S600: Aggregate the multi-scale fusion series of feature maps to obtain a high-order feature map.

[0086] In this embodiment, all feature maps in the multi-scale fusion series feature maps are aggregated to obtain higher-order feature maps.

[0087] Step S700: Perform multi-level fusion on the high-order feature map, the first multi-scale feature map and the second multi-scale feature map to obtain a multi-level fusion output result, and process the multi-level fusion output result to obtain a segmentation result.

[0088] In this embodiment, the high-order feature map, the first multi-scale feature map, and the second multi-scale feature map are fused at least twice, i.e., multi-level fusion. Specifically, some feature maps from the high-order feature map, the first multi-scale feature map, and the second multi-scale feature map can be fused together. The resulting feature map is then fused with other feature maps from the first and second multi-scale feature maps.

[0089] Furthermore, other processing can be applied to the feature maps during the multi-level fusion process. For example, upsampling can be performed on the high-order feature maps before multi-level fusion, and the output feature maps of each layer of the multi-level fusion can be upsampled.

[0090] The feature map output by multi-level fusion contains segmentation results of underground targets from multiple channels. By processing, the segmentation results of different underground targets are displayed in the same image to obtain the final segmentation result.

[0091] Through the above steps S100-S700, the underground target segmentation method provided in this embodiment performs multimodal fusion of the image stream containing the B-Scan image and the original data stream containing the original B-Scan data to obtain richer target features; at the same time, the B-Scan image is used as an auxiliary stream and fused with the original B-Scan data to guide the optimization of features in the original data stream; then, multi-scale fusion and multi-level fusion are used to further enhance the feature utilization of the B-Scan image and the original B-Scan data, thereby improving the segmentation performance.

[0092] Figure 2 This is an architecture diagram of an underground target segmentation system according to some embodiments of this application. This underground target segmentation system can execute the underground target recognition method of any embodiment of this application. The following is in conjunction with... Figure 2 Some alternative embodiments are described below.

[0093] In some embodiments, step S200 includes:

[0094] In step S210, the B-Scan image is downsampled by five convolutional layers. Each convolutional layer sequentially outputs feature maps g1, g2, g3, g4, and g5, which constitute the first multi-scale feature map.

[0095] Step S300 includes:

[0096] Step S310: Perform five-layer pooling processing on the B-Scan image to obtain a multi-scale pooling feature map.

[0097] Step S320: Perform five-layer processing on the B-Scan raw data. The intermediate feature map after Vgg block processing of the input of each layer is fused with the pooling feature map of the same layer and output as the input of the next layer. Each layer outputs feature maps s1, s2, s3, s4, and s5 in sequence. The feature maps s1, s2, s3, s4, and s5 constitute the second multi-scale feature map.

[0098] In this embodiment, the B-Scan image undergoes five layers of pooling processing, with each layer yielding a corresponding pooling feature map, thus creating a multi-scale pooling feature map. In the first layer of processing of the original data stream, the original B-Scan data is processed by the first layer's Vgg block and then fused with the first layer's pooling feature map to obtain the first layer feature map s1. In the second layer processing, feature map s1 is processed by the second layer's Vgg block and then fused with the second layer's pooling feature map to obtain the second layer feature map s2. This process continues until the fifth layer feature map s5 is obtained.

[0099] In some embodiments, the multimodal fusion series feature map is F n :

[0100] F n-1 =4-fusion(s) n ,s n+1 ,g n ,g n+1 ), n∈{2,3,4},

[0101] Here, 4-fusion means fusing four different feature map inputs.

[0102] In this embodiment, the 4-fusion module is used for fusion between the image stream and the original data stream. Adjacent layers fuse adjacent position features from different streams through a 4-fusion module to obtain F. n .like Figure 2As shown, feature maps g2, g3 and s2, s3 are fused together by a 4-fusion module to obtain feature map F1; feature maps g3, g4 and s3, s4 are fused together by a 4-fusion module to obtain feature map F2; and feature maps g4, g5 and s4, s5 are fused together by a 4-fusion module to obtain feature map F3.

[0103] In some embodiments, the multi-scale fused series feature map is D k k∈{1,2,3,4}:

[0104]

[0105] Here, 3-fusion means fusing three different feature map inputs.

[0106] In this embodiment, four 3-fusion modules were used to obtain mixed features from different layers and locations, ultimately resulting in a multi-scale fusion result {D1, D2, D3, D4}. Figure 2 As shown, feature maps F1, F2, and F3 are fused together using a 3-fusion module to obtain feature map D1; feature maps F1, g4, and s4 are fused together using a 3-fusion module to obtain feature map D2; feature maps F2, g5, and s5 are fused together using a 3-fusion module to obtain feature map D3; and feature maps F3, g3, and s3 are fused together using a 3-fusion module to obtain feature map D4.

[0107] In some embodiments, the step of performing multi-level fusion of the higher-order features, the first multi-scale feature map, and the second multi-scale feature map to obtain a multi-level fusion output result includes:

[0108] The higher-order feature is δ1, where δ1 = 4-fusion(D1,D2,D3,D4);

[0109]

[0110] G i =concat(3-fusion(g 3-i ,s 3-i ,k i ),k i ), i∈{1,2},

[0111] Among them, k3 is the multi-level fusion output result.

[0112] In this embodiment, in order to compensate for the loss of spatial details caused by downsampling, a 3-fusion module is used to fuse high-order features with low-order features in the original data stream and image stream. concat is the fusion operation.

[0113] In some embodiments, the intermediate feature map after Vgg block processing of the input of each layer in step S320, fused with the pooled feature map of the same layer, and output as the input of the next layer, further includes:

[0114] intermediate feature maps of at least some layers Pooling feature map of the same layer The shallow attention module fuses and outputs the feature map as the input for the next layer of processing, where C is the number of channels. The working process of the shallow attention module is as follows:

[0115] The intermediate feature map F is sequentially processed through pooling, a multilayer perceptron (MLP), and element-wise recombination to obtain the channel attention. Extended M c Multiplying it by the intermediate feature map F yields...

[0116] F c (c,h,w)=F(c,h,w)×M c (c);

[0117] The pooling feature map G is processed by convolution to obtain the intermediate layer. The intermediate layer G m The spatial attention map is obtained after convolution processing. Extended M s Multiplying with the intermediate feature map F yields

[0118] F s (c,h,w)=F(c,h,w)×M s (h,w);

[0119] Output the final result F':

[0120] F'=aF s +(1-a)F c , where a is the weight parameter.

[0121] like Figure 3As shown, this embodiment uses a shallow attention module to transfer information from the auxiliary stream to the original data stream. The attention maps obtained from the image stream and the original data stream are multiplied by the input feature map of the original data stream, respectively, to adaptively refine the features. The shallow attention module is a lightweight module that can be placed between each layer in a multi-stream fusion architecture. The shallow attention module takes an intermediate feature map F from a certain layer and a pooled feature map G from the same layer as input. The intermediate feature map F represents the features of the original B-Scan data, and the pooled feature map G represents the features of the B-Scan image. The purpose of the shallow attention module is to obtain a two-dimensional spatial attention map through spatial attention and channel attention mechanisms. and one-dimensional channel attention map

[0122] For the raw data stream, this embodiment aims to emphasize the importance of channel information and generating channel attention maps. First, it compresses and aggregates spatial information using pooling operations to obtain a description of the channel information. Then, it uses a Multi-Layer Perceptron (MLP) to extract the features of the description. Finally, it obtains the channel attention map through element-wise recombination. c The expanded M c Multiply by F to get

[0123] F c (c,h,w)=F(c,h,w)×M c (c)

[0124] For the auxiliary flow, the focus is on learning spatial information and generating a spatial attention map. This embodiment first obtains the intermediate layer through convolution operations. Where C represents the number of channels, which is typically set to 2 in experiments. Subsequently, convolution is used to restore the number of channels to 1, forming the spatial attention map M. s That is, obtaining spatial feature description weights from the auxiliary flow channel. The extended M s Multiplying by F yields

[0125] F s (c,h,w)=F(c,h,w)×M s (h,w).

[0126] F s and F c Let F represent the spatial information and channel information in the intermediate feature map F, respectively. The final result F' is obtained by weighted summing of these two items. This process can be expressed as: F' = aF s +(1-a)F cWhere 'a' is the weight parameter. In experiments, to ensure that spatial and channel information are learned with equal importance, the weight parameter 'a' is usually set to 0.5.

[0127] Furthermore, in some embodiments, the intermediate feature map F is sequentially processed through pooling, a multilayer perceptron (MLP), and element-wise recombination to obtain channel attention. The steps include:

[0128] The intermediate feature map F is obtained by performing max pooling and average pooling. avg and F max ,in Unified representation as

[0129] Using a multilayer perceptron (MLP) from F m Features are extracted from each element, and channel attention is obtained through element-wise recombination.

[0130] In some embodiments, the intermediate feature map after Vgg block processing of the input of each layer in step S320, fused with the pooled feature map of the same layer, and output as the input of the next layer, further includes:

[0131] intermediate feature maps of at least some layers Pooling feature map of the same layer The deep attention module fuses and outputs feature maps, which are then used as input for the next layer of processing. The deep attention module works as follows:

[0132] The intermediate feature map F is expanded by convolution to... The pooling feature map G is expanded by convolution to...

[0133] F′ c With F′ c Multiplying by the transposes yields the channel relation matrix.

[0134] Obtain feature map

[0135]

[0136] Where α is the scale parameter, which has an initial value of 0 and can be trained;

[0137] G′ s transpose of G′ s Multiplying them together yields the spatial relation matrix.

[0138] Obtain feature map

[0139]

[0140] Where β is the scale parameter, its initial value is 0 and it can be trained;

[0141] To obtain the final output result F':

[0142] F'=λF+γG s +(1-λ-γ)F c

[0143] Where λ and γ are weighting parameters.

[0144] Since using only shallow attention modules cannot pass more high-dimensional information to the original data stream, therefore, as Figure 4 As shown, this embodiment designs a deep attention module to convey high-dimensional information. Similar to the shallow attention module, channel information and spatial information are provided by the original data stream and the auxiliary stream, respectively. However, the deep attention module has a larger number of parameters and can only be used for deep structures in cross-modal fusion.

[0145] The deep attention module also uses the intermediate feature map F and the pooled feature map G from the same layer as input, but it first needs to expand the two inputs of the deep attention module using convolution to obtain... and C represents the dimension of the intermediate layer; then the channel relationship matrix is ​​calculated. Spatial Relationship Matrix

[0146] On the original data stream, by F′ c With F′ c Multiplying by the transposes yields the channel relation matrix. This embodiment will use N c Multiply by F′ c Then multiply the result by the scaling parameter α and then combine it with the feature F′. c Perform element-wise summation to obtain the feature map.

[0147]

[0148] Where α is the scale parameter, its initial value is 0 and it can be trained. From the above formula, it can be seen that Fi for each channel... c The result is a weighted sum of the features of all channels and the original features, thus allowing intensity-related information in the original data to be selectively aggregated.

[0149] Corresponding to F c On the auxiliary flow, by using G′ stranspose of G′ s Multiplying them together yields the spatial relation matrix. This embodiment will use N s Multiply by G′ s Then multiply the result by the scaling parameter β and then combine it with the feature G′. s Perform element-wise summation to obtain the feature map.

[0150]

[0151] Where β is the scale parameter, its initial value is 0 and it can be trained. G s It is a weighted sum of all spatial features and original features, so texture relationships in an image can be selectively aggregated.

[0152] F c and G s This refers to the current spatial and channel information. Various feature information is fused by combining these two with the intermediate feature map F, ultimately yielding the output F' of the current block:

[0153] F'=λF+γG s +(1-λ-γ)F c ,

[0154] λ and γ are weighting parameters used to balance the influence of the three types of features.

[0155] The main features of the deep attention module in this embodiment are: (1) spatial and channel information are accumulated and fused into the original data stream, which can effectively prevent feature weakening and loss during feature fusion; (2) higher inter-channel dependencies and inter-position dependencies are established. High dependencies are established by calculating feature maps of all positions and spatial relationships, thereby improving segmentation performance.

[0156] The underground target segmentation method of the present invention has the following experimental effects.

[0157] The specific experimental procedure is as follows:

[0158] 14-channel Raptor TM GPR collected experimental data from Lhasa Gonggar Airport in Tibet, obtaining 5450 raw data points, each 256*256 pixels in size. After preprocessing, 5450 B-scan images were obtained, which were then divided into training and test sets at a 4:1 ratio for use in experiments.

[0159] The experiment focused on three typical underground targets: voids, pipes, and steel mesh. These three targets have different characteristics. Voids appear as horizontal black and white stripes in the images, but are blurred in the raw data. Pipes are clearly shown as curves in both the raw data and B-scan images. Due to the varying diameters, burial depths, and irregular distribution of the steel mesh, its appearance in the images is more cluttered. Visual comparisons of the three objects are as follows: Figure 5 As shown, (a) is a void, (b) is a pipe, and (c) is a steel mesh. It can be seen that there is a lot of interference in the B-scan image, which can lead to segmentation errors.

[0160] The model was implemented using PyTorch and optimized using the SGD algorithm. During training, the learning rate was initially set to 5e-7, decreasing by a factor of 0.1 every 20 epochs. We achieved state-of-the-art performance at epoch 23. To quantitatively evaluate the performance of our network, we used precision, recall, and F1-measure, which are common metrics for object segmentation.

[0161] Extensive ablation experiments were conducted on the effects and combinations of attention modules to analyze their impact on segmentation results. Experiments using only deep attention (DA) involved using DA in layers 3 and 4. Considering the effects of different numbers of parameters, two different combinations of SA were conducted for using only shallow attention (SA): (1) SA1: SA was used only in layers 1, 2, and 5, without DA layers; (2) SA2: SA was used in every layer of the network.

[0162] from Figure 6 As can be seen, the Fl-measure of the five combined schemes using the multi-stream attention mechanism is higher than that of the combined scheme without the multi-stream attention mechanism. Therefore, the multi-stream attention mechanism proposed in this application can effectively improve segmentation performance. Using DA and SA2 may lead to overfitting and slightly lower performance. Although the combination of DA and SA2 is not the best on some metrics, it has the best segmentation performance on average.

[0163] To quantitatively evaluate the overall performance of different methods on different target types, we selected five popular segmentation networks (U-Net, VGG16-seg, ResNet50-seg, DeepLab-v3, and SegNet) as benchmarks. The segmentation results are compared below. Figure 7 As shown, Figure 7In the image, (a) is the original data, (b) is the B-Scan image, (c) is the labeled image, (d) is the segmentation result of the method in this application, (e) is the segmentation result of U-Net, (f) is the segmentation result of VGG16-seg, (g) is the segmentation result of ResNet50-seg, (h) is the segmentation result of DeepLab-v3, and (i) is the segmentation result of SegNet. It can be seen that the segmentation result of the method in this application is more refined and accurate. Figure 8 As shown, the underground target segmentation method of this application can achieve the best segmentation results on all three types of targets, with Fl-measure values ​​being 2.12%, 4.75%, and 3.19% higher than the highest values ​​of other algorithms in voids, pipes, and steel mesh, respectively.

[0164] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0165] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0166] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0167] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for segmenting underground targets, the method comprising: Acquire raw B-Scan data and preprocessed B-Scan images; The B-Scan image is processed to obtain a first multi-scale feature map; The B-Scan image and the original B-Scan data are fused together. The positional information of the B-Scan image is used to guide the optimization of features in the original B-Scan data to obtain a second multi-scale feature map. Multimodal fusion is performed on the first multi-scale feature map and the second multi-scale feature map to obtain a series of multimodal fused feature maps; Multi-scale fusion is performed on the first multi-scale feature map, the second multi-scale feature map, and the multi-modal fusion series feature map to obtain a multi-scale fusion series feature map; By aggregating the multi-scale fused feature maps, a higher-order feature map is obtained; The high-order feature map, the first multi-scale feature map, and the second multi-scale feature map are fused at multiple levels to obtain a multi-level fusion output result. The multi-level fusion output result is then processed to obtain a segmentation result.

2. The method as described in claim 1, characterized in that, The step of processing the B-Scan image to obtain the first multi-scale feature map includes: The B-Scan image is downsampled using five convolutional layers, with each convolutional layer sequentially outputting a feature map. The feature map This constitutes the first multi-scale feature map; The step of fusing the B-Scan image and the original B-Scan data, and using the positional information of the B-Scan image to guide the optimization of parameters in the original B-Scan data to obtain the second multi-scale feature map includes: The B-Scan image is subjected to five-layer pooling to obtain a multi-scale pooling feature map; The B-Scan raw data is processed in five layers. The intermediate feature map after Vgg block processing is fused with the pooling feature map of the same layer and output as the input of the next layer. Each layer outputs feature maps s1, s2, s3, s4 and s5 in sequence. The feature maps s1, s2, s3, s4 and s5 constitute the second multi-scale feature map.

3. The method as described in claim 2, characterized in that, The multimodal fusion series feature maps are as follows: F n : ， Here, 4-fusion means fusing four different feature map inputs.

4. The method as described in claim 3, characterized in that, The multi-scale fusion series feature maps are D k , : Here, 3-fusion means fusing three different feature map inputs.

5. The method as described in claim 4, characterized in that, The step of performing multi-level fusion of the higher-order features, the first multi-scale feature map, and the second multi-scale feature map to obtain a multi-level fusion output result includes: The higher-order features are , ; , , in, For the first Feature map after sub-fusion Take 1 or 2. This is for multi-level fusion output results.

6. The method according to any one of claims 2-5, characterized in that, In the step of fusing the intermediate feature map after Vgg block processing of the input of each layer with the pooled feature map of the same layer and outputting the feature map as the input of the next layer: intermediate feature maps of at least some layers Pooling feature map of the same layer The shallow attention module fuses and outputs the feature maps, which are then used as input for the next layer of processing. C Given the number of channels, the shallow attention module operates as follows: The intermediate feature map Channel attention is obtained by sequentially processing pooling, multilayer perceptron (MLP), and element-wise recombination. The expanded and F Multiplying them together yields a shallow attention feature map. : ； The pooling feature map G The intermediate layer is obtained after convolution processing. The intermediate layer The spatial attention map is obtained after convolution processing. The expanded and F Multiply to get : ； Output the final result : ,in These are the weight parameters.

7. The method as described in claim 6, characterized in that, The intermediate feature map Channel attention is obtained by sequentially processing pooling, multilayer perceptron (MLP), and element-wise recombination. The steps include: The intermediate feature map F After max pooling and average pooling, we obtain and ,in Unified representation as ; Using a multilayer perceptron (MLP) from Features are extracted from each element, and channel attention is obtained through element-wise recombination. .

8. The method as described in claim 6, characterized in that, In the step of fusing the intermediate feature map after Vgg block processing of the input of each layer with the pooled feature map of the same layer and outputting the feature map as the input of the next layer: intermediate feature maps of at least some layers Pooling feature map of the same layer The deep attention module fuses and outputs feature maps, which are then used as input for the next layer of processing. The deep attention module works as follows: The intermediate feature map After convolution expansion The pooling feature map After convolution expansion ; Will and Multiplying by the transposes yields the channel relation matrix. ; Obtaining deep attention feature maps : in The initial value is 0 and it can be trained; Will transpose and Multiplying them together yields the spatial relation matrix. ; Obtain feature map : in The initial value is 0 and it can be trained; Obtain the final output result : in These are the weight parameters.

9. The method as described in claim 8, characterized in that, The shallow attention module is used to fuse the intermediate feature maps of layers 1, 2, and 5 with the corresponding pooled feature maps; The deep attention module is used to fuse the intermediate feature maps of the 3rd and 4th layers with the corresponding pooled feature maps.

10. An underground target segmentation system, characterized in that, The system includes modules for performing the method according to any one of claims 1-9.

Images