Ground penetrating radar signal image processing method, device, equipment and medium

By extracting global semantic and local detail information and performing feature fusion in ground-penetrating radar signal image processing, the problem of time-consuming and labor-intensive manual analysis in existing technologies is solved, and efficient and accurate identification of underground targets is achieved.

CN118658037BActive Publication Date: 2024-11-19SHANTOU UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411140607.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-19
Estimated Expiration
2044-08-20

AI Technical Summary

Technical Problem

Current ground-penetrating radar signal image data processing relies on manual analysis, which is time-consuming and labor-intensive, and it is difficult to cope with the increase in data volume. Furthermore, existing image segmentation models have shortcomings in processing efficiency and accuracy.

Method used

A feature extraction method based on the resolution of ground-penetrating radar signal images is adopted to extract global semantic information and local detail information respectively. Through feature fusion processing, a fused feature map is generated to automatically identify underground targets.

Benefits of technology

It improves the efficiency and accuracy of ground-penetrating radar signal image processing, reduces reliance on human experience, and enables rapid and accurate identification of underground targets.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118658037B_ABST
    Figure CN118658037B_ABST
Patent Text Reader

Abstract

The present invention is mainly used in the field of radar and detection signal processing technology. The present invention provides a ground penetrating radar signal image processing method, device, equipment and medium, the method comprising: based on the resolution of the acquired ground penetrating radar signal image, feature extraction processing is performed on the ground penetrating radar signal image to obtain a first resolution feature including global semantic information and a second resolution feature including local detail information; the first resolution feature and the second resolution feature are subjected to feature fusion processing to obtain a fusion feature, and a fusion feature map corresponding to the fusion feature is generated; based on the global semantic information and the local detail information, the target marking part is determined in the fusion feature map, and the target marking part is used as the underground target object in the ground penetrating radar signal image. The present invention can improve the processing efficiency of ground penetrating radar signal image data and find underground targets, which is conducive to more accurate and timely discovery of road internal diseases and road safety.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of radar and detection signal processing technology, and in particular to a ground penetrating radar signal image processing method, device, equipment and medium. Background Art

[0002] Ground penetrating radar is an efficient underground detection technology that uses high-frequency and ultra-high-frequency electromagnetic waves to image underground media. It is widely used in road detection, underground pipeline detection, geological exploration, archeology and other fields. In road detection, ground penetrating radar can non-destructively detect the internal structure of the road and find potential hidden defects such as cavities, cracks, loose areas and steel mesh. The information detected by ground penetrating radar is an important basis for road maintenance and repair. In actual engineering detection scenarios, the ground penetrating radar signal image data obtained by ground penetrating radar detection mainly relies on personnel with rich engineering experience to analyze in order to extract more valuable information. However, this analysis method is not only time-consuming and labor-intensive, but also inefficient and difficult to cope with the increasing amount of ground penetrating radar signal image data. Summary of the invention

[0003] The present invention provides a ground penetrating radar signal image processing method, device, equipment and medium, which can improve the processing efficiency of ground penetrating radar signal image data and find underground targets, thereby facilitating more accurate and timely discovery of road internal diseases and ensuring road safety.

[0004] In a first aspect, the present invention provides a method for processing a ground penetrating radar signal image, the method comprising:

[0005] Based on the resolution of the acquired ground penetrating radar signal image, performing feature extraction processing on the ground penetrating radar signal image to obtain a first resolution feature and a second resolution feature, wherein the first resolution feature includes global semantic information and the second resolution feature includes local detail information;

[0006] Performing feature fusion processing on the first resolution feature and the second resolution feature to obtain a fused feature, and generating a fused feature map corresponding to the fused feature;

[0007] Based on the global semantic information and the local detail information, a target marker portion is determined in the fused feature map, and the target marker portion is used as an underground target object in the ground penetrating radar signal image.

[0008] Optionally, based on the resolution of the acquired ground penetrating radar signal image, feature extraction processing is performed on the ground penetrating radar signal image to obtain a first resolution feature and a second resolution feature, wherein the first resolution feature includes global semantic information and the second resolution feature includes local detail information, including:

[0009] When the first resolution feature is obtained, the ground penetrating radar signal image is subjected to feature extraction processing in a plurality of stages in sequence, and a first feature map is output after the last stage of processing is completed, wherein downsampling processing is performed in each of the stages;

[0010] When acquiring the second resolution feature, the ground penetrating radar signal image is subjected to feature extraction processing in a plurality of stages in sequence, and a second feature map is output after the last stage of processing is completed, wherein downsampling processing is performed in a preset stage.

[0011] Optionally, the performing feature fusion processing on the first resolution feature and the second resolution feature to obtain a fused feature, and generating a fused feature map corresponding to the fused feature, includes:

[0012] When acquiring the first resolution feature or the second resolution feature, performing feature fusion processing in the target stage to obtain a fusion feature corresponding to the target stage;

[0013] The fused feature map is generated based on the first feature map and the second feature map including the fused feature.

[0014] Optionally, when acquiring the first resolution feature or the second resolution feature, performing feature fusion processing in a target stage to obtain a fusion feature corresponding to the target stage includes:

[0015] In a target stage, a first vector and a second vector are generated based on a first target feature, and a third vector is generated based on a second target feature, wherein the first target feature and the second target feature are respectively a first resolution feature and a second resolution feature corresponding to the target stage, or the first target feature and the second target feature are respectively a second resolution feature and a first resolution feature corresponding to the target stage;

[0016] Based on a preset attention mechanism, determining an attention value through the first vector and the third vector;

[0017] Based on the attention value and the second vector, the fusion feature is determined, wherein the fusion feature is a first resolution feature or a second resolution feature after feature fusion processing corresponding to the target stage.

[0018] Optionally, the determining a target marker part in the fused feature map based on the global semantic information and the local detail information, and using the target marker part as an underground target object in the ground penetrating radar signal image, comprises:

[0019] Determining edge information of the second feature map;

[0020] Based on the edge information, semantic segmentation processing is performed on the feature map corresponding to the target stage to divide the feature map corresponding to the target stage into a plurality of pixel groups carrying category labels;

[0021] Based on the plurality of pixel groups, the target marking portion is generated, and the target marking portion is used as an underground target object in the ground penetrating radar signal image.

[0022] Optionally, determining edge information of the second feature map includes:

[0023] When the second resolution feature is obtained, feature extraction processing is performed on the feature map outputted by the target stage to obtain visual features and generate a visual feature map;

[0024] Detecting each pixel in the visual feature map, and classifying each pixel according to the detection result to obtain an edge map after pixel classification;

[0025] Edge information is extracted from the edge map to serve as edge information of the second feature map.

[0026] Optionally, the ground penetrating radar signal image processing method further includes:

[0027] Obtain the real edge map and the target edge map;

[0028] determining a target loss value between the true edge map and the target edge map;

[0029] According to the target loss value, an edge information recognition model is adjusted, wherein the edge information recognition model is used to determine edge information of the second feature map.

[0030] In a second aspect, the present invention provides a ground penetrating radar signal image processing device, the device comprising:

[0031] An extraction module, configured to perform feature extraction processing on the ground penetrating radar signal image based on the resolution of the acquired ground penetrating radar signal image to obtain a first resolution feature and a second resolution feature, wherein the first resolution feature includes global semantic information and the second resolution feature includes local detail information;

[0032] A fusion module, used for performing feature fusion processing on the first resolution feature and the second resolution feature to obtain a fused feature, and generating a fused feature map corresponding to the fused feature;

[0033] An output module is used to determine a target marker part in the fused feature map based on the global semantic information and the local detail information, and use the target marker part as an underground target object in the ground penetrating radar signal image.

[0034] In a third aspect, the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the ground penetrating radar signal image processing method as described in any one of the above items when executing the computer program.

[0035] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the ground penetrating radar signal image processing method as described in any one of the above items.

[0036] The present invention has at least the following beneficial effects:

[0037] This application performs feature extraction processing on the ground penetrating radar signal image based on the resolution of the ground penetrating radar signal image, and can extract high-resolution first resolution features and low-resolution second resolution features, thereby extracting global semantic information and local detail information respectively, and integrating multi-scale features by performing feature fusion processing on the first resolution features and the second resolution features. Finally, based on the global semantic information and local detail information, the target marking part is determined in the fusion feature map, and the target marking part is used as the underground target in the ground penetrating radar signal image, so as to realize automatic identification of the target in the ground penetrating radar signal image, accurately locate the underground target area, reduce the dependence on manual experience, and improve the detection efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0039] Figure 1 is a flow chart of steps of a method for processing a ground penetrating radar signal image;

[0040] Figure 2 The invention is a schematic diagram of a process for respectively extracting a first resolution feature and a second resolution feature in a ground penetrating radar signal image processing method;

[0041] Figure 3 is a schematic diagram of the structure of a BasicBlocks module including a downsampling submodule;

[0042] Figure 4 It is a schematic diagram of the structure of a BasicBlocks module;

[0043] Figure 5 It is a structural schematic diagram of a ground penetrating radar signal image processing device;

[0044] Figure 6 It is a structural diagram of an electronic device. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0046] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0047] Ground penetrating radar is a highly efficient underground detection technology that uses high-frequency and ultra-high-frequency electromagnetic waves to image underground media. It is widely used in road detection, underground pipeline detection, geological exploration, archaeology and other fields. In road detection, ground penetrating radar can non-destructively detect the internal structure of the road and discover potential hidden diseases such as cavities, cracks, loose areas and steel mesh, thus providing an important basis for road maintenance and repair.

[0048] After a road is put into use, various hidden diseases gradually increase due to disturbances caused by engineering projects, water intrusion, and repeated effects of vehicle loads. If these diseases are not discovered and treated in time, they will seriously affect the service life of the road and driving safety. Therefore, it is crucial to quickly and accurately detect and diagnose disease problems in the internal structure of the road to extend the service life of the road and ensure driving safety.

[0049] Traditional road disease detection methods such as on-site excavation and drilling sampling can provide detailed disease information, but they have the disadvantages of being highly destructive, long construction period, and high cost. Ground penetrating radar technology has been widely promoted and applied in recent years due to its advantages of being non-destructive, fast, and low-cost. Through ground penetrating radar scanning, the internal structural status of the road can be effectively reflected, hidden diseases can be discovered, and dynamic monitoring can be performed.

[0050] However, in actual engineering inspection scenarios, the interpretation of GPR signal image data mainly relies on personnel with rich engineering experience. This method is not only time-consuming and labor-intensive, but also inefficient and difficult to cope with the increasing amount of GPR data. Therefore, developing a method that can quickly identify targets in GPR images and accurately segment their areas has important practical significance and application value.

[0051] In the field of semantic segmentation, a variety of technologies and methods have been proposed and applied, including:

[0052] End-to-end convolutional neural networks (FCNs): FCNs directly use convolution operations for pixel-level predictions, but lack the full utilization and fusion of features of different scales, resulting in poor performance in complex scenes.

[0053] U-Net neural network: Although the U-Net structure performs well in medical image segmentation, when processing natural images, due to the use of a symmetrical encoder-decoder structure, the process of fusing features through jump connections requires a lot of computing resources and memory, especially in high-resolution image processing.

[0054] DeepLab neural network: It uses dilated convolution and multi-scale information fusion technology to capture contextual information at different scales, but it cannot fully capture subtle boundary information, resulting in low segmentation accuracy in boundary areas.

[0055] Transformer-based model: By introducing a complex attention mechanism and a large number of parameters, the segmentation performance is improved. The Transformer-based model performs well in semantic segmentation, but it also brings about a significant increase in computing and resource consumption, limiting its application in scenarios with limited resources.

[0056] In view of the deficiencies of the prior art in semantic segmentation and interpretation of ground penetrating radar signal image data, the present invention aims to propose a ground penetrating radar signal image processing method to improve the accuracy and efficiency of image segmentation, reduce dependence on manual experience, and be able to quickly and accurately identify and segment targets in ground penetrating radar images. The embodiment of the present invention performs feature extraction processing on the ground penetrating radar signal image based on the resolution of the ground penetrating radar signal image, and can extract high-resolution first resolution features and low-resolution second resolution features, thereby respectively extracting global semantic information and local detail information, and integrating multi-scale features by performing feature fusion processing on the first resolution features and the second resolution features. Finally, based on the global semantic information and local detail information, the target marking part is determined in the fused feature map, and the target marking part is used as the underground target in the ground penetrating radar signal image, so as to realize automatic identification of the target in the ground penetrating radar signal image, accurately locate the underground target area, reduce dependence on manual experience, and improve detection efficiency and accuracy. The various embodiments of the present invention are as follows:

[0057] Please refer to Figure 1 , Figure 1 It is a flow chart of the steps of the ground penetrating radar signal image processing method.

[0058] In a first aspect, this embodiment provides a ground penetrating radar signal image processing method comprising:

[0059] S100. Based on the acquired resolution of the ground penetrating radar signal image, perform feature extraction processing on the ground penetrating radar signal image to obtain a first resolution feature and a second resolution feature, wherein the first resolution feature includes global semantic information and the second resolution feature includes local detail information.

[0060] S200, performing feature fusion processing on the first resolution feature and the second resolution feature to obtain a fused feature, and generating a fused feature map corresponding to the fused feature.

[0061] S300: Based on the global semantic information and the local detail information, determine the target marking part in the fused feature map, and use the target marking part as the underground target object in the ground penetrating radar signal image.

[0062] In some embodiments, based on the resolution of the acquired ground penetrating radar signal image, feature extraction processing is performed on the ground penetrating radar signal image. This can be done according to a preset resolution threshold through a dual-resolution network architecture, including a low-resolution branch for extracting global semantic information to perform feature extraction processing and obtain low-resolution first resolution features; and a high-resolution branch for capturing local detail information to use small-scale structural features as high-resolution second resolution features.

[0063] It can be understood that this embodiment uses a specific downsampling strategy and feature fusion mechanism to perform feature fusion processing on the first resolution feature and the second resolution feature, thereby avoiding the information loss that may be caused by simple splicing or addition, improving the correlation utilization and importance weight calculation between features of different resolutions, making feature fusion more accurate and flexible, and better combining global features and local features.

[0064] Please refer to Table 1, and experimentally compare the ground penetrating radar signal image processing method of the present embodiment with the existing ground penetrating radar signal image processing method based on image segmentation model, and the existing image segmentation models are BiseNetV2, Unet, SegNet, ShuffleNetV2, PidNet, and DDRNet. According to the comparison results, the ground penetrating radar signal image processing method of the present embodiment shows better comprehensive performance, wherein the average intersection-over-union ratio (mIOU), F1-score (F1-Score), precision (Precision) and recall (Recall) in the data results of the ground penetrating radar signal image processing method of the present embodiment are better than the methods of the other six models, and the inference time (inference time) of the ground penetrating radar signal image processing method of the present embodiment is 0.0212 seconds, which proves that the ground penetrating radar signal image processing method of the present embodiment can provide an inference speed that meets real-time detection while maintaining a high average intersection-over-union ratio, high precision, high recall rate and high F1 value, and is very suitable for application scenarios that require accurate and fast detection.

[0065] Table 1 - Comparison results experimental data table

[0066]

[0067] In some embodiments, after the target marking part is determined by fusing the feature map, a recognition model that has been trained for the ground penetrating radar signal image can be called to identify underground targets in the ground penetrating radar signal image.

[0068] In some embodiments, step S100 includes:

[0069] When the first resolution feature is obtained, the ground penetrating radar signal image is subjected to feature extraction processing in sequence through multiple stages, and the first feature map is output after the last stage of processing is completed, wherein downsampling processing is performed in each stage.

[0070] When acquiring the second resolution feature, the ground penetrating radar signal image is subjected to feature extraction processing in sequence through multiple stages, and a second feature map is output after the last stage of processing is completed, wherein downsampling processing is performed in a preset stage.

[0071] In an actual scenario, only one specific embodiment is used to illustrate the actual application of the ground penetrating radar signal image processing method of the above embodiment.

[0072] See also Figure 2 , Figure 2 The invention is a flow chart of respectively extracting a first resolution feature and a second resolution feature in a ground penetrating radar signal image processing method.

[0073] The process of extracting the first resolution feature and the second resolution feature can be divided into 6 stages, namely the stem stage, stage 1 stage, stage 2 stage, stage 3 stage, stage 4 stage and stage 5 stage. When obtaining the first resolution feature, the ground penetrating radar signal image is first subjected to feature extraction processing in the stem stage. Then, after being processed in the stem stage, the processed feature is output, processed in the stage 1 stage, and input to the stage 2 stage for processing after the processing in the stage 1 stage is completed, so that the first feature map is output after being processed in the stage 3 stage and the stage 4 stage in sequence, and finally processed in the stage 5 stage. Similarly, when obtaining the second resolution feature, the second feature map is output after being processed in the stem stage, stage 1 stage, stage 2 stage, stage 3 stage, stage 4 stage and stage 5 stage in sequence. In this specific embodiment, when the first resolution feature is obtained through the low-resolution branch, downsampling processing is also performed in each stage from the stage 2 stage to the stage 5 stage; when the second resolution feature is obtained through the high-resolution branch, downsampling processing is only performed in the stage 2 stage and the stage 5 stage to prevent the loss of detail information.

[0074] It can be understood that when the first-resolution features are obtained through the low-resolution branch, downsampling is continuously performed from Stage 2 to Stage 5, which is beneficial to expanding the receptive field and improving processing efficiency; when the second-resolution features are obtained through the high-resolution branch, downsampling is only performed at Stage 2 and Stage 5, which is beneficial to optimizing network performance and appropriately reducing the feature map resolution at critical stages, while still focusing on maintaining sufficient detail information.

[0075] In some embodiments, in the stem stage, two Conv-BN-ReLU modules are included, each of which performs downsampling once to reduce the output feature resolution to one-fourth of the input. Taking the process of obtaining the first resolution feature through the low-resolution branch as an example, in the stem stage, the first Conv-BN-ReLU module is used to process the input ground penetrating radar signal image to obtain the first feature map, and the first feature map is downsampled to obtain the second feature map, and the second Conv-BN-ReLU module is used to perform feature extraction on the second feature map to obtain the third feature map, and finally the third feature map is downsampled and input to the Stage 1 stage. Similarly, the same process is performed to obtain the second resolution feature through the high-resolution branch.

[0076] Specifically, the Conv-BN-ReLU module includes a 3x3 convolution layer, a BatchNorm layer, and a ReLU activation function.

[0077] In some embodiments, when the first resolution feature is obtained through the low-resolution branch, in the stage 1 stage, the stage 2 stage, the stage 3 stage, and the stage 4 stage, each stage includes two BasicBlocks modules. In the stage 2 stage, the stage 3 stage, and the stage 4 stage, the BasicBlocks module of each stage also includes a downsampling submodule Downsample. In the Stage 5 stage, the DAPPM module is first used for processing, and then the bottleneck module (Bottleneck module) including the downsampling submodule is used for processing.

[0078] In some embodiments, when the second resolution feature is obtained through the high-resolution branch, in the stage 1 to the stage 4, each stage includes two BasicBlocks modules, and in addition, the BasicBlocks module of the Stage 2 stage also includes a downsampling submodule Downsample. In the Stage 5 stage, a bottleneck module (Bottleneck module) including a downsampling submodule is used for processing.

[0079] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of a BasicBlocks module including a downsampling submodule. In this BasicBlocks module, there are two Conv-BN-ReLU modules. The stride of the 3x3 convolution layer in the first Conv-BN-ReLU module is set to 2, and the stride of the 3x3 convolution layer in the second Conv-BN-ReLU module is set to 1. The feature dimension at the output of the BasicBlocks module is twice the feature dimension at the input. Before the residual connection, the input is calculated through the downsampling submodule Downsample (including a 1x1 convolution layer with a stride of 2 and a BatchNorm layer) to align the feature dimensions.

[0080] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of a BasicBlocks module. In this BasicBlocks module, there are two Conv-BN-ReLU modules. The stride of the 3x3 convolution layer in the first Conv-BN-ReLU module is set to 1, and the stride of the 3x3 convolution layer in the second Conv-BN-ReLU module is set to 1.

[0081] It is understandable that this embodiment uses the BasicBlocks module, which takes advantage of the residual connection structure to help alleviate the gradient vanishing problem. The BasicBlocks module including the downsampling submodule is characterized in that the feature dimension can be adjusted, while the BasicBlocks module without the downsampling submodule is conducive to keeping the feature dimension unchanged.

[0082] It is understandable that the DAPPM module (Dense Attention Pyramid Pooling Module) enhances the expressiveness of features through multi-scale processing and attention mechanism. DAPPM can focus on features of different scales and improve the model's understanding of complex scenes by aggregating these features. This module improves the richness and relevance of features while keeping the size of the feature map stable.

[0083] Bottleneck module, used to further refine and compress features while reducing computational complexity. Bottleneck modules are common in deep learning networks, especially in scenarios where feature compression and long-distance dependency learning are required. It contains convolution operations with large strides, aiming to reduce the spatial size of the feature map while maintaining information flow through residual connections.

[0084] In some embodiments, step S200 includes:

[0085] When the first resolution feature or the second resolution feature is obtained, feature fusion processing is performed in the target stage to obtain a fused feature of the corresponding target stage; and a fused feature map is generated based on the first feature map and the second feature map including the fused feature.

[0086] It can be understood that in some embodiments, after the ground penetrating radar signal image is processed by feature extraction through a low-resolution branch, a first feature map is obtained; after the ground penetrating radar signal image is processed by feature extraction through a high-resolution branch, a second feature map is obtained; the first feature map and the second feature map are fused to generate a fused feature map.

[0087] In a specific embodiment, the process of extracting the first resolution feature and the second resolution feature can be divided into 6 stages, namely the stem stage, the stage 1 stage, the stage 2 stage, the stage 3 stage, the stage 4 stage and the stage 5 stage. When the first resolution feature or the second resolution feature is obtained, feature fusion processing is performed in the stage 3 stage, the stage 4 stage and the stage 5 stage.

[0088] In some embodiments, when acquiring the first resolution feature or the second resolution feature, a feature fusion process is performed in the target stage to obtain a fusion feature corresponding to the target stage. The specific implementation method is:

[0089] In the target stage, a first vector and a second vector are generated based on the first target feature, and a third vector is generated based on the second target feature, wherein the first target feature and the second target feature are respectively the first resolution feature and the second resolution feature of the corresponding target stage, or the first target feature and the second target feature are respectively the second resolution feature and the first resolution feature of the corresponding target stage; based on a preset attention mechanism, an attention value is determined through the first vector and the third vector; based on the attention value and the second vector, a fusion feature is determined, wherein the fusion feature is the first resolution feature or the second resolution feature after the feature fusion processing of the corresponding target stage.

[0090] In some embodiments, taking the process of obtaining the first resolution feature as an example, when the feature fusion processing is performed in the target stage, the specific feature fusion processing method is: through a 1x1 convolution layer, a first vector K and a second vector V are generated from the first resolution feature, the first vector K is a key (Key, K), and the second vector V is a value (Value, V), and a third vector Q is generated from the second resolution feature through a 1x1 convolution layer, and the third vector Q is a query (Query, Q). Based on the preset attention mechanism, the attention value is calculated according to the first vector K and the second vector Q, and then the output value is calculated according to the attention value and the second vector V, and the output value is merged into the output value of the target stage (that is, by generating the first resolution feature after the feature fusion processing of the target stage). Similarly, in the process of obtaining the first resolution feature, the first vector K and the second vector V are calculated according to the second resolution feature, and the third vector Q is calculated using the first resolution feature, and finally the first resolution feature after the feature fusion processing of the target stage is obtained.

[0091] In particular, the present application also provides an embodiment of calculating an attention value. In this embodiment, the product between the transposed matrix of the first vector and the transposed matrix of the second vector is determined; the square root value between the product and the target dimension is determined, wherein the target dimension is the dimension of the first vector; and the square root value is normalized to obtain the attention value.

[0092] In a specific embodiment, the transposed matrix of the first vector K is multiplied by the transposed matrix of the second vector Q, and then the product is divided by the square root of the dimension of the first vector K, and then normalized by the softmax function. In this specific embodiment, the calculation process uses the formula including:

[0093] ;

[0094] ;

[0095] ;

[0096] It is understandable that compared to achieving feature fusion by simple feature concatenation or addition, which may lead to information loss and fail to fully consider the correlation and importance between features, this embodiment calculates the correlation and importance weights between features through a cross-attention mechanism, and can more accurately and flexibly fuse features of different resolutions, thereby improving the effectiveness of feature fusion. Moreover, by introducing an efficient attention mechanism, redundant calculations and the amount of parameters are reduced, thereby improving computing efficiency while maintaining high segmentation accuracy and reducing the demand for hardware resources.

[0097] In some embodiments, step S300 includes:

[0098] Determine edge information of the second feature map; based on the edge information, perform semantic segmentation processing on the feature map corresponding to the target stage to divide the feature map corresponding to the target stage into multiple pixel groups carrying category labels; based on the multiple pixel groups, generate a target marking part, and use the target marking part as an underground target object in the ground penetrating radar signal image.

[0099] It can be understood that the purpose of extracting the first resolution feature based on the low-resolution standard is to obtain the global semantic information of the ground-penetrating radar signal image, specifically, to obtain the contour of the ground-penetrating radar signal image; the purpose of extracting the second resolution feature based on the high-resolution standard is to obtain the local detail information of the ground-penetrating radar signal image, and in order to obtain the local detail information, it is necessary to extract edge information. The edge information specifically refers to the intensity (brightness and dark contrast) and direction (directivity of the edge), the texture and other detail information in the image, and the local structural features in the image, such as corners, intersections, etc.

[0100] In a specific embodiment, the process of extracting high-resolution features can be divided into 6 stages, namely, the stem stage, stage 1 stage, stage 2 stage, stage 3 stage, stage 4 stage and stage 5 stage. In the target stage, i.e., stage 3 stage, the feature map is semantically segmented by the SegmentHead module, and the feature map is converted into the final classification result, i.e., the category label corresponding to each pixel, to generate a semantic segmentation map. The marking part in the semantic segmentation map helps the image recognition model to quickly lock the underground target area in the ground penetrating radar signal image in the subsequent image recognition step.

[0101] In a specific embodiment, the SegmentHead module is a segmentation prediction module, which includes a 3x3 Conv-BN-ReLU module and a 1x1 convolutional layer for generating the final segmentation prediction.

[0102] In some embodiments, the method of enhancing the semantic segmentation capability of the SegmentHead module through training includes: obtaining a sample image with a marked identification part, and performing cross-entropy loss calculation on the label values ​​in the semantic segmentation map generated by the SegmentHead module and the label values ​​in the sample image, so as to supervise the training process of the SegmentHead module according to the loss value.

[0103] In some embodiments, the specific implementation of the step of determining the edge information of the second feature map is:

[0104] When acquiring the second resolution feature, feature extraction processing is performed on the feature map output by the target stage to obtain visual features and generate a visual feature map; each pixel in the visual feature map is detected, and each pixel is classified according to the detection result to obtain an edge map after pixel classification; edge information is extracted from the edge map as edge information of the second feature map.

[0105] In a specific embodiment, the process of extracting high-resolution features can be divided into 6 stages, namely, the stem stage, the stage 1 stage, the stage 2 stage, the stage 3 stage, the stage 4 stage, and the stage 5 stage. In the target stage, i.e., the stage 4 stage, the step of determining the edge information of the second feature map is performed by the edge segmentation auxiliary head module (BoundaryHead module). Specifically, the BoundaryHead module is used to process the same or different feature maps, learn to identify the boundary information in the image, and help the semantic segmentation task to more accurately divide different areas to generate an edge image.

[0106] In some embodiments, the structure of the BoundaryHead module includes a Conv-BN-ReLU module, a binary classification convolution layer, and a ground truth edge information acquisition module. The Conv-BN-ReLU module includes a convolution layer with a kernel size of 3, a BatchNorm layer, and a ReLU activation function; the binary classification convolution layer is used to perform binary classification in edge detection tasks, that is, to determine whether each pixel is an edge in the image, thereby generating an edge map; the ground truth edge information acquisition module is used to generate edge information by applying the Laplacian kernel to the ground truth image.

[0107] The BoundaryHead module first extracts features from the input image through the Conv-BN-ReLU module, including edges, textures, and other important visual information, to obtain a feature map containing important features of the image; then, the feature map is edge detected through a binary convolution layer, and a rough edge map after pixel classification is output, in which the pixel value indicates whether the corresponding position is the edge of the image. Specifically, the sigmoid activation function can be used to determine whether the pixel point of the image is the edge of the image, and the output of the convolution layer is mapped between 0 and 1 for binary classification, that is, the pixel point is an edge or a non-edge. Finally, the rough edge map is further processed through the ground truth edge information acquisition module, and more refined edge information is extracted from the rough edge map, and a refined edge map is generated. Specifically, the edge information is generated by applying the Laplacian kernel to the ground truth image.

[0108] In some embodiments, when the Laplacian kernel is used to process an image, the target feature image y is transformed into gt The pixel value at each position in the Laplace kernel is multiplied by the corresponding weight and then summed to generate a new image, namely the edge map y boundary_gt The calculation process is performed using the following formula:

[0109] ;

[0110] In a specific embodiment, edge pixels of an edge map are expanded to generate a ground truth image three times the size of the edge map.

[0111] It is understandable that traditional semantic segmentation networks usually only focus on the overall pixel classification task, and less processing of edge details, resulting in insufficient ability to retain edge information. This embodiment enhances the network's learning and utilization of edge features by introducing a boundary segmentation auxiliary head module (BoundaryHead module), which can better retain and identify detail information in the image, and generates edge information through convolution operations and Laplacian kernels, thereby enhancing the network's learning and utilization of edge features. Compared with the prior art that processes less edge details, the BoundaryHead module can better retain and identify detail information in the image, reduce the phenomenon of mis-segmentation in the boundary area, and thus improve the detail retention ability and accuracy of the segmentation results.

[0112] In some embodiments, the ground penetrating radar signal image processing method further includes:

[0113] Obtain a true edge map and a target edge map; determine a target loss value between the true edge map and the target edge map; and adjust an edge information recognition model according to the target loss value, wherein the edge information recognition model is used to determine edge information of a second feature map.

[0114] Specifically, the method of enhancing the edge recognition ability of the BoundaryHead module through training includes: obtaining a real edge image, and calculating the cross entropy loss and Dice loss between the edge image generated by the BoundaryHead and the real edge image, so as to supervise the training process of the BoundaryHead module according to the loss value. Specifically, the loss function for calculating the loss value between the edge image generated by the BoundaryHead and the real edge image is as follows:

[0115] ;

[0116] Among them, each λ is a hyperparameter, l0 represents the cross entropy loss between the edge image generated by the BoundaryHead module and the true edge image, l1 represents the cross entropy loss between the output of the SegmentHead module and the final true label, l2 represents the cross entropy loss between the output of the BoundaryHead module and the true edge label, and l3 represents the Dice loss value used to alleviate the impact of category imbalance.

[0117] Please refer to Figure 5 , Figure 5 The invention is a structural schematic diagram of a ground penetrating radar signal image processing device.

[0118] In a second aspect, this embodiment provides a ground penetrating radar signal image processing device comprising:

[0119] The extraction module 501 is used to perform feature extraction processing on the ground penetrating radar signal image based on the resolution of the acquired ground penetrating radar signal image to obtain a first resolution feature and a second resolution feature, wherein the first resolution feature includes global semantic information and the second resolution feature includes local detail information.

[0120] The fusion module 502 is used to perform feature fusion processing on the first resolution feature and the second resolution feature to obtain a fused feature, and generate a fused feature map corresponding to the fused feature.

[0121] The output module 503 is used to determine the target marking part in the fused feature map based on the global semantic information and the local detail information, and use the target marking part as the underground target object in the ground penetrating radar signal image.

[0122] It will be appreciated by those skilled in the art that all or some of the steps and devices in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. As is well known to those skilled in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0123] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0124] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing any of the above ground penetrating radar signal image processing methods when executing the computer program.

[0125] refer to Figure 6 , Figure 6 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0126] The processor 601 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0127] The memory 602 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 602 can store operating devices and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 602, and the processor 601 is called to execute the ground penetrating radar signal image processing method of the embodiment of this application;

[0128] Input / output interface 603, used to implement information input and output;

[0129] Communication interface 604, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0130] A bus 605 that transmits information between various components of the device (e.g., processor 601, memory 602, input / output interface 603, and communication interface 604);

[0131] The processor 601 , the memory 602 , the input / output interface 603 and the communication interface 604 are connected to each other in communication within the device via a bus 605 .

[0132] It can be understood that the contents of the above method embodiments are all applicable to the electronic device embodiment, the functions specifically implemented by the electronic device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0133] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to implement the ground penetrating radar signal image processing method as described in any one of the above specific embodiments.

[0134] An embodiment of the present application also discloses a computer program product, including a computer program or computer instructions, wherein the computer program or computer instructions are stored in a computer-readable storage medium, a processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions, so that the computer device executes the ground penetrating radar signal image processing method as described in any of the previous embodiments.

[0135] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0136] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0137] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0138] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0139] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0140] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0141] Although the description of the present application has been quite detailed and specifically describes several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad possible interpretation of these claims by reference to the attached claims, taking into account the prior art, so as to effectively cover the intended scope of the present application. In addition, the above description of the present application is based on the embodiments foreseeable by the inventor, and its purpose is to provide a useful description, and those non-substantial changes to the present application that have not yet been foreseen may still represent equivalent changes to the present application.

Claims

1. A ground penetrating radar signal image processing method, characterized in that: The method comprises: Based on the resolution of the acquired ground penetrating radar signal image, performing feature extraction processing on the ground penetrating radar signal image to obtain a first resolution feature and a second resolution feature, wherein the first resolution feature includes global semantic information and the second resolution feature includes local detail information; When acquiring the first resolution feature or the second resolution feature, performing feature fusion processing in the target stage to obtain a fusion feature corresponding to the target stage; In a target stage, a first vector and a second vector are generated based on a first target feature, and a third vector is generated based on a second target feature, wherein the first target feature and the second target feature are respectively a first resolution feature and a second resolution feature corresponding to the target stage, or the first target feature and the second target feature are respectively a second resolution feature and a first resolution feature corresponding to the target stage; Based on a preset attention mechanism, determining an attention value through the first vector and the third vector; The specific steps of determining the attention value include: determining the product between the transposed matrix of the first vector and the transposed matrix of the third vector; determining the square root value between the product and the target dimension, wherein the target dimension is the dimension of the first vector; normalizing the square root value to obtain the attention value; Determine the fusion feature based on the attention value and the second vector, wherein the fusion feature is a first resolution feature or a second resolution feature after feature fusion processing corresponding to the target stage; Based on the first feature map and the second feature map including the fused feature, the fused feature map is generated, wherein the ground penetrating radar signal image is subjected to feature extraction processing through a low-resolution branch to obtain the first feature map; the ground penetrating radar signal image is subjected to feature extraction processing through a high-resolution branch to obtain the second feature map, wherein the resolution of the ground penetrating radar signal image in the low-resolution branch is lower than the resolution of the ground penetrating radar signal image in the high-resolution branch; Based on the global semantic information and the local detail information, a target marker portion is determined in the fused feature map, and the target marker portion is used as an underground target object in the ground penetrating radar signal image.

2. A ground penetrating radar signal image processing method according to claim 1, characterized in that: Based on the resolution of the acquired ground penetrating radar signal image, feature extraction processing is performed on the ground penetrating radar signal image to obtain a first resolution feature and a second resolution feature, wherein the first resolution feature includes global semantic information and the second resolution feature includes local detail information, including: When the first resolution feature is obtained, the ground penetrating radar signal image is subjected to feature extraction processing in a plurality of stages in sequence, and a first feature map is output after the last stage of processing is completed, wherein downsampling processing is performed in each of the stages; When acquiring the second resolution feature, the ground penetrating radar signal image is subjected to feature extraction processing in a plurality of stages in sequence, and a second feature map is output after the last stage of processing is completed, wherein downsampling processing is performed in a preset stage.

3. A ground penetrating radar signal image processing method according to claim 2, characterized in that: The step of determining a target marker portion in the fused feature map based on the global semantic information and the local detail information, and using the target marker portion as an underground target object in the ground penetrating radar signal image, includes: Determining edge information of the second feature map; Based on the edge information, semantic segmentation processing is performed on the feature map corresponding to the target stage to divide the feature map corresponding to the target stage into a plurality of pixel groups carrying category labels; Based on the plurality of pixel groups, the target marking portion is generated, and the target marking portion is used as an underground target object in the ground penetrating radar signal image.

4. A method for processing ground penetrating radar signal images according to claim 3, characterized in that: The determining edge information of the second feature map includes: When the second resolution feature is obtained, feature extraction processing is performed on the feature map outputted by the target stage to obtain visual features and generate a visual feature map; Detecting each pixel in the visual feature map, and classifying each pixel according to the detection result to obtain an edge map after pixel classification; Edge information is extracted from the edge map to serve as edge information of the second feature map.

5. A method for processing ground penetrating radar signal images according to claim 4, characterized in that: Also includes: Obtain the real edge map and the target edge map; determining a target loss value between the true edge map and the target edge map; According to the target loss value, an edge information recognition model is adjusted, wherein the edge information recognition model is used to determine edge information of the second feature map.

6. A ground penetrating radar signal image processing device, characterized in that: The device comprises: An extraction module, configured to perform feature extraction processing on the ground penetrating radar signal image based on the resolution of the acquired ground penetrating radar signal image to obtain a first resolution feature and a second resolution feature, wherein the first resolution feature includes global semantic information and the second resolution feature includes local detail information; a fusion module, configured to perform feature fusion processing in a target stage when acquiring the first resolution feature or the second resolution feature, so as to obtain a fusion feature corresponding to the target stage; In a target stage, a first vector and a second vector are generated based on a first target feature, and a third vector is generated based on a second target feature, wherein the first target feature and the second target feature are respectively a first resolution feature and a second resolution feature corresponding to the target stage, or the first target feature and the second target feature are respectively a second resolution feature and a first resolution feature corresponding to the target stage; Based on a preset attention mechanism, determining an attention value through the first vector and the third vector; The specific steps of determining the attention value include: determining the product between the transposed matrix of the first vector and the transposed matrix of the third vector; determining the square root value between the product and the target dimension, wherein the target dimension is the dimension of the first vector; normalizing the square root value to obtain the attention value; Determine the fusion feature based on the attention value and the second vector, wherein the fusion feature is a first resolution feature or a second resolution feature after feature fusion processing corresponding to the target stage; Based on the first feature map and the second feature map including the fused feature, the fused feature map is generated, wherein the ground penetrating radar signal image is subjected to feature extraction processing through a low-resolution branch to obtain the first feature map; the ground penetrating radar signal image is subjected to feature extraction processing through a high-resolution branch to obtain the second feature map, wherein the resolution of the ground penetrating radar signal image in the low-resolution branch is lower than the resolution of the ground penetrating radar signal image in the high-resolution branch; An output module is used to determine a target marker part in the fused feature map based on the global semantic information and the local detail information, and use the target marker part as an underground target object in the ground penetrating radar signal image.

7. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the ground penetrating radar signal image processing method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the ground penetrating radar signal image processing method according to any one of claims 1 to 5 is implemented.