A GPR relative positioning method based on saliency detection and discontinuous fusion

Abstract

A kind of GPR relative positioning method based on saliency detection and discontinuous fusion.It includes collecting observation data of mobile carrier to be positioned;Conversion B-scan image pair;Construct multi-sensor fusion positioning system;Saliency detection module is handled to B-scan image pair, and effective B-scan image is identified;Ground penetrating radar mileage estimation module is handled to effective B-scan image pair, and one-dimensional relative displacement estimation value is obtained;Adaptive discontinuous data fusion is carried out by factor graph model, and finally the optimal estimation of motion trajectory is obtained and the like steps.The effect of the present application is that saliency detection module realizes real-time discrimination of underground feature saliency by horizontal / vertical / standard three-channel feature extraction of B-scan image pair.GPR displacement estimation is only added as constraint to factor graph model when saliency detection module determines that image pair feature is salient.Adjust the factor graph optimization strategy, when GPR data is long-term missing, factor graph only estimates system motion state according to existing sensor data.

Description

Technical Field

[0001] This invention belongs to the field of mobile robot localization technology, and specifically relates to a GPR relative localization method based on saliency detection and discontinuous fusion. Background Technology

[0002] In non-open spaces such as tunnels, indoor environments, and underground pipe networks, achieving stable and accurate autonomous positioning of mobile robots is a key technological foundation for many fields, including autonomous driving, robotic exploration, and emergency rescue. However, such environments pose severe challenges to traditional positioning technologies: GPS signals are severely blocked or completely disabled; while inertial measurement units (IMUs) can operate independently, they suffer from significant cumulative errors, leading to a sharp decline in accuracy over time; optical sensors (such as cameras and lidar) are susceptible to interference from sudden changes in ambient light, smoke, and dust, and their performance degrades in low-light or corridors with repetitive features. The inherent limitations of single sensors in complex and variable underground environments have prompted researchers to turn to multi-sensor fusion techniques. By collaboratively utilizing sensors with different physical characteristics, the system can maintain its output even when one sensor temporarily fails, relying on other sensors to improve the overall robustness and accuracy of the positioning system.

[0003] In recent years, ground-penetrating radar (GPR) has been regarded as a highly promising supplementary sensor to compensate for the aforementioned shortcomings in environmental perception due to its non-destructive penetration capabilities into subsurface media. Unlike surface features, which are susceptible to dynamic changes in the Earth's surface, subsurface media structures (such as soil interfaces, buried pipelines, and rock bedding) are generally more stable, providing unique subsurface characteristics for positioning. GPR technology has proven its value in fields such as civil engineering inspection (e.g., road defect investigation), planetary science exploration (e.g., Martian subsurface structure exploration), and mineral resource assessment. Relative positioning (or odometry) methods based on GPR subsurface feature matching are beginning to be introduced into multi-sensor fusion positioning frameworks, aiming to enhance the environmental adaptability of systems by utilizing stable subsurface information.

[0004] The existing technical solutions include the following:

[0005] ① Learning-based GPR odometry model: Using a residual network (ResNet-18), feature maps are extracted from two B-scan images with significant features. A sliding window is used to compare the similarity of the two sets of feature maps to obtain a set of most similar window indices. The indices are put into a fully connected layer for regression to obtain the distance estimate.

[0006] ② CNN-LSTM model: Multiple consecutive pairs of adjacent B-scan images are fed into a convolutional neural network (CNN) to obtain a set of feature vectors. The set of feature vectors is then fed into a long short-term memory network (LSTM) to obtain an estimated distance.

[0007] ③ Subsurface Feature Matrix Model: By extracting key reflectance peak points from B-scan images, an SFM matrix that characterizes the spatial distribution of subsurface features is constructed. By calculating the optimal horizontal matching offset between SFM matrices at adjacent time points and converting it into physical distance, an estimated distance is obtained.

[0008] ④ OdomNet model: Input two consecutive B-scan image pairs, and obtain the estimated distance between the B-scan image pairs by extracting the similarity and differences between the image pairs.

[0009] However, the drawbacks of these existing technical solutions are as follows:

[0010] 1. Existing GPR relative positioning methods lack a front-end evaluation mechanism;

[0011] Current GPR displacement estimation methods based on B-scan image pairs generally assume that all consecutively acquired image pairs can be used for effective matching and displacement estimation. However, in practical applications, when the carrier passes through areas with sparse or repetitive underground features, or when the carrier's own motion state changes drastically, the feature matching relationship between adjacent B-scan image pairs becomes unreliable, leading to significant errors in displacement estimation. Existing methods lack an effective front-end quality detection step, failing to identify and remove these "invalid" or "low-quality" image pairs, resulting in displacement estimation results with large errors being directly fed into subsequent multi-sensor fusion systems.

[0012] 2. Existing fusion strategies for GPR odometers and multi-sensor systems are not robust enough;

[0013] In current fusion positioning frameworks, GPR odometry is typically treated as a continuously providing sensor module, and its output displacement estimates (whether reliable or not) are continuously and uninterruptedly added as constraint factors to the back-end optimizer. This "continuous fusion" strategy has a drawback: when GPR produces incorrect estimates for the reasons mentioned above, the system uses them indiscriminately, thereby affecting the state estimation of the entire system's position and attitude, leading to decreased positioning accuracy or even trajectory divergence, which contradicts the original intention of fusion to improve robustness.

[0014] 3. The existing optimization framework's handling of GPR data gaps is unreasonable;

[0015] Existing fusion optimization methods face problems when GPR refuses to output data due to unreliable data detected by the front end, or when there are no effective observations for a long time due to other reasons in practical applications. If conventional interpolation methods are used to fill the missing periods of GPR observations, it will introduce assumption errors that do not conform to the actual motion model. Summary of the Invention

[0016] To address the aforementioned problems, the present invention aims to provide a GPR relative localization method based on saliency detection and discontinuity fusion.

[0017] To achieve the above objectives, the GPR relative localization method based on saliency detection and discontinuity fusion provided by the present invention includes the following steps performed in sequence:

[0018] 1) The vehicle-mounted ground-penetrating radar, wheel encoder, and inertial measurement unit are used to collect in real time the original A-scan image with a complete trajectory, one-dimensional motion distance, angular velocity, and linear acceleration of the mobile vehicle to be located, as well as the timestamps of the corresponding data collected by the clock, as observation data.

[0019] 2) The original A-scan images are preprocessed to convert them into B-scan image pairs with overlapping regions;

[0020] 3) Construct a multi-sensor fusion positioning system; the system includes a saliency detection module, a ground-penetrating radar odometer estimation module, and a factor graph model;

[0021] 4) Input the B-scan image pairs obtained in step 2) into the multi-sensor fusion positioning system as input images. Use the saliency detection module to process the B-scan image pairs, identify the effective B-scan image pairs with salient features, and transmit them to the ground penetrating radar mileage estimation module.

[0022] 5) The effective B-scan image pairs with significant features mentioned above are processed using the ground penetrating radar odometer estimation module to obtain one-dimensional relative displacement estimates.

[0023] 6) Finally, the factor graph model uses the cumulative number of inertial measurement units and the fixed time interval as the triggering optimization conditions to perform adaptive discontinuous data fusion on the above one-dimensional relative displacement estimate, the wheel encoder obtained in step 1), and the observation data of the inertial measurement units, and finally obtains the optimal estimate of the motion trajectory of the mobile carrier to be located.

[0024] In step 2), the method for preprocessing the original A-scan image to convert it into a B-scan image pair with overlapping regions is as follows:

[0025] 2.1) A DC drift removal filter is used, and the low-frequency DC component and baseline drift in the original A-scan image are removed by polynomial fitting.

[0026] 2.2) Butterworth filters are used to eliminate high-frequency noise and low-frequency drift;

[0027] 2.3) A finite impulse response bandpass filter is used to retain the effective reflected signal and suppress out-of-band noise;

[0028] 2.4) The attenuation effect of electromagnetic waves during propagation in underground media is expanded and exponentially compensated for gain;

[0029] 2.5) The discrete wavelet transform method is used for multi-resolution analysis, and thresholding is used to suppress random wavelet noise and retain effective reflection features;

[0030] 2.6) Finally, a one-dimensional horizontal Gaussian filter is performed along the spatial axis of the B-scan image, thereby converting the original A-scan image into a pair of B-scan images with overlapping regions.

[0031] In step 4), the method of inputting the B-scan image pairs obtained in step 2) into the multi-sensor fusion positioning system as input images, processing the B-scan image pairs using the saliency detection module, identifying valid B-scan image pairs with salient features, and transmitting them to the ground-penetrating radar mileage estimation module is as follows:

[0032] The saliency detection module includes three parallel horizontal, vertical and standard feature extraction branches and a subsequent feature fusion module;

[0033] In the horizontal feature extraction branch, using convolution operations to extract features along the horizontal direction helps to capture continuous features along the scanning direction and horizontal layers, as shown in the following formula:

[0034] ;

[0035] Among them, F horiz For the feature map extracted by the horizontal feature extraction branch, I input For the input image, W horiz Here, S is the convolution kernel used for horizontal convolution, and S is the stride of the convolution operation.

[0036] Similarly, the vertical feature extraction branch uses convolution operations to extract reflection features along the vertical direction, enhancing the ability to capture vertical reflection features, as shown in the following formula:

[0037] ;

[0038] Among them, F vert I is the feature map extracted by the vertical feature extraction branch. input For the input image, W vert Here, S is the convolution kernel used for vertical convolution, and S is the stride of the convolution operation.

[0039] For general local feature extraction, the standard feature extraction branch uses convolution operations to capture typical texture information within the image, as shown in the following formula:

[0040] ;

[0041] Among them, F std For the feature map extracted by the standard feature extraction branch, I input For the input image, W std Here, S is the convolution kernel used for standard convolution, and S is the stride of the convolution operation.

[0042] After extracting feature maps from the three feature extraction branches mentioned above, these feature maps are concatenated along the channel dimension to form a combined feature map for further processing. The concatenation formula is as follows:

[0043] ;

[0044] Among them, F concat The first step is to concatenate the feature maps. Then, the concatenated feature maps are input into the feature fusion module, which integrates the information and extracts the final fused feature map through convolution operations, as shown in the following formula:

[0045] ;

[0046] Among them, F fusion To fuse feature maps, W fusion S is the convolution kernel used in the final convolution operation in the feature fusion module, and S is the stride of the convolution operation.

[0047] Finally, based on the fusion feature map, the saliency of the B-scan image pairs is detected to determine whether they contain enough features to support reliable displacement estimation. If a B-scan image pair with saliency features is identified from the fusion feature map, it is transmitted as a valid B-scan image pair to the subsequent ground-penetrating radar mileage estimation module for relative displacement estimation, and the estimation result is passed to the factor graph model for multi-sensor fusion optimization. Conversely, if the B-scan image pair is identified as having sparse features or poor quality, the relative displacement estimation and fusion of the image pair are skipped, and the next B-scan image pair is input for processing.

[0048] In step 5), the method for processing the above-mentioned effective B-scan image pairs with significant features using the ground-penetrating radar odometer estimation module to obtain one-dimensional relative displacement estimates is as follows:

[0049] The ground-penetrating radar mileage estimation module employs the OdomNet neural network for estimating the relative displacement between consecutive B-scan image pairs. This module includes two parallel difference detection modules and a similarity detection module, as well as a subsequent fully connected regression module. It utilizes the difference detection module and the similarity detection module to obtain the difference and similarity information of valid B-scan image pairs, respectively. The difference detection module calculates the absolute difference between feature maps and uses an attention mechanism to emphasize motion-related patterns. The similarity detection module evaluates the consistency of inter-frame images through the cosine similarity of high-level feature representations. Then, the features from the above two modules are concatenated and fused before being sent to the fully connected regression module to regress the displacement values.

[0050] In step 6), the method for adaptively fusing the one-dimensional relative displacement estimate, the observation data of the wheel encoder obtained in step 1), and the observation data of the inertial measurement units using the factor graph model with the cumulative number of inertial measurement units and a fixed time interval as triggering optimization conditions, to finally obtain the optimal estimate of the motion trajectory of the mobile vehicle to be located is as follows:

[0051] In the factor graph model, system state variables, including position, velocity, attitude, and other kinematic states, are set as state nodes, while the observation data from ground penetrating radar, wheel encoder, and inertial measurement unit are set as constraint edges connecting the corresponding state nodes.

[0052] Each keyframe includes four state nodes: representing the rotation of 3D R... t With three-dimensional translation P t Composed of six-degree-of-freedom pose and three-dimensional velocity v t 3D gyroscope zero bias b g,t and the three-dimensional accelerometer zero bias b a,t Together, they constitute a 15-dimensional system state vector; containing five types of constraint edges: IMU pre-integration constraint edges are used to associate pose, velocity, and zero bias between adjacent keyframes; BI constraint edges describe the random walk characteristics of the zero bias of the gyroscope and accelerometer, forming constraints on the zero bias state nodes at adjacent time points; WE constraint edges and GPR constraint edges are based on the observation data of the wheel encoder and ground penetrating radar, respectively, and are only included in the factor graph model to constrain the system velocity when there is corresponding observation data between adjacent keyframes;

[0053] By constructing a factor graph model containing the aforementioned state nodes, the multi-sensor fusion problem is transformed into a maximum a posteriori probability estimation problem for node states. The Levenberg-Marquardt linear optimization algorithm is then used to solve the factor graph model, thereby finally obtaining the optimal estimate of the motion trajectory of the mobile vehicle to be located.

[0054] The GPR relative localization method based on saliency detection and discontinuity fusion provided by this invention has the following beneficial effects:

[0055] 1. Front-end data quality dynamic assessment capability

[0056] Compared to existing technologies, this invention utilizes a self-constructed saliency detection module to extract horizontal / vertical / standard three-channel features from B-scan image pairs, achieving real-time discrimination of underground feature saliency. Experiments show that this module reduces the displacement estimation error of the OdomNet model by 7.82%, demonstrating its effectiveness in discriminating the saliency of B-scan image features.

[0057] 2. Adaptive fusion mechanism improves system robustness

[0058] Unlike the continuous fusion strategy of traditional GPR (Geometric Particle Reduction) models, this invention proposes a "discontinuous fusion" mechanism: GPR displacement estimation is only added as a constraint to the factor graph model when the saliency detection module determines that the image pair features are saliency. In tests on the CMU-GPR dataset, compared with the original OdomNet model, the root mean square error of the multi-sensor fusion localization system after adding the saliency detection module was reduced by 7.82% (0.209 cm). Furthermore, the proposed discontinuous fusion strategy significantly improves system-level localization accuracy. Compared with continuous fusion, the overall absolute trajectory error (ATE) is reduced by 18.04% (0.081 m).

[0059] 3. Innovation in optimizer triggering mechanism

[0060] To address the error accumulation problem caused by the lack of interpolation in existing technologies when GPR data is missing for extended periods, this invention adjusts the factor graph optimization strategy, using the accumulated IMU data and a fixed time interval as triggering mechanisms. When GPR data is missing for a long time, the factor graph estimates the system motion state solely based on existing sensor data. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of the GPR relative localization method based on saliency detection and discontinuous fusion provided by the present invention.

[0062] Figure 2 This is a diagram illustrating the configuration of the multi-sensor fusion positioning system in this invention.

[0063] Figure 3 These are preprocessed B-scan images of different scenes in the embodiments of the present invention, where (a) is a parking lot, (b) is a basement, and (c) is a factory ground.

[0064] Figure 4 This is a schematic diagram of the saliency detection module in this invention.

[0065] Figure 5 This is a schematic diagram of the ground-penetrating radar mileage estimation module in this invention.

[0066] Figure 6 This is a schematic diagram of the factor graph model in this invention. Detailed Implementation

[0067] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0068] like Figure 1 As shown, the GPR relative localization method based on saliency detection and discontinuity fusion provided by the present invention includes the following steps performed in sequence:

[0069] 1) The vehicle-mounted ground-penetrating radar, wheel encoder, and inertial measurement unit are used to collect in real time the original A-scan image with a complete trajectory, one-dimensional motion distance, angular velocity, and linear acceleration of the mobile vehicle to be located, as well as the timestamps of the corresponding data collected by the clock, as observation data.

[0070] The wheel encoder, mounted on the wheel of the mobile vehicle, is a sensor that converts the angular displacement of the wheel into an electrical signal to measure the mileage of the mobile vehicle.

[0071] An inertial measurement unit (IMU) is a device used to measure the three-axis attitude angles or angular velocities and three-axis linear accelerations of an object. It typically contains three single-axis accelerometers and three single-axis gyroscopes to measure the linear acceleration and angular velocity of a carrier in three-dimensional space and to calculate the object's attitude through integration.

[0072] 2) The original A-scan images are preprocessed to convert them into B-scan image pairs with overlapping regions;

[0073] The method is as follows:

[0074] 2.1) A dewow filter is used to remove low-frequency DC components and baseline drift from the original A-scan image using a polynomial fitting method;

[0075] 2.2) Butterworth filters are used to eliminate high-frequency noise and low-frequency drift;

[0076] 2.3) A finite impulse response (FIR) bandpass filter is used to retain the effective reflected signal and suppress out-of-band noise; the passband frequency of the finite impulse response bandpass filter is 200-850MHz, and the sampling frequency is 5GHz;

[0077] 2.4) Expand and exponentially compensate (SEC) gain to address the attenuation effect of electromagnetic waves propagating in underground media;

[0078] 2.5) The discrete wavelet transform method is used for multi-resolution analysis, and thresholding is used to suppress random wavelet noise and retain effective reflection features;

[0079] 2.6) Finally, a one-dimensional horizontal Gaussian filter is applied along the spatial axis of the B-scan image to suppress random noise, preserve the edge features of the reflective interface, and improve the image signal-to-noise ratio; thereby converting the original A-scan image into a pair of B-scan images with overlapping regions.

[0080] 3) Construct a multi-sensor fusion positioning system as shown in Figure 2; the system includes a saliency detection module, a ground-penetrating radar odometer estimation module, and a factor graph model;

[0081] 4) Input the B-scan image pairs obtained in step 2) into the multi-sensor fusion positioning system as input images. Use the saliency detection module to process the B-scan image pairs, identify the effective B-scan image pairs with salient features, and transmit them to the ground penetrating radar mileage estimation module.

[0082] like Figure 4 As shown, the saliency detection module includes three parallel horizontal, vertical, and standard feature extraction branches and a subsequent feature fusion module, which are used to capture structural information and texture patterns in different directions in the B-scan image and perform feature fusion to obtain a fused feature map. Finally, the module determines whether to perform ground-penetrating radar mileage estimation on the corresponding B-scan image pair based on whether the fused feature map contains sufficiently saliency underground features.

[0083] In the aforementioned B-scan images, the reflection of underground targets often exhibits a hyperbolic feature with a clear directionality. Simultaneously, the layering structure and material inhomogeneity of the medium contribute rich textural information. The saliency detection module designed in this invention is primarily based on the inherent imaging characteristics of B-scan images. In the horizontal feature extraction branch, convolution operations are used to extract features along the horizontal direction, which helps capture continuous features along the scanning direction and horizontal layering. The formula is as follows:

[0084] ;

[0085] Among them, F horiz For the feature map extracted by the horizontal feature extraction branch, I input For the input image, W horiz Here, S is the convolution kernel used for horizontal convolution, and S is the stride of the convolution operation.

[0086] Similarly, the vertical feature extraction branch uses convolution operations to extract reflection features along the vertical direction, enhancing the ability to capture vertical reflection features, as shown in the following formula:

[0087] ;

[0088] Among them, F vert I is the feature map extracted by the vertical feature extraction branch. input For the input image, W vert Here, S is the convolution kernel used for vertical convolution, and S is the stride of the convolution operation.

[0089] For general local feature extraction, the standard feature extraction branch uses convolution operations to capture typical texture information within the image, as shown in the following formula:

[0090] ;

[0091] Among them, F std For the feature map extracted by the standard feature extraction branch, I input For the input image, W std Here, S is the convolution kernel used for standard convolution, and S is the stride of the convolution operation.

[0092] After extracting feature maps from the three feature extraction branches mentioned above, these feature maps are concatenated along the channel dimension to form a combined feature map for further processing. The concatenation formula is as follows:

[0093] ;

[0094] Among them, F concat The first step is to concatenate the feature maps. Then, the concatenated feature maps are input into the feature fusion module, which integrates the information and extracts the final fused feature map through convolution operations, as shown in the following formula:

[0095] ;

[0096] Among them, F fusion To fuse feature maps, W fusion S is the convolution kernel used in the final convolution operation in the feature fusion module, and S is the stride of the convolution operation. By fusing the features of these three feature extraction branches, the multi-sensor fusion positioning system can extract information from the B-Scan image more comprehensively, thereby enhancing its ability to determine whether the B-Scan image pair contains sufficient salient features.

[0097] Finally, the saliency of the B-scan image pairs is detected based on the fused feature map to determine whether they contain sufficient features to support reliable displacement estimation. If a B-scan image pair with saliency features is identified from the fused feature map, it is passed as a valid B-scan image pair to the subsequent ground-penetrating radar odometer estimation module for relative displacement estimation, and the estimation result is passed to the factor graph model for multi-sensor fusion optimization. Conversely, if a B-scan image pair is identified as having sparse features or poor quality, the relative displacement estimation and fusion of that image pair is skipped, and the next B-scan image pair is input for processing. This selective processing mechanism can effectively filter out low-information image pairs caused by excessive noise or weak features, preventing them from introducing significant errors in the odometer estimation stage, enhancing the robustness of the multi-sensor fusion positioning system, and improving the overall positioning accuracy.

[0098] 5) The effective B-scan image pairs with significant features mentioned above are processed using the ground penetrating radar odometer estimation module to obtain one-dimensional relative displacement estimates.

[0099] The core task of the ground-penetrating radar odometer estimation module is to deduce the one-dimensional relative displacement of the moving vehicle to be located based on the aforementioned effective B-scan images. The basic principle is that during continuous radar scanning along the motion trajectory, adjacent B-scan images exhibit significant data overlap and feature correlation. By quantifying the feature changes or signal correlations between two frames, the sensor's motion increment within the corresponding time period can be indirectly inferred.

[0100] like Figure 5 As shown, the ground-penetrating radar (GPR) mileage estimation module employs the OdomNet neural network for estimating the relative displacement between consecutive B-scan image pairs. This module includes two parallel difference detection modules and a similarity detection module, followed by a fully connected regression module. It utilizes the difference and similarity detection modules to acquire difference and similarity information for valid B-scan image pairs, respectively. The difference detection module calculates the absolute difference between feature maps and employs an attention mechanism to emphasize motion-related patterns. The similarity detection module evaluates the consistency between frames using cosine similarity of high-level feature representations. Then, the features from the two modules are concatenated and fused before being fed to the fully connected regression module to regress the displacement values. This complementary learning of difference and similarity significantly improves the robustness and accuracy of GPR displacement estimation.

[0101] 6) Finally, the factor graph model uses the cumulative number of inertial measurement units and the fixed time interval as the triggering optimization conditions to perform adaptive discontinuous data fusion on the above one-dimensional relative displacement estimate, the wheel encoder obtained in step 1), and the observation data of the inertial measurement units, and finally obtains the optimal estimate of the motion trajectory of the mobile carrier to be located.

[0102] In the factor graph model, system state variables, including position, velocity, attitude, and other kinematic states, are set as state nodes, while the observation data from ground penetrating radar, wheel encoder, and inertial measurement unit are set as constraint edges connecting the corresponding state nodes.

[0103] In traditional factor graph optimization strategies, interpolation is typically performed on high-frequency sensors to align with the timestamps of low-frequency sensors, and the data updates from low-frequency sensors are used as the trigger for optimization. To ensure uniform sampling of sensors for the factor graph model, uniform interpolation of ground-penetrating radar (GPR) data is required. However, considering that GPR data may experience prolonged data gaps after filtering, interpolation is not performed on these data-free periods. Similarly, to maintain the accuracy of wheel encoder data, interpolation is also omitted. Based on these considerations, to achieve discontinuous fusion of one-dimensional relative displacement estimates with observation data from other sensors, the factor graph model optimization trigger condition is set to: data fusion only occurs when the inertial measurement unit (IMU) data accumulates to a certain quantity or reaches a fixed time interval.

[0104] As shown in Figure 6, each keyframe includes four state nodes: representing the state nodes formed by the three-dimensional rotation R. t With three-dimensional translation P t Composed of six-degree-of-freedom pose and three-dimensional velocity v t 3D gyroscope zero bias b g,t and the three-dimensional accelerometer zero bias b a,t Together, they form a 15-dimensional system state vector. The graph contains five types of constraint edges: IMU pre-integration constraint edges are used to associate pose, velocity, and zero bias between adjacent keyframes; BI constraint edges describe the random walk characteristics of gyroscope and accelerometer zero bias, constraining the zero bias state nodes at adjacent time points; WE constraint edges and GPR constraint edges are based on observation data from wheel encoders and ground penetrating radar, respectively, and are only included in the factor graph model to constrain the system velocity when there is corresponding observation data between adjacent keyframes. This method avoids interpolating the ground penetrating radar data portion with long time intervals due to filtering, thereby reducing the errors introduced by it.

[0105] By constructing a factor graph model containing the aforementioned state nodes, the multi-sensor fusion problem can be transformed into a maximum a posteriori probability estimation problem for node states. The Levenberg-Marquardt (LM) nonlinear optimization algorithm is then used to solve the factor graph model, ultimately obtaining the optimal estimate of the motion trajectory of the mobile vehicle to be located. This factor graph model can effectively fuse sensor observation data of different frequencies and characteristics. Furthermore, even when a sensor temporarily fails or its observation quality deteriorates, the system can still maintain its positioning performance by relying on the remaining sensors, demonstrating good robustness and environmental adaptability.

[0106] Example:

[0107] To verify the effectiveness of the method of the present invention, the inventors used the publicly released CMU-GPR dataset from Carnegie Mellon University (CMU) for experimental evaluation. This dataset simultaneously collects data from wheel encoders, ground penetrating radar (GPS), inertial measurement units (IMUs), and high-precision total stations. In this embodiment, the measurements from the IMU, wheel encoder, and GPR are used as inputs to the multi-sensor fusion positioning system, while the trajectory measured by the total station is used as a benchmark for evaluating positioning accuracy. The dataset contains three scenarios: parking lot (gates_g), basement (nsh_b), and factory ground (nsh_h). Complete motion trajectories are selected from each scenario for training and testing to evaluate the generalization ability of the method of the present invention in different underground structures and environments.

[0108] During the training and testing phases of the saliency detection module, each trajectory in the dataset is segmented into multiple B-scan images. Based on prior knowledge, these B-scan images are manually classified into two categories: salient feature class and feature sparse class. The former contains image pairs with clear reflection structures and obvious features, suitable for displacement estimation. The latter covers cases with weak underground reflection signals and blurred structures, in which case the displacement estimation results are often unreliable.

[0109] Figure 3 shows the preprocessed B-scan images of different scenes, where (a) is a parking lot, (b) is a basement, and (c) is a factory floor. It can be observed that the B-scan images of different scenes show significant differences in features. The parking lot scene has a sparser reflection structure and fewer features, while the factory floor scene has richer reflection features and more obvious structural information.

[0110] To evaluate the impact of the saliency detection module on improving the accuracy of ground-penetrating radar (GPR) mileage estimation, the inventors used the GPR relative displacement estimation model OdomNet as a benchmark and compared its performance before and after the introduction of the saliency detection module, as well as between different variant network models. The root mean square error (RMSE) was used as the evaluation metric, defined as:

[0111] ;

[0112] Where n is the total number of samples, y i Let x be the true value of the i-th displacement, and let x be the true value of the displacement. i This is the corresponding predicted value. The smaller the root mean square error (RMSE), the smaller the difference between the predicted displacement and the actual displacement.

[0113] The experimental results of the GPR relative displacement estimation model OdomNet and its variants are shown in Table 1. Here, OdomNet represents the GPR relative displacement estimation model, HVS-OdomNet represents the complete network model with the added saliency detection module, HV-OdomNet represents the variant network model without the standard convolutional branch, HS-OdomNet represents the variant network without the vertical feature extraction branch, and VS-OdomNet represents the variant network model without the horizontal feature extraction branch. All network models are based on the same set of pre-trained weights and no additional retraining was performed after integrating the saliency detection module. Experimental results show that, after introducing the three-branch saliency detection module, the complete network model HVS-OdomNet with the added saliency detection module reduces the average root mean square error by 0.209 cm (a relative reduction of 7.82%) compared to the GPR relative displacement estimation model OdomNet in all scenarios. Furthermore, its performance surpasses all other variant network models.

[0114] ;

[0115] Comparing the full three-branch network model with the full three-branch model reveals that the performance of the two-branch variant network model is generally lower than that of the full three-branch model, and even inferior to the GPR relative displacement estimation model OdomNet in some scenarios. This indicates that embedding a three-branch saliency detection module in a multi-sensor fusion localization system can significantly improve its robustness and accuracy of mileage estimation in different scenarios, and each branch in the saliency detection module contributes positively to the final performance.

[0116] Although the total number of samples differs before and after filtering, the filtered sample error decreases only if the error of the removed samples is greater than the average error of all samples. Therefore, the above results indicate that some prediction errors in the original GPR do indeed originate from sparse B-scan image pairs with underground features. The saliency detection module improves the reliability of GPR relative displacement estimation by identifying and filtering out these low-quality image pairs, enhancing the estimation accuracy of GPR across different backbone networks and scenarios.

[0117] To evaluate the performance of the proposed intermittent fusion mechanism in a global positioning system, comparative experiments were conducted within a factor graph fusion framework. The baseline system included a fusion scheme of an inertial measurement unit (IMU) and a wheel encoder (IMU & WheelEncoder), and a method that continuously fused all GPR image pairs using OdomNet and other GPR one-dimensional odometry estimation methods. The root mean square error (RMSE) of the absolute trajectory error (ATE) considering the trajectory length was used as an evaluation metric for the system's positioning accuracy, defined as follows:

[0118] ;

[0119] Where N is the total number of trajectories, W i Let ATE be the length of the i-th trajectory. i Let be the average trajectory error of the i-th trajectory.

[0120] The average trajectory error ATE for the i-th trajectory is calculated using the following formula:

[0121] ;

[0122] Where N is the total number of poses in the trajectory, j is the position index, and P j Let Q be the attitude estimated by the multi-sensor fusion localization system at time j, and let Q be the attitude estimated by the multi-sensor fusion localization system at time j. j This represents the corresponding true attitude. The lower the error value, the smaller the overall deviation between the estimated trajectory and the true trajectory.

[0123] Table 2 shows the average trajectory error (ATE) results obtained through different methods. "Learned GPR Method" refers to the learning-based sensor model proposed by Carnegie Mellon University (CMU), while "SFM" is a method that combines GPR with the subsurface feature matrix. DEC is an absolute positioning method based on a prior map, utilizing dominant energy curve (DEC) feature extraction and matching. IMU & wheel encoder represent the fusion positioning results of the inertial measurement unit (IMU) and wheel encoder, serving as a benchmark for comparison. OdomNet is a network based on similarity and difference detection for continuous fusion comparison. HVS-OdomNet is the method of this invention, which combines the saliency detection module HVS-Net to achieve intermittent fusion.

[0124] ;

[0125] Experimental results show that the method of this invention, which combines a saliency detection module and a discontinuity fusion strategy, performs best among all the methods listed in the table. In the first two scenarios, the average trajectory error ATE is reduced. In the parking lot gates_g scenario, as shown in Figure 3(a), a typical B-scan image is presented. Compared to the other two scenarios, the features in this scenario are relatively sparse, resulting in the largest one-dimensional localization estimation error. Due to the lack of strong reflections in the underground structure, it is difficult to estimate the platform displacement. In the basement nsh_b scenario, as shown in Figure 3(b), the image contains a large amount of steel reinforcement, resulting in continuous and periodic reflections in the data. Due to the complexity of the feature patterns, this also leads to a high estimation error. Therefore, removing the feature-sparse B-scan images from these two scenarios significantly reduces the absolute trajectory error in subsequent processing. On the other hand, the smallest estimation error is obtained in the factory ground nsh_h scenario (i.e., the case with the clearest reflection features and the lowest level of clutter with the least interference), as shown in Figure 3(c). Although the error increases slightly in this case, the inventors believe this is because the one-dimensional odometry estimation error in the factory ground nsh_h scenario is already minimal in all cases. Furthermore, the trajectory is influenced by both the inertial measurement unit (IMU) and the wheel encoder sensor. Therefore, even if some image pairs with larger errors are excluded from the GPR one-dimensional odometry estimation, these image pairs still contribute positively to the overall trajectory. Compared to the continuous fusion framework, the intermittent fusion framework, enhanced by adding a saliency detection module, reduces the average trajectory error ATE by 0.081 meters, equivalent to an improvement of approximately 18.04%.

Claims

1. A GPR relative localization method based on saliency detection and discontinuity fusion, characterized in that: The method includes the following steps performed in sequence: 1) The vehicle-mounted ground-penetrating radar, wheel encoder, and inertial measurement unit are used to collect in real time the original A-scan image with a complete trajectory, one-dimensional motion distance, angular velocity, and linear acceleration of the mobile vehicle to be located, as well as the timestamps of the corresponding data collected by the clock, as observation data. 2) The original A-scan images are preprocessed to convert them into B-scan image pairs with overlapping regions; 3) Construct a multi-sensor fusion positioning system; the system includes a saliency detection module, a ground-penetrating radar odometer estimation module, and a factor graph model; 4) Input the B-scan image pairs obtained in step 2) into the multi-sensor fusion positioning system as input images. Use the saliency detection module to process the B-scan image pairs, identify the effective B-scan image pairs with salient features, and transmit them to the ground penetrating radar mileage estimation module. 5) The effective B-scan image pairs with significant features mentioned above are processed using the ground penetrating radar odometer estimation module to obtain one-dimensional relative displacement estimates. 6) Finally, the factor graph model uses the cumulative number of inertial measurement units and the fixed time interval as the triggering optimization conditions to perform adaptive discontinuous data fusion on the above one-dimensional relative displacement estimate, the wheel encoder obtained in step 1), and the observation data of the inertial measurement units, and finally obtains the optimal estimate of the motion trajectory of the mobile carrier to be located.

2. The GPR relative localization method based on saliency detection and discontinuity fusion according to claim 1, characterized in that: In step 2), the method for preprocessing the original A-scan image to convert it into a B-scan image pair with overlapping regions is as follows: 2.1) A DC drift removal filter is used, and the low-frequency DC component and baseline drift in the original A-scan image are removed by polynomial fitting. 2.2) Butterworth filters are used to eliminate high-frequency noise and low-frequency drift; 2.3) A finite impulse response bandpass filter is used to retain the effective reflected signal and suppress out-of-band noise; 2.4) The attenuation effect of electromagnetic waves during propagation in underground media is expanded and exponentially compensated for gain; 2.5) The discrete wavelet transform method is used for multi-resolution analysis, and thresholding is used to suppress random wavelet noise and retain effective reflection features; 2.6) Finally, a one-dimensional horizontal Gaussian filter is performed along the spatial axis of the B-scan image, thereby converting the original A-scan image into a pair of B-scan images with overlapping regions.

3. The GPR relative localization method based on saliency detection and discontinuous fusion according to claim 1, characterized in that: In step 4), the method of inputting the B-scan image pairs obtained in step 2) into the multi-sensor fusion positioning system as input images, processing the B-scan image pairs using the saliency detection module, identifying valid B-scan image pairs with salient features, and transmitting them to the ground-penetrating radar mileage estimation module is as follows: The saliency detection module includes three parallel horizontal, vertical and standard feature extraction branches and a subsequent feature fusion module; In the horizontal feature extraction branch, using convolution operations to extract features along the horizontal direction helps to capture continuous features along the scanning direction and horizontal layers, as shown in the following formula: ; Among them, F horiz For the feature map extracted by the horizontal feature extraction branch, I input For the input image, W horiz Here, S is the convolution kernel used for horizontal convolution, and S is the stride of the convolution operation. Similarly, the vertical feature extraction branch uses convolution operations to extract reflection features along the vertical direction, enhancing the ability to capture vertical reflection features, as shown in the following formula: ; Among them, F vert I is the feature map extracted by the vertical feature extraction branch. input For the input image, W vert Here, S is the convolution kernel used for vertical convolution, and S is the stride of the convolution operation. For general local feature extraction, the standard feature extraction branch uses convolution operations to capture typical texture information within the image, as shown in the following formula: ; Among them, F std For the feature map extracted by the standard feature extraction branch, I input For the input image, W std Here, S is the convolution kernel used for standard convolution, and S is the stride of the convolution operation. After extracting feature maps from the three feature extraction branches mentioned above, these feature maps are concatenated along the channel dimension to form a combined feature map for further processing. The concatenation formula is as follows: ; Among them, F concat The first step is to concatenate the feature maps. Then, the concatenated feature maps are input into the feature fusion module, which integrates the information and extracts the final fused feature map through convolution operations, as shown in the following formula: ; Among them, F fusion To fuse feature maps, W fusion S is the convolution kernel used in the final convolution operation in the feature fusion module, and S is the stride of the convolution operation. Finally, based on the fusion feature map, the saliency of the B-scan image pairs is detected to determine whether they contain enough features to support reliable displacement estimation. If a B-scan image pair with saliency features is identified from the fusion feature map, it is transmitted as a valid B-scan image pair to the subsequent ground-penetrating radar mileage estimation module for relative displacement estimation, and the estimation result is passed to the factor graph model for multi-sensor fusion optimization. Conversely, if the B-scan image pair is identified as having sparse features or poor quality, the relative displacement estimation and fusion of the image pair are skipped, and the next B-scan image pair is input for processing.

4. The GPR relative localization method based on saliency detection and discontinuity fusion according to claim 1, characterized in that: In step 5), the method for processing the above-mentioned effective B-scan image pairs with significant features using the ground-penetrating radar odometer estimation module to obtain one-dimensional relative displacement estimates is as follows: The ground-penetrating radar mileage estimation module employs the OdomNet neural network for estimating the relative displacement between consecutive B-scan image pairs; this module includes two parallel difference detection modules and a similarity detection module, as well as a subsequent fully connected regression module. It utilizes a difference detection module and a similarity detection module to obtain difference and similarity information of valid B-scan image pairs, respectively. The difference detection module calculates the absolute difference between feature maps and uses an attention mechanism to emphasize motion-related patterns. The similarity detection module evaluates the consistency of images between frames through the cosine similarity of high-level feature representations. Then, the features from the above two modules are concatenated and fused and sent to the fully connected regression module to regress the displacement values.

5. The GPR relative localization method based on saliency detection and discontinuity fusion according to claim 1, characterized in that: In step 6), the method for adaptively fusing the one-dimensional relative displacement estimate, the observation data of the wheel encoder obtained in step 1), and the observation data of the inertial measurement units using the factor graph model with the cumulative number of inertial measurement units and a fixed time interval as triggering optimization conditions, to finally obtain the optimal estimate of the motion trajectory of the mobile vehicle to be located is as follows: In the factor graph model, system state variables, including position, velocity, attitude, and other kinematic states, are set as state nodes, while the observation data from ground penetrating radar, wheel encoder, and inertial measurement unit are set as constraint edges connecting the corresponding state nodes. Each keyframe includes four state nodes: representing the rotation of 3D R... t With three-dimensional translation P t Composed of six-degree-of-freedom pose and three-dimensional velocity v t 3D gyroscope zero bias b g,t and the three-dimensional accelerometer zero bias b a,t Together, they constitute a 15-dimensional system state vector; containing five types of constraint edges: IMU pre-integration constraint edges are used to associate pose, velocity, and zero bias between adjacent keyframes; BI constraint edges describe the random walk characteristics of the zero bias of the gyroscope and accelerometer, forming constraints on the zero bias state nodes at adjacent time points; WE constraint edges and GPR constraint edges are based on the observation data of the wheel encoder and ground penetrating radar, respectively, and are only included in the factor graph model to constrain the system velocity when there is corresponding observation data between adjacent keyframes; By constructing a factor graph model containing the aforementioned state nodes, the multi-sensor fusion problem is transformed into a maximum a posteriori probability estimation problem for node states. The Levenberg-Marquardt linear optimization algorithm is then used to solve the factor graph model, thereby finally obtaining the optimal estimate of the motion trajectory of the mobile vehicle to be located.

Images