A ranging method based on multimodal deep learning

Abstract

This invention discloses a ranging method based on multimodal deep learning. When GNSS signals are available, the distance and clock offset between the communication base station and the mobile terminal are calculated using GNSS coordinates. When GNSS signals are unavailable, the signal arrival time is calculated using the base station signal, and a clock offset sequence is calculated. The channel impulse response is obtained from the CRS (Constant Reflection System). The clock offset sequence and channel impulse response are input into a multimodal neural network for training. Simultaneously, features are extracted from the channel impulse response. These features are fused and nonlinearly transformed within the network, and the output is adjusted according to the multipath factor weight ratio to predict the ranging error. The prediction result is then used to compensate for the TOA (Time of Arrival) estimation result, improving ranging accuracy. This invention suppresses time-domain ranging errors, as well as ranging errors caused by non-line-of-sight and clock offset, achieving continuous and accurate ranging and positioning of the mobile terminal.

Description

Technical Field

[0001] This invention belongs to the field of positioning and navigation technology, specifically a ranging method based on multimodal deep learning. Background Technology

[0002] Mobile terminals such as drones face two distinct scenarios in low-altitude environments: First, when Global Navigation Satellite System (GNSS) signals are available, they are strong and stable in open airspace, providing high-precision positioning information. Second, when GNSS signals are unavailable, such as in complex low-altitude environments like urban areas, mountain valleys, or forests, GNSS signals are often subject to unintentional interference or malicious spoofing, potentially rendering them unusable and significantly reducing positioning accuracy. Drones and other mobile terminals frequently switch between GNSS-available environments and challenging scenarios where GNSS is unavailable, increasing their reliance on cellular signals as an alternative navigation method.

[0003] However, mobile communication-based ranging methods face challenges from non-line-of-sight (NLOS) and clock skew. In mobile communication ranging, the sources of error vary significantly depending on the propagation environment. In line-of-sight (LOS) environments, ranging errors primarily originate from base station clock skew; due to the dominance of the direct path, multipath effects and NLOS have relatively minor impacts. However, in complex environments (such as urban canyons and indoor spaces), reflections, diffractions, and scattering from buildings and obstacles lead to a significant increase in ranging errors due to multipath propagation and NLOS effects. This makes it difficult for traditional methods to stably adapt to different propagation environments.

[0004] In general, the main challenges of navigation using cellular signals include unknown clock skew at cellular base stations and errors caused by non-line-of-sight (NLOS) conditions. Currently, some methods have been proposed to mitigate multipath effects at the signal processing level; however, many existing methods tend to focus on only one aspect of these challenges at a time. To ensure continuous and reliable navigation in complex environments, both types of errors must be mitigated simultaneously. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, the purpose of this invention is to provide a ranging method based on multimodal deep learning. This method integrates base station clock drift history with CIR observations to jointly suppress temporal ranging errors, while also suppressing ranging errors caused by non-line-of-sight (NLOS) and clock skew, enabling continuous and accurate ranging and positioning for mobile terminals such as drones. The ranging results exhibit strong robustness in mixed NLOS and non-line-of-sight (LOS) environments.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A ranging method based on multimodal deep learning calculates the distance and clock offset between a communication base station and a mobile terminal using GNSS coordinates when GNSS signals are available.

[0008] When GNSS signals are unavailable, the signal arrival time and clock offset sequence are calculated using base station signals. The channel impulse response is obtained from the CRS. The clock offset sequence and channel impulse response are input into a multimodal neural network to train the input clock offset sequence. Simultaneously, features are extracted from the channel impulse response to reflect the path characteristics of the signal during propagation. These features are fused and nonlinearly transformed within the network. The output is adjusted according to the weight ratio of the multipath factor to predict the ranging error. The prediction result is then used to compensate for the TOA estimation result, thereby improving ranging accuracy.

[0009] As a further improvement to the above technical solution:

[0010] When the receiver of the mobile terminal is stationary, the TOA (Time of Arrival) theoretically remains stable. In practice, any observed fluctuations can be attributed to base station clock deviation. When the receiver of the mobile terminal is in motion, the actual position of the receiver is calculated using GPS or RTK, and the change in the actual value of the signal arrival time is derived. If the change in the estimated value of the signal arrival time is inconsistent with the change in the actual value, the difference between the estimated value and the actual value of the signal arrival time is the base station clock deviation. This base station clock deviation is used as a one-dimensional time series to form a clock deviation sequence.

[0011] The model for estimating the signal arrival time is as follows:

[0012]

[0013] in, It is an estimate of the signal arrival time. For the three-dimensional position of the receiver of the mobile terminal, Let represent the three-dimensional position of the i-th base station, and T represent the transpose of the matrix. It is the time step. At the speed of light, , These are the receiver clock offset and the clock offset of the i-th LTE base station, respectively. The deviation is caused by non-line-of-sight and multipath signals. It measures noise. Modeled with a mean of zero and a variance of A Gaussian random sequence.

[0014] The multimodal neural network model includes a model with an attention mechanism to handle clock drift and a model to handle non-line-of-sight errors. The clock drift model uses two or more independent networks to extract features from each input data at different time scales, focusing on the time-varying characteristics of the signal. The features extracted from each time scale are then combined. To fully consider information from different time scales, a multi-head attention mechanism is introduced to measure the importance of different resolutions, where resolution is the sampling rate of the baseband signal. The non-line-of-sight error model extracts features from the channel impulse response through three convolutional layers, combining ReLU activation, zero-padding, and max-pooling operations. A Flatten layer converts the output activation map of the convolutional layers into a one-dimensional format, allowing the data to be passed to the fully connected layers.

[0015] The two models of the multimodal neural network are the echo-CNN-LSTM model and the CNN-based model. The echo-CNN-LSTM model is a parallel CNN-LSTM model with an attention mechanism to handle clock drift. The CNN-based model is used to handle non-line-of-sight errors and is a model based on a convolutional neural network.

[0016] The echo-CNN-LSTM model uses two or more independent CNN-LSTM networks to extract features from each input data at different time scales to focus on the time-varying characteristics of the signal. Then, the features extracted at each time scale are combined. In order to fully consider the information at different time scales, a multi-head attention mechanism is introduced to measure the importance of different resolutions, where the resolution is the sampling rate of the baseband signal.

[0017] The CNN-based model extracts features from the channel impulse response through three convolutional layers. It also combines ReLU activation function, zero padding, and max pooling operations to extract features. The Flatten layer converts the output activation map of the convolutional layers into a one-dimensional format, so that the data can be passed to the fully connected layer.

[0018] The CNN-based model consists of a first two-dimensional convolutional layer, a first max pooling layer, a second two-dimensional convolutional layer, a third two-dimensional convolutional layer, a first batch normalization layer, a first dropout layer, a flatten layer, a first fully connected layer, a second batch normalization layer, a second dropout layer, a second fully connected layer, a third dropout layer, and a third fully connected layer, all connected in sequence. The first two-dimensional convolutional layer is used to extract primary features, and the first max pooling layer is used for temporal downsampling. The second two-dimensional convolutional layer is used to extract depth features, and the third two-dimensional convolutional layer is used to compress the number of channels. Each batch normalization layer is used to prevent overfitting.

[0019] To dynamically adjust the weight allocation ratio of the neural network and evaluate the impact of non-line-of-sight on localization, a multipath factor weight calculation method was designed: First, the channel impulse response is normalized; then, a threshold is set, and the normalized channel impulse response components exceeding this threshold are considered as multipath components, and the set of these multipath components is... Finally, calculate the multipath factor weights. : ,in, and These are preset parameters. .

[0020] The channel impulse response is obtained by demodulating the received signal using a known pilot signal.

[0021] When GNSS is available, the receiver receives the base station signal and obtains an initial estimate of the frame start position through the primary synchronization signal and the secondary synchronization signal. Then, it uses the ESPRIT algorithm to estimate the signal arrival time and then uses GNSS coordinates to calculate the distance and clock deviation between the base station and the mobile terminal.

[0022] The beneficial effects of this invention are:

[0023] (1) The ranging method integrates the base station clock drift history and CIR observations to jointly suppress time-domain ranging errors, while also suppressing ranging errors caused by non-line-of-sight (NLOS) and clock deviations, thereby enabling continuous and accurate ranging and positioning of mobile terminals such as UAVs. The ranging results show strong robustness in mixed NLOS and non-line-of-sight (LOS) environments.

[0024] (2) Based on the existing mobile communication network and infrastructure, a multimodal neural network was designed to balance the network output under different environments. The neural network includes two branches: the echo-CNN-LSTM model and the CNN-based model. The echo-CNN-LSTM model is used to handle the positioning error caused by the base station clock deviation, and the CNN-based model handles the positioning error caused by non-line-of-sight (NLOS). The input data of the echo-CNN-LSTM model is the base station clock deviation sequence, and the input data of the CNN-based model is the channel impulse response (CIR). These two parts of information together constitute the input feature space, which is a multimodal neural network. Moreover, the neural network can adaptively adjust the feature weights (multipath factor weights) to improve the ranging and positioning accuracy.

[0025] (3) The multimodal neural network is specifically designed to mitigate ranging errors caused by base station clock deviations and non-line-of-sight (NLOS) conditions in complex environments. By utilizing multimodal learning, this method achieves at least a 52% improvement in ranging accuracy compared to the baseline model. Experimental results show that the ranging accuracy of this scheme is superior to existing models in GNSS-free environments. Specifically, under non-line-of-sight (NLOS) conditions, the proposed method maintains stable accuracy with an average ranging error of 13.76 meters.

[0026] (4) A multipath factor weighting mechanism based on channel impulse response (CIR) is designed, which can adaptively adjust the influence of CIR characteristics according to the current signal environment and significantly improve performance under NLOS conditions.

[0027] (5) In order to dynamically adjust the weight distribution ratio of the neural network and evaluate the impact of NLOS on localization, a multipath factor weight calculation method was designed to ensure that the multimodal neural network can provide accurate localization results under both LOS and NLOS conditions. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the distance measurement method of the present invention.

[0029] Figure 2 This is a schematic diagram of the architecture of the multimodal neural network of this invention.

[0030] Figure 3 This is a schematic diagram of network performance under different proportions of NLOS datasets in one embodiment of the present invention.

[0031] Figure 4 This is a schematic diagram of an ablation experiment according to an embodiment of the present invention. Detailed Implementation

[0032] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0033] For ease of description, spatial relative terms such as "above," "on top of," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation beyond the orientation of the device as described in the figures. For example, if the device in the figures were inverted, a device described as "above" or "on top of" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0034] A ranging method based on multimodal deep learning, such as Figure 1 As shown:

[0035] When GNSS is available, the receiver receives the base station signal and obtains an initial estimate of the frame start position using the primary synchronization signal (PSS) and secondary synchronization signal (SSS). Then, the ESPRIT algorithm is used to estimate the time of arrival (TOA) and initial Doppler frequency. Next, the distance and clock offset between the base station and the mobile terminal are calculated using GNSS coordinates.

[0036] The receiver refers to a receiver mounted on a drone or other mobile terminal, capable of receiving the raw baseband signal transmitted by a commercial LTE macro base station. In this embodiment, the receiver model used is the USRP X310 software-defined radio.

[0037] When GNSS is available, the clock skew between the receiver and the base station The pseudorange can be estimated by the difference between the pseudorange measured by LTE and the true pseudorange calculated by GPS.

[0038] (1)

[0039] in, It is the pseudorange obtained from LTE measurements. This is the true pseudorange calculated by GPS, where c is the speed of light. It consists of two parts: initial fixed clock bias and clock drift. In this paper, LTE pseudorange (pseudorange obtained through LTE measurement) is measured using the Esprit method of CRS. The bias caused by multipath can be calculated by the complex CIR at the k-th time step, expressed as:

[0040] (2)

[0041] in It is the number of multipath components; and They represent the first The relative magnitude and delay components of each path relative to the first path; (·) is the Dirac function. It is the channel impulse response.

[0042] The ESPRIT algorithm requires the CIR length provided by the MDL method. The MDL method tends to overestimate channel length, leading to incorrect path delay estimation. Therefore, in this paper, the path with the strongest amplitude is selected as the direct path.

[0043] When GNSS signals are unavailable, the TOA (Time of Arrival) is calculated using base station signals, and the clock offset sequence is obtained. The Channel Impulse Response (CIR) is then acquired from the CRS (Channel Response System). The previously acquired clock offset sequence and CIR are fed into a multimodal neural network, which estimates the ranging error. This error is subsequently compensated for in the TOA results, ultimately generating a more accurate pseudorange.

[0044] CRS is a cell-specific reference signal, and its value needs to be extracted by analyzing its frame structure and performing OFDM demodulation.

[0045] The calculation of the clock bias sequence, the channel impulse response (CIR), and the multimodal neural network are described below.

[0046] The ranging model based on the time of arrival (TOA) of cellular signals is as follows:

[0047] (3)

[0048] in, This is an estimate of TOA. For the three-dimensional position of the receiver, Let represent the three-dimensional position of the i-th base station, and T represent the transpose of the matrix. It is the time step. It is the speed of light. , These are the receiver clock offset and the i-th LTE base station clock offset, respectively. The deviation is caused by non-line-of-sight (NLOS) and multipath signals. It measures noise. Modeled with a mean of zero and a variance of A Gaussian random sequence.

[0049] The ranging system in this scheme mainly involves three clock systems: a satellite clock system. Receiver clock system and the clock system of LTE base stations Typically, satellite clocks This can be considered a precise reference time. In mobile communication systems, the crystal oscillators of base stations are often not very precise (for cost reasons), exhibiting slight drift. This drift introduces additional time delays, leading to ranging errors. Furthermore, the accuracy of the receiver clock is usually limited by the characteristics of its own oscillator or the performance of the GPS DO (GPS Disciplined Oscillator). Therefore, the deviation between the base station clock and the receiver clock can be expressed as:

[0050] (4)

[0051] Since the start position of the frame transmitted by the base station cannot be directly obtained in a real-world application environment, the relative change of the start position of the receiver frame between two consecutive moments is calculated instead, thereby obtaining the distance difference between the start positions of the two received frames at consecutive moments. Its mathematical representation is as follows:

[0052] (5)

[0053] It is the starting position of the frame received by the receiver for the kth time; It is the starting position of the frame received by the receiver for the (k-1)th time; It is the change in the frame start position caused by receiver movement; The amount of change in the frame start position due to clock skew; The change in Doppler frequency shift.

[0054] When the receiver is stationary, the TOA (Total Range) should theoretically remain stable, and any observed fluctuations can be attributed to base station clock skew. When the receiver is in motion, its true position can be calculated using GPS or RTK (Real-Time Kinematic), and the actual ranging changes can be derived. If the change in the estimated TOA differs from the actual ranging change, the error (between the estimated TOA and the actual ranging) can be attributed to the base station clock skew. The base station clock skew is then input into the network as a one-dimensional time series.

[0055] Channel Impulse Response (CIR) is a crucial parameter characterizing the state of a wireless channel. It reflects the time-varying characteristics of the wireless channel from the base station to the receiver and describes signal propagation in multipath environments. CIR reflects Channel State Information (CSI), providing channel characteristics for neural networks and revealing the effects of NLOS (Non-Line-of-Sight) and multipath effects. This scheme uses it as the input to the multimodal neural network.

[0056] CIR is calculated based on the LTE reference signal CRS, and the calculation formula is as follows:

[0057] (6)

[0058] in, This is the channel estimate, which is the estimated channel frequency response of the u-th user on the k-th subcarrier. This is the final result obtained by the receiver and is used for subsequent data demodulation, equalization, and signal detection. This represents the frequency response of the actual channel. This represents the noise item caused by interference from other cells. This is the Gaussian white noise of the receiver. This is the LTE reference signal sequence. It is the conjugate of the pilot signal. It is a known pilot signal. It receives signals.

[0059] Ideally, the power of the LTE reference signal is normalized, i.e. Therefore, we can obtain:

[0060] (7)

[0061] To obtain the CIR in the time domain, the frequency domain channel estimate needs to be processed using an inverse Fourier transform (IFFT).

[0062] (8)

[0063] It is the estimated time-domain channel impulse response. It is the actual time-domain channel impulse response. It's noise.

[0064] The overall architecture of the multimodal neural network is as follows: Figure 2 As shown, by combining features from multimodal learning, more relevant features can be extracted from the data, thereby improving model performance.

[0065] The multimodal neural network comprises two branches: an echo-CNN-LSTM model and a CNN-based model. The echo-CNN-LSTM model, based on the CNN-LSTM architecture, handles clock drift and can be viewed as a clock drift feature extraction network. To mitigate non-line-of-sight (NLOS) errors, a CNN-based model is applied in the second branch, which can be considered as a CSI (Channel State Information) feature extraction network. Therefore, the outputs of both models are adjusted according to the weight ratio of the multipath factor.

[0066] First, we introduce the echo-CNN-LSTM model. To enhance and improve the prediction capability of base station clock drift, this scheme designs a parallel CNN-LSTM model with an attention mechanism, named Echo-CNN-LSTM, to fully utilize information at different time resolutions (patch-based and non-patch-based), where a patch represents a feature block. For inputs at different time scales, independent CNN-LSTM networks are used to extract features separately to focus on the time-varying characteristics of the signal. Then, the features extracted from each time scale are combined. Since the importance of base station clock drift features differs at different time scales, a multi-head attention mechanism is introduced to measure the importance of different resolutions in order to fully consider information at different scales. By combining the characteristics of the base station clock drift itself with the adaptive capabilities of deep learning, the network can effectively cope with the uncertainties in complex algorithms. In other words, features at each time scale are extracted by an independent CNN-LSTM network. A multi-head attention mechanism is introduced to evaluate the importance of features at different resolutions. Figure 2 As shown in the figure. Here, the time scale refers to the time of the received signal, and the resolution is the sampling rate of the baseband signal.

[0067] Assume the input time series is X, , where T represents the time step. The input data is divided into multiple time slices, and the original time series is retained as additional input. It is then divided into multiple feature blocks: ,in This represents the i-th time slice. It is a dimension of The vector, where k is the time step size for each block. Let R be the feature dimension, R be the set of real numbers, and N be the number of segments. The original time series X is retained as additional input. For the ... A time slice The features extracted by the CNN-LSTM network are represented as follows:

[0068] (9)

[0069] in, It is the i-th time slice The feature representation obtained after processing by the CNN-LSTM network It refers to a network architecture that combines convolutional neural networks (CNN) and long short-term memory networks (LSTM).

[0070] Equation (9) shows how the model applies to each independent time segment. The processing is performed. Through a separate CNN-LSTM network, the model is able to extract specific features from each time scale (patch) in parallel.

[0071] To integrate features extracted at different time resolutions, the model employs a multi-head self-attention mechanism to evaluate the importance of each feature. This mechanism captures the correlations between different features and assigns them appropriate weights. Let the concatenated feature vector be F:

[0072] (10)

[0073] F contains the features extracted from all N segments. , ,..., and features extracted from the original time series This ensures that the model can perceive the macroscopic characteristics of the overall sequence when making local decisions.

[0074] After passing through a multi-head self-attention mechanism, the fused feature representation is obtained:

[0075] (11)

[0076] in, The features obtained after processing by a multi-head self-attention mechanism are the fused features. This refers to a multi-head self-attention mechanism used to capture the correlation between different features and assign different weights.

[0077] Equation (11) illustrates the feature fusion process. Through the multi-head attention mechanism, the model is able to evaluate the importance of each input feature (from segments with different time resolutions) and combine them in a weighted manner to generate a comprehensive feature that better reflects the global clock drift characteristics.

[0078] The following section introduces a CNN-based model, namely the CSI feature extraction network. The CSI feature extraction network is designed based on a CNN architecture, and its specific structure is shown in Table 1. This branch extracts key features from the input signal using CNN, paying particular attention to non-line-of-sight (NOS) errors. By extracting channel features, CNN can effectively mitigate interference in multipath propagation and NLOS environments, improving ranging accuracy. This network extracts features from CIR through three convolutional layers. It also combines ReLU activation function, zero padding, and max-pooling operations to extract features. Subsequently, the output activation maps of the convolutional layers are converted to a one-dimensional format through a Flatten layer, allowing the data to be passed to the fully connected layers.

[0079] Table 1. CSI Feature Extraction Network Parameters

[0080]

[0081] This network extracts local patterns of signal features through convolutional layers, further reduces dimensionality through pooling layers to decrease computational complexity, and flattens the extracted features for subsequent input. Regularization operations (such as Dropout) in the network structure effectively prevent overfitting, while BatchNorm ensures model stability during training. This design effectively captures the spatial features of CIR while providing high-quality feature representations for downstream tasks.

[0082] During the transition from open to complex environments, ranging errors under line-of-sight (LOS) conditions primarily originate from base station clock drift. In contrast, in complex urban canyon environments with multipath and non-line-of-sight (NLOS) conditions, ranging errors are influenced by both base station clock drift and equipment clock errors, with a significant portion caused by NLOS and multipath effects.

[0083] To dynamically adjust the weight distribution ratio of the neural network and evaluate the impact of NLOS on localization, a simple multipath factor weight calculation method was designed to ensure that the multimodal neural network can provide accurate localization results under both LOS and NLOS conditions.

[0084] This method does not require an additional training process. First, the channel impulse response (CIR) is normalized. Let the original CIR be h(t), and the normalized CIR be... :

[0085] (12)

[0086] This represents the minimum value of CIR. This represents the maximum value of CIR. Next, a threshold value will be set. Normalized CIR components exceeding this threshold are considered multipath components. The set of these multipath components is defined as follows:

[0087] (13)

[0088] in Indicates at time The normalized CIR value. The number of multipath components. Defined as a set The size, that is: .

[0089] Subsequently, the multipath factor weights are calculated using the following formula. :

[0090] (14)

[0091] in, Depend on and Adjustment, and These are preset parameters.

[0092] By normalizing the CIR and setting appropriate thresholds and uniform weighting factors, the feature weight distribution of the neural network under different propagation conditions can be dynamically adjusted, thereby effectively reducing errors caused by multipath propagation and improving accuracy. In a LOS environment (where NLOS has a smaller impact and base station clock skew is dominant), the multipath factor... In other words, the CIR branch contributes less to the final TOA estimation, while the echo-CNN-LSTM branch dominates TOA prediction. In the NLOS environment (where multipath propagation is dominant and clock skew has a relatively small impact), The increased feature weights in the CNN branches make the neural network more reliant on CIR information to adapt to the propagation characteristics in complex environments.

[0093] The following is an experimental case to illustrate this:

[0094] The main equipment used in this experiment was the USRPX310 for signal reception and analysis. This device is equipped with two LTE antennas and one GPS antenna, with an integrated GPS Disciplinated Oscillator (GPSDO) to ensure high-precision signal synchronization. In this experiment, the LTE carrier frequencies were 1835MHz and 2140MHz. The sampling rate was set to 15.36Msps, and the acquired base station signals were recorded on a Lenovo Legion Y9000P. Furthermore, we used an RTK device to calculate the precise GPS location. This device also included two GPS antennas. All equipment was mounted on a DJIT25 drone, and two laptops were provided for data processing.

[0095] In the experimental environment, three LTE base stations were deployed on the rooftops of three buildings. The surrounding sports field encompassed both open LOS (Left Over Right) and NLOS (Normally Unseen) conditions, obstructed by mountains, trees, and buildings. During the experiment, drones flew over the sports field, collecting data from different angles and altitudes.

[0096] In the LOS scenario, 50 sampling points were selected, each spaced 10 meters apart, for static data reception during neural network training. These 50 sampling points, spaced at the same intervals as the LOS condition, were affected by signal occlusion from trees and buildings. After removing null values, the proportions in the dataset are shown in Table 2.

[0097] Table 2. Proportions in the dataset

[0098]

[0099] After data collection, the collected data was used to train the multimodal neural network model offline. The model input included the base station clock drift time series and the CIR extracted from the base station signal. The time step was 12, and the prediction points, training set, and test set were set to 1%, 80%, and 20%, respectively. Through parameter tuning, the optimal training model was determined, and the TOA ranging results were obtained using the ranging framework.

[0100] The multimodal network was compared with several baseline networks in the dataset. The results are shown in Table 3. It can be seen that the proposed model significantly outperforms other models, achieving a ranging accuracy of 13 meters in a mixed LOS and NLOS environment. Figure 3 It can be seen that the performance of other networks significantly decreases as the proportion of NLOS datasets increases. By incorporating CRS features into the network to account for channel information, higher accuracy is achieved under NLOS conditions. The measurement accuracy remains stable as the proportion of NLOS data increases.

[0101] Table 3. Distance Measurement Results

[0102]

[0103] To verify the effectiveness of this scheme, ablation experiments were conducted on multipath factors, patches (feature blocks), and multi-head attention mechanisms. From... Figure 4 It can be seen that this scheme can effectively improve ranging accuracy. The multipath factor can balance the weights of the two network outputs, achieving good ranging results in both LOS and NLOS environments. Through patching and multi-head attention mechanisms, information at different time scales can be effectively extracted.

[0104] It should be noted that this solution focuses on the methods used in the ranging process, while the process from ranging to positioning is existing technology and will not be elaborated upon in this solution.

[0105] Finally, it is necessary to state that the above embodiments are only used to further illustrate the technical solution of the present invention in detail, and should not be construed as limiting the scope of protection of the present invention. Any non-essential improvements and adjustments made by those skilled in the art based on the above content of the present invention shall fall within the scope of protection of the present invention.

Claims

1. A ranging method based on multi-modal deep learning, characterized in that, when GNSS signals are available, the distance between a communication base station and a mobile terminal and the clock bias are calculated using GNSS coordinates; when GNSS signals are unavailable, the signal arrival time is calculated using base station signals and the clock bias sequence is calculated, the channel impulse response is obtained from the CRS, the clock bias sequence and the channel impulse response are input into a multi-modal neural network, the input clock bias sequence is trained; at the same time, the channel impulse response is extracted for features, reflecting the path characteristics of the signal in the propagation process, these features are fused and nonlinearly transformed inside the network, and the prediction error is predicted according to the weight proportion of the multipath factor, the prediction result is compensated to the signal arrival time estimation result, and the ranging accuracy is improved; wherein the multi-modal neural network is a network containing two branch models, including a model for processing clock bias and a model for processing non-line-of-sight error, and the model for processing non-line-of-sight error takes the channel impulse response as input.

2. The ranging method of claim 1, wherein: When the receiver of the mobile terminal is in a stationary state, the observed fluctuations are attributed to the base station clock bias; when the receiver of the mobile terminal is in a moving state, the true position of the receiver is calculated through GNSS or RTK, and the change of the true value of the signal arrival time is derived, if the change of the estimated value of the signal arrival time is inconsistent with the change of the true value, then the difference between the estimated value and the true value of the signal arrival time is the base station clock bias, and the base station clock bias is formed as a one-dimensional time sequence to form a clock bias sequence.

3. The method of Claim 1, wherein: The model for processing clock bias is a model with attention mechanism.

4. The method of Claim 1, wherein: The model for processing clock bias uses two or more independent networks to extract features of each input data at different time scales, in order to focus on the time-varying characteristics of the signal, and then combines the features extracted at each time scale, introduces a multi-head attention mechanism to measure different resolutions, and integrates information at different time scales, with the resolution being the sampling rate of the baseband signal.

5. The method of Claim 1, wherein: The model for processing non-line-of-sight error extracts features from the channel impulse response through three convolutional layers, while combining ReLU activation function, zero padding and maximum pooling operation to extract features, and maps the output activation of the convolutional layer to one-dimensional format through the Flatten layer, so that the data is transmitted to the fully connected layer.

6. The method of claim 5, wherein: The model for processing non-line-of-sight error includes a first two-dimensional convolutional layer, a first maximum pooling layer, a second two-dimensional convolutional layer, a third two-dimensional convolutional layer, a first batch normalization layer, a first Dropout layer, a Flatten layer, a first fully connected layer, a second batch normalization layer, a second Dropout layer, a second fully connected layer, a third Dropout layer, and a third fully connected layer connected in sequence, wherein the first two-dimensional convolutional layer is used to extract primary features, and the first maximum pooling layer is used for time domain down-sampling; the second two-dimensional convolutional layer is used to extract deep features, and the third two-dimensional convolutional layer is used to compress the number of channels, and each batch normalization layer is used to prevent overfitting.

7. The method of Claim 1, wherein: To dynamically adjust the distribution ratio of neural network weights and evaluate the impact of non-line-of-sight on positioning, a multipath factor weight calculation method is designed: first, the channel impulse response is normalized; then, a threshold is set, and the normalized channel impulse response components exceeding the threshold are regarded as multipath components, and the set of these multipath components is ; Finally, the multipath factor weight is calculated : wherein, and are preset parameters, .

8. The method of Claim 1, wherein: The channel impulse response is obtained by demodulating the received signal with a known pilot signal.

9. The method of any one of claims 1-8, wherein: When GNSS is available, the receiver receives base station signals and obtains an initial estimate of the frame start position through the primary and secondary synchronization signals, then estimates the time of signal arrival using the ESPRIT algorithm, and finally calculates the distance and clock bias between the base station and the mobile terminal using the GNSS coordinates.

Images