Self-supervised ultra-wideband positioning method based on improved double-delay deep deterministic policy

Abstract

The application discloses a self-supervised ultra-wideband positioning method based on an improved double-delay deep deterministic strategy, which comprises the following steps: acquiring channel impulse response data CIR and preprocessing; constructing an improved double-delay deep deterministic strategy model to correct UWB ranging error, wherein the improved double-delay deep deterministic strategy model is based on a double-delay deep deterministic strategy gradient model, a double-critic network structure is designed, a space-time feature coding layer based on a self-attention mechanism is introduced, a long-range dependence relationship is captured through a dynamic weight distribution mechanism, and a cross-time-stamp space-time correlation feature mapping is established; a target network delay updating mechanism is introduced, so that the updating frequency of the target network is lower than that of the critic network and the actor network, and the action noise generated by the strategy is regularized; an extended Kalman filter is used to generate pseudo labels, iterative optimization is performed, and a positioning result is obtained. The method is particularly suitable for processing noise and multipath effects in CIR data, and high-precision UWB positioning is realized.

Description

Technical Field

[0001] This invention belongs to the field of ultra-wideband positioning technology, specifically relating to a self-supervised ultra-wideband positioning method based on an improved dual-delay depth deterministic strategy. Background Technology

[0002] Ultra-wideband (UWB), with its wide bandwidth (>500MHz) and extremely short pulse duration (approximately 2ns), can theoretically achieve centimeter-level positioning accuracy, providing a feasible solution for precise indoor positioning. However, in practical applications, especially under non-line-of-sight (NLOS) conditions, UWB positioning faces intolerable ranging errors. When signals pass through obstacles such as walls and furniture, signal delay, multipath effects, and energy attenuation occur, causing ranging errors to reach tens of centimeters or even more in certain complex environments. This severely restricts the large-scale application of UWB technology in scenarios requiring high precision.

[0003] With the rapid development of artificial intelligence technology, deep learning methods have been introduced into the field of UWB ranging error correction, demonstrating significant advantages. Neural network-based methods can learn the complex mapping relationship between signal features and ranging errors from channel impulse response (CIR) data, effectively identifying and correcting errors under NLOS conditions, improving positioning accuracy to the centimeter level. This data-driven approach is more adaptable and robust than traditional signal processing methods, opening up new avenues for improving the performance of UWB positioning systems. A typical example is the hierarchical graph learning model proposed by Gu et al., which, by constructing a base station-tag heterogeneous graph structure and using node embedding technology to explicitly encode the geometric relationship between spatial coordinates and ranging values, improves positioning accuracy by up to 41% compared to traditional methods in non-line-of-sight scenarios. Existing mature models have demonstrated that neighborhood information aggregation strategies based on graph attention mechanisms (GAT) can effectively correct ranging biases caused by multipath effects, while cross-layer graph pooling operations can extract collaborative positioning features from multi-tag systems.

[0004] However, a key bottleneck in existing deep learning methods lies in the model training process. Traditional supervised learning methods require the collection of large amounts of labeled UWB ranging data, which typically relies on high-precision positioning systems as ground truth references. These systems are not only costly but also complex to set up and require specialized personnel to operate. An even more serious challenge is that trained models often face severe generalization problems in deployment environments—when the environment changes, new obstacles are added, or anchor node positions are altered, model performance degrades significantly, requiring re-collection of data and retraining. This severely limits the system's practicality and scalability.

[0005] Reinforcement learning (RL) is a machine learning method that learns optimal decisions through interaction with the environment. It does not rely on static labeled data. Although reinforcement learning has been explored in signal processing and wireless communication, its application in UWB positioning error correction is still in its early stages. This invention is therefore based on that. Summary of the Invention

[0006] The purpose of this invention is to provide a self-supervised ultrawideband (UWB) localization method based on an improved dual-delay deep deterministic strategy. This method utilizes a dual-commentator network structure, a delayed update mechanism, and action noise regularization, introducing a spatiotemporal feature encoding layer based on a self-attention mechanism. This effectively solves the Q-value overestimation problem in traditional deep reinforcement learning algorithms, significantly improving learning stability and model accuracy. This method is particularly suitable for handling noise and multipath effects in CIR data, achieving high-precision UWB localization.

[0007] The technical solution to achieve the purpose of this invention is as follows:

[0008] A self-supervised ultrawideband localization method based on an improved dual-delay depth deterministic strategy includes the following steps:

[0009] Acquire channel impulse response (CIR) data and preprocess it;

[0010] An improved dual-delay deep deterministic policy model is constructed for UWB ranging error correction. The improved dual-delay deep deterministic policy model is based on the dual-delay deep deterministic policy gradient model. A dual-commentator network structure is designed, and a spatiotemporal feature encoding layer based on a self-attention mechanism is introduced. Long-range dependencies are captured through a dynamic weight allocation mechanism, and a spatiotemporal correlation feature mapping across timestamps is established. A target network delayed update mechanism is introduced, so that the update frequency of the target network is lower than that of the commentator network and the actor network. The action noise generated by the policy is regularized to prevent the policy from overestimating the region using the Q function.

[0011] Pseudo-labels are generated using extended Kalman filtering, and the model parameters and learning strategy are dynamically adjusted for iterative optimization to obtain the localization results.

[0012] In the preferred technical solution, the preprocessing includes:

[0013] Complex IQ sampling is performed on the channel impulse response (CIR) data, and the complex IQ sampled array is converted into an RSSI sampled array: I is the real part, Q is the imaginary part, and the RSSI array is cropped to a certain number of samples to focus on signal features near the first diameter;

[0014] Minimum-maximum normalization is applied to the cropped CIR: CIRnorm = (CIR - min(CIR)) / (max(CIR) - min(CIR));

[0015] CIRnorm represents the normalized CIR data, and min(CIR) and max(CIR) represent the minimum and maximum values ​​of the CIR data, respectively.

[0016] The normalized CIR data is subjected to noise filtering and dimensionality reduction to eliminate the influence of hardware transmit power differences and environmental distance variations on absolute signal strength.

[0017] In the preferred technical solution, the improved dual-delay deep deterministic strategy model includes an Actor network and a Critic network. The Actor network acts as a strategy generator, directly analyzing the time-domain characteristics of the channel impulse response and dynamically outputting error correction amounts to adapt to channel distortion. The Critic network plays the role of a value evaluator, constructing an implicit quality evaluation system by analyzing the historical correction effects and the physical consistency of the motion trajectory. The two networks achieve coupled optimization through self-supervised signals formed by environmental feedback: the Actor network generates correction strategies based on the current channel state, while the Critic network evaluates the effectiveness of the strategies through the reward function derived by the trajectory smoothing module, thereby guiding the parameter update direction.

[0018] In a preferred embodiment, the Actor network includes a current Actor network and a target Actor network; the Critic network module includes two current Critic networks Q1 and Q2 and two target Critic networks, and the minimum value output by the two current Critic networks is selected when calculating the target Q value.

[0019] The Actor network has a similar structure to the Critic network, including an input layer that receives preprocessed CIR data, an input layer that adds the action 'a' output from the Actor network to the Critic network as input, a feature extraction layer composed of convolutional layers that captures local features of the data, a transformer encoder layer that captures global dependencies between features, and finally a fully connected layer that enables the Actor network and the Critic network to output action 'a' and action value 'Q', respectively.

[0020] The Ctiric network module obtains the appropriate action value y through the mathematical expression: y=r+γ·min(Q1(s',π(s')),Q2(s',π(s'))).

[0021] Where r is the immediate reward, γ is the discount factor, s' is the next state, π is the policy network, and Q1 and Q2 are the outputs of the two current Critic networks.

[0022] In the preferred technical solution, the regularization of action noise generated by the strategy includes:

[0023] Add noise reduction to the output 'a' of the Actor network to obtain a':

[0024] a'=clip(π(s')+clip(ε,-c,c),a min ,a max )

[0025] Where clip() is the truncation function, ε is random noise, c is the noise amplitude limit, and a min ,a max These are the upper bound and the lower bound of the action, respectively.

[0026] In the preferred technical solution, iterative optimization includes:

[0027] The ranging information after reinforcement learning correction is processed by extended Kalman filtering, combined with historical observation data for state estimation and prediction, and the filtered data is smoothed by a circular buffer to generate pseudo-labels.

[0028] The pseudo-label data is input into the smoothing buffer for processing. After the Euclidean distance to distance value conversion process, the distance benchmark used for reward calculation is obtained. Based on the correlation between the converted distance value and the actual distance measurement, the reward is calculated through the reward function and fed back to the experience replay buffer to provide optimization signals. This drives the continuous iteration of the improved dual-delay deep deterministic strategy model, forming a closed-loop optimization.

[0029] In the preferred technical solution, generating pseudo-tags includes:

[0030] The current Actor network determines the range correction 'a', and the distance 'Δ' is measured at the current time 't'. t The two are combined to obtain the corrected range estimate Δ t ′=Δ t -a;

[0031] Next, the extended Kalman filter is used to process Δ. t ′, in Δ t Convert ′ to position p EKF,t =EKF(Δ t In the process of EKF(), which is an extended Kalman filter, prediction is first made based on the dynamic change model of "range → position". The position at the current moment is estimated using the position state at the previous moment, while taking into account the influence of process noise. Then, combined with the observation equation, the uncertainty of prediction and the uncertainty of observation are balanced by Kalman gain. The prediction result is corrected with the observation information, and finally a more accurate position is obtained. The abstract range information is mapped into position coordinates to realize the conversion from range to position.

[0032] In the preferred technical solution, driving the continuous iteration of the improved dual-delay deep deterministic strategy model to form a closed-loop optimization includes:

[0033] Position p EKF,t The data is stored in a circular buffer C of length N and smoothed. When the buffer is full, new data is added and the oldest data is removed. Then, the average position of the middle position of the buffer is calculated, and the index of the middle position is determined. To ensure correlation with data before and after the buffer, the average position formula is used. x i y i These are position coordinates;

[0034] Then take the average position p avg,m Inverse conversion to corrected range estimate Δ avg,m reward function Quantitative correction effect, Δ′ m The range estimate for the intermediate position m after correction by the Actor network is Δ′. m Δ avg,m The closer the target is, the higher the reward, which guides the agent to optimize range correction. This reward provides feedback for the agent's actions. Since updating the network with single-step samples is inefficient, training is performed by batch sampling from the experience replay buffer using experience data, which helps the agent learn a better range correction strategy.

[0035] This invention also discloses a self-supervised ultrawideband positioning system based on an improved dual-delay depth deterministic strategy, comprising:

[0036] The data acquisition and preprocessing module acquires and preprocesses the channel impulse response (CIR) data.

[0037] The reinforcement learning module constructs an improved dual-delay deep deterministic policy model for UWB ranging error correction. This improved model is based on a dual-delay deep deterministic policy gradient model, employs a dual-commentator network structure, introduces a spatiotemporal feature encoding layer based on a self-attention mechanism, captures long-range dependencies through a dynamic weight allocation mechanism, and establishes a cross-timestamp spatiotemporal correlation feature mapping. A target network delayed update mechanism is introduced, making the target network's update frequency lower than that of the commentator and actor networks, regularizing the action noise generated by the policy, and preventing the policy from overestimating the region using the Q-function.

[0038] The self-supervised learning module uses extended Kalman filtering to generate pseudo-labels, dynamically adjusts model parameters and learning strategies for iterative optimization, and obtains localization results.

[0039] The present invention also discloses a computer storage medium storing a computer program thereon, wherein when the computer executes the computer program, it implements the self-supervised ultrawideband positioning method based on the improved dual-delay depth deterministic strategy described above.

[0040] Compared with the prior art, the significant advantages of this invention are:

[0041] 1. This paper is the first to apply an improved dual-delay deep deterministic policy gradient algorithm to the field of UWB ranging error correction. Through key technologies such as a dual-commentator network structure, a delayed update mechanism, and action noise regularization, it effectively solves the Q-value overestimation problem in traditional deep reinforcement learning algorithms, significantly improving learning stability and model accuracy. This method is particularly suitable for handling noise and multipath effects in CIR data, providing a new technical path for high-precision UWB positioning.

[0042] 2. A hierarchical extraction of temporal features is achieved through a multi-level convolutional structure, progressively stripping away local impulse features and multipath interference patterns from channel distortion. Subsequently, a spatiotemporal feature coding layer (Transformer encoder layer) based on a self-attention mechanism is introduced, capturing long-range dependencies through a dynamic weight allocation mechanism, thus overcoming the limitations of the local receptive field in traditional convolutional operations. This "local-global" collaborative feature learning paradigm, while maintaining the discriminative power of local features, establishes a spatiotemporal correlation feature mapping across timestamps, significantly improving the model's ability to represent complex multipath propagation patterns.

[0043] 3. An innovative self-supervised reinforcement learning training method without labeled data is proposed. By combining trajectory predictability with extended Kalman filtering, it automatically generates iteratively improved pseudo-labels, completely solving the core problem of traditional supervised learning methods relying on large amounts of labeled data. This mechanism enables the system to learn and adapt autonomously in real-world deployment environments, significantly reducing deployment costs and technical barriers, and laying the foundation for the widespread application of UWB positioning technology. Attached Figure Description

[0044] Figure 1 The flowchart shows the self-supervised ultrawideband localization method based on an improved dual-delay depth deterministic strategy.

[0045] Figure 2 The diagram shows the module block diagram of the self-supervised ultrawideband localization method based on the improved dual-delay depth deterministic strategy.

[0046] Figure 3 Here is the overall flowchart of a self-supervised ultrawideband positioning system based on an improved dual-delay depth deterministic strategy;

[0047] Figure 4 This is a laboratory scene image;

[0048] Figure 5This is a diagram of an intelligent robot. Detailed Implementation

[0049] The principle of this invention is to apply an improved dual-delay deep deterministic policy gradient algorithm to the field of UWB ranging error correction. Through key technologies such as a dual-commentator network structure, a delayed update mechanism, and action noise regularization, a spatiotemporal feature encoding layer based on a self-attention mechanism is introduced. A dynamic weight allocation mechanism captures long-range dependencies, establishing a spatiotemporal correlation feature mapping across timestamps. This effectively solves the Q-value overestimation problem in traditional deep reinforcement learning algorithms, significantly improving learning stability and model accuracy. This method is particularly suitable for handling noise and multipath effects in CIR data, providing a new technical path for high-precision UWB positioning.

[0050] Example:

[0051] like Figure 1 As shown, a self-supervised ultrawideband localization method based on an improved dual-delay depth deterministic strategy includes the following steps:

[0052] Acquire channel impulse response (CIR) data and preprocess it;

[0053] An improved dual-delay deep deterministic policy model is constructed for UWB ranging error correction. The improved dual-delay deep deterministic policy model is based on the dual-delay deep deterministic policy gradient model. A dual-commentator network structure is designed, and a spatiotemporal feature encoding layer based on a self-attention mechanism is introduced. Long-range dependencies are captured through a dynamic weight allocation mechanism, and a spatiotemporal correlation feature mapping across timestamps is established. A target network delayed update mechanism is introduced, so that the update frequency of the target network is lower than that of the commentator network and the actor network. The action noise generated by the policy is regularized to prevent the policy from overestimating the region using the Q function.

[0054] Pseudo-labels are generated using extended Kalman filtering, and the model parameters and learning strategy are dynamically adjusted for iterative optimization to obtain the localization results.

[0055] In another embodiment, a computer storage medium stores a computer program thereon, wherein when a computer executes the computer program, it implements the self-supervised ultrawideband positioning method based on the improved dual-delay depth deterministic strategy described above.

[0056] like Figure 2 As shown, this embodiment mainly includes the following:

[0057] 1) UWB signal feature analysis and preprocessing

[0058] This study investigates the propagation characteristics of UWB signals under multipath and NLOS conditions, analyzes the range error-related features contained in CIR data, designs effective CIR data preprocessing methods, including converting complex IQ sampling arrays to RSSI sampling arrays, data pruning, and normalization to reduce data dimensionality and noise impact, and explores optimal preprocessing parameter settings, such as the number of sampling points, normalization methods, and filtering techniques, to provide high-quality input features for subsequent reinforcement learning models.

[0059] 2) Design of a dual-delay deep deterministic policy gradient reinforcement learning framework

[0060] Based on the dual-delay deep deterministic policy gradient algorithm, a reinforcement learning framework suitable for UWB error correction is designed. The state space (preprocessed CIR data) and action space (range error correction values) are explicitly defined. A dual-commentator network structure is designed to reduce Q-value overestimation and improve learning stability. A delayed update mechanism is implemented to ensure the policy network updates based on accurate value assessments. A target policy smoothing technique is designed to prevent the policy from exploiting the overestimation region of the Q-function. A reward function suitable for UWB range error correction is constructed to effectively guide the agent in learning the correct correction behavior.

[0061] 3) Development, application, and verification of self-supervised learning mechanisms

[0062] A self-supervised learning mechanism based on trajectory predictability and filtering smoothing is developed to generate pseudo-labels by utilizing the continuity and smoothness of the moving target trajectory; an extended Kalman filter is implemented for state estimation and trajectory smoothing to effectively reduce the impact of noise; a cyclic buffer smoothing mechanism is designed to improve the accuracy of trajectory estimation through multi-frame data fusion; an iterative improvement process without labeled data is established to automatically improve the quality of pseudo-labels as learning progresses; and multi-environmental adaptability experiments are designed, including scenarios with different obstacle layouts and changes in anchor node positions, to comprehensively verify the performance of the algorithm under changing environmental conditions.

[0063] Specifically, addressing the challenges of high dimensionality, strong noise interference, and high information redundancy in raw UWB data, this study researches and designs an efficient data preprocessing scheme, including complex IQ sampled data conversion, signal feature extraction, and data dimensionality reduction and denoising methods. The plan is to convert the complex IQ sampled array into an RSSI sampled array. I represents the real part and Q represents the imaginary part. The RSSI array is pruned to a certain number of samples, such as 150, to focus on signal features near the first diameter, reducing redundant data and thus improving model efficiency while ensuring accuracy.

[0064] The clipped Channel Impulse Response (CIR) data is subjected to min-max normalization: CIRnorm = (CIR - min(CIR)) / (max(CIR) - min(CIR)). After normalization, the numerical range is compressed to [0,1]. Noise filtering and dimensionality reduction are then applied to the normalized CIR data to eliminate the influence of hardware transmit power differences and environmental distance variations on absolute signal strength, allowing the model to focus on relative characteristics such as signal-to-noise ratio (SNR) and peak shape. This enhances the sensitivity and recognition ability of the reinforcement learning algorithm to complex environmental signal characteristics, laying a solid foundation for the accuracy and stability of subsequent error correction models.

[0065] An improved dual-delay depth deterministic strategy model is constructed to correct UWB ranging errors. The improved dual-delay depth deterministic strategy model is based on the dual-delay depth deterministic strategy gradient model and includes an Actor network and a Critic network.

[0066] The Actor network consists of the current Actor network and the target Actor network; the Critic network module consists of two current Critic networks, Q1 and Q2, and two target Critic networks. Introducing two Critic networks, Q1 and Q2, and selecting the minimum of the two network outputs when calculating the target Q-value effectively reduces the Q-value overestimation problem and improves learning stability.

[0067] The Actor network has a similar structure to the Critic network. It includes an input layer that receives preprocessed CIR data. The Critic network also needs to add the action 'a' output from the Actor network as input. Then, a feature extraction layer composed of convolutional layers is input to capture the local features of the data. Next, a transformer encoder layer is input to capture the global dependencies between features. Finally, through a fully connected layer, the Actor network and the Critic network output the action 'a' and the action value 'Q', respectively.

[0068] The Critic network module obtains the appropriate action value through the mathematical expression: y = r + γ·min(Q1(s',π(s')),Q2(s',π(s'))), where r is the immediate reward, γ is the discount factor, s' is the next state, π is the policy network, and Q1 and Q2 are the outputs of the two current Critic networks. This design is particularly suitable for UWB ranging error correction tasks because noise and multipath effects in CIR data can easily lead to uncertainty in value assessment, and the dual-critic structure can provide a more conservative and stable value estimate.

[0069] Meanwhile, to ensure the stability of the training process, a delayed update mechanism for the target network is introduced. The target network is updated less frequently than the commentator network and the actor network, and parameters are updated using a soft update method: θ'←τθ+(1-τ)θ', where θ is the target network parameter, θ' is the updated parameter, and τ is a very small constant (e.g., 0.001). This delayed update mechanism makes the target value more stable and reduces oscillations during training, which is particularly important for tasks requiring precise estimation, such as UWB error correction.

[0070] Furthermore, during training, noise reduction a' = clip(π(s') + clip(ε, -c, c), a' is added to the actions generated by the policy (the output of the Actor network). min ,a max ), where clip is the cutoff function, ε is random noise, c is the noise amplitude limit, and a min ,a max These are the upper and lower bounds for the action, respectively. This technique prevents the strategy from exploiting deterministic errors in the critic network, enhances exploration capabilities and generalization performance, and enables the system to better adapt to different UWB signal conditions.

[0071] A spatiotemporal feature encoding layer (Transformer encoder layer) based on a self-attention mechanism is introduced. This layer captures long-range dependencies through dynamic weight allocation, overcoming the limitations of the local receptive field in traditional convolutional operations. This "local-global" collaborative feature learning paradigm, while maintaining the discriminative power of local features, establishes a spatiotemporal correlation feature mapping across timestamps, significantly improving the model's ability to represent complex multipath propagation patterns. This provides discriminative and robust feature support for subsequent error correction decisions.

[0072] Specifically, methods for establishing spatiotemporal correlation feature mappings across timestamps include:

[0073] The cross-attention mechanism is used to mine potential temporal dependency patterns in the actions generated by the strategy to obtain the attention calculation results;

[0074] A temporal comparison model is constructed based on the attention calculation results. A strong generalized embedding of temporal modeling is obtained through a self-supervised method. The embedding obtained by temporal modeling is used as temporal features. Spatial domain modeling is performed through a graph convolutional network to obtain the spatiotemporal correlation feature mapping results across timestamps.

[0075] Specifically, using cross-attention mechanisms to uncover potential temporal dependency patterns includes:

[0076] S11: Represent the input of each relevant module as H∈R L×d Where R is a real number, L and d are the length and dimension of the input, respectively, and the initial hidden state H0 = X pX p For temporal feature blocks, for single-head attention, the input sequence H is projected by three projection matrices to obtain the query Q, key K, and value V;

[0077] Query segment Q i and key segment K j The correlation measure between them, c ij for:

[0078]

[0079] Here, ⊙ represents the dot product operation between two matrices of the same size;

[0080] Each query segment Q i Relevance measure c with all key segments i1 ,c i2 ,…,c in The aggregate weights will be obtained by normalization using the Softmax function.

[0081] By concatenating all output Y i The output of the attention module is obtained through the following calculation process:

[0082] H p =Concat(Y1,…,Y m S12: Calculate the temporal reconstruction loss function L R for:

[0083]

[0084] Where, N R This indicates the total number of reconstructions. This represents the actual value of the reconstruction. This represents the result of the temporal cross-modeling for reconstruction.

[0085] The goal of the temporal reconstruction loss function is to minimize the overall loss.

[0086] Specifically, spatial domain modeling using graph convolutional networks includes:

[0087] Calculate the learned adaptive adjacency matrix A k :

[0088] A k =Softmax(Relu(X) c (X c ) T ))

[0089] Where ReLU is the activation function;

[0090] Get the predefined adjacency matrix As The adaptive adjacency matrix A k and A s Combined, we define multi-directional multi-graph space convolution, A s Including the forward spatial adjacency matrix P f and backward adjacency matrix P b The calculation process for spatial domain modeling is as follows:

[0091]

[0092] Among them, H l-1 It is the hidden state of layer l-1. This is the result of spatial modeling, W s1 W s2 and W s3 These are linear transformation parameters;

[0093] The final augmented result is obtained using residual joins and batch normalization.

[0094] Loss function L in the spatial domain modeling stage spa for:

[0095]

[0096] Where, N s This represents the total number of points involved in the airspace loss calculation. It is the output of spatial domain modeling. The label value corresponding to the spatial domain model.

[0097] Based on a self-supervised learning mechanism that leverages the continuity and smoothness of the target trajectory, this project fully utilizes the system's own data features and spatiotemporal correlations to autonomously generate high-quality pseudo-labels. Combining extended Kalman filtering (EKF) and other filtering methods with a cyclic buffer smoothing mechanism, and incorporating historical observation data for state estimation and prediction, it effectively reduces noise interference in trajectory estimation and improves trajectory prediction accuracy. Furthermore, an iterative optimization process requiring no additional manually labeled data is established. As the model is gradually trained and iterated, the quality of pseudo-labels is autonomously improved, and system performance is progressively enhanced. Simultaneously, this project will explore environmental change detection and adaptive adjustment mechanisms, investigating how to perceive changes in the indoor environment (such as obstacle position adjustments and anchor node layout changes) and dynamically adjust model parameters and learning strategies. This will enable the system to quickly adapt to environmental changes and maintain high-precision positioning performance without requiring the re-collection of manually labeled data or complete retraining.

[0098] In another embodiment, a self-supervised ultrawideband positioning system based on an improved dual-delay depth deterministic strategy includes:

[0099] The data acquisition and preprocessing module acquires and preprocesses the channel impulse response (CIR) data.

[0100] The reinforcement learning module constructs an improved dual-delay deep deterministic policy model for UWB ranging error correction. This improved model is based on a dual-delay deep deterministic policy gradient model, employs a dual-commentator network structure, introduces a spatiotemporal feature encoding layer based on a self-attention mechanism, captures long-range dependencies through a dynamic weight allocation mechanism, and establishes a cross-timestamp spatiotemporal correlation feature mapping. A target network delayed update mechanism is introduced, making the target network's update frequency lower than that of the commentator and actor networks, regularizing the action noise generated by the policy, and preventing the policy from overestimating the region using the Q-function.

[0101] The self-supervised learning module uses extended Kalman filtering to generate pseudo-labels, dynamically adjusts model parameters and learning strategies for iterative optimization, and obtains localization results.

[0102] In one possible embodiment, this UWB positioning system utilizes multi-technology collaboration as its core to construct a complete process from data acquisition to closed-loop optimization, achieving accurate ranging and error correction. The detailed process is described below:

[0103] The first step is data acquisition and preprocessing. Raw channel impulse response (CIR) data is acquired using UWB equipment, which serves as the foundational signal source for subsequent processing. The acquired CIR data undergoes preprocessing operations such as filtering and noise reduction to format the data and prepare it for subsequent modules.

[0104] The preprocessed CIR data is input into the UWB ranging system, which outputs initial ranging correlation quantities, namely the channel impulse response data (CIR) and the initial ranging estimate. This serves as the input for improving the dual-delay deep deterministic strategy model, thus formally initiating the error correction process.

[0105] The core reinforcement learning process is then implemented using the Actor-Critic framework. The current Actor network (μ(CIR|θ)) takes CIR as input and generates actions (related to the error correction amount, i.e., the a-related output) based on the current Actor network parameters θ, expressed by the formula μ(CIR|θ). Its output is used for error correction. Achieve initial ranging value correction. Target Actor Network Target Actor Network Parameters According to the soft update rules (τ actor The update coefficients (which control the update magnitude) are used to update the Actor network, providing a stable target reference. The output of this update also participates in the network's updates. The error correction logic assists in the stable iteration of the reinforcement learning process.

[0106] The Actor network output (including CIR, a, and subsequent associated reward R) is stored in the experience replay buffer ((CIR, a, R)). Subsequently, a small batch of data (denoted as small batch b) is randomly sampled and used for Critic network training to break data correlation and improve training stability.

[0107] For Critic network training, the current Critic network Using CIR and a as inputs, based on parameters The loss function is used to evaluate the value of an action and output the Q-value. (B represents the small batch data size, y) i The target value function is output, where i is the iteration number. The network parameters are updated by minimizing this loss to learn action value assessment. Target Critic Network parameters Based on soft update (τ critic The soft update coefficients are updated to calculate the target value. (R i The reward function output (γ is the discount factor, balancing current and future reward weights) provides a supervisory signal for the current Critic network. Based on the Critic network output, the Actor network is updated using a sampling strategy with gradients. The objective function is... By maximizing this objective through gradient ascent, the action generation strategy of the Actor network is optimized, making error correction more aligned with system requirements.

[0108] Next comes the trajectory processing and pseudo-label generation stage. The ranging information, corrected by reinforcement learning, participates in trajectory prediction, and the output data is input into an extended Kalman filter for processing. The extended Kalman filter formula (state prediction) is then used. Covariance prediction Kalman gain Status update Covariance Update Where f is the state transition function, h is the observation function, and Q is the state transition function. k Let F represent the system noise covariance matrix. k Let H be the state transition function at time k. k The Jacobian matrix representing the observation function, z k (Measured values) are used to filter and reduce noise in the trajectory data, which is then output to a circular buffer for smoothing. The filtered data is smoothed again in the circular buffer, generating pseudo-labels (p...). EKF This provides a reference benchmark for the system, used for closed-loop reward calculation and model optimization.

[0109] Finally, there is the reward calculation and closed-loop optimization stage. EKF The input is smoothed by a buffer, and the output p is processed further. avg The process of converting Euclidean distance to distance measurement values ​​(using the Euclidean distance formula) Convert location information into a distance value Δ avg This yields the ranging benchmark used for reward calculation. Based on the transformed ranging value Δ avg The quantities related to the actual ranging of the system are used to calculate the reward R through the reward function, which is then fed back to the experience replay buffer to provide optimization signals for the reinforcement learning module. This drives the Actor-Critic network to continuously iterate, thereby gradually improving the ranging accuracy of the system and forming a closed-loop optimization process.

[0110] This UWB ranging system is based on UWB data acquisition and preprocessing. It uses the initial output of the UWB ranging system to perform error correction through the Actor-Critic reinforcement learning framework. Combined with trajectory prediction, Kalman filtering, and buffer smoothing, it constructs pseudo-labels and a reward mechanism, forming a closed-loop process of "acquisition-preprocessing-ranging-reinforcement learning optimization-filtering smoothing-reward feedback". Each step works together through formulaic algorithms and parameter update rules to continuously iterate and improve ranging accuracy, adapting to the precise ranging needs in complex scenarios.

[0111] The implementation of a self-supervised learning mechanism based on the continuity and smoothness of the target motion trajectory includes the following steps:

[0112] First, the actor determines the range correction 'a', and the target actor generates the correction 'e'. t The two are combined to obtain the corrected range estimate Δ t ′=Δ t -a, where t is time, and Δ t This is the original range-related basic data. Then, an Extended Kalman Filter (EKF) is used to process the Δ... t EKF is a method for state estimation that makes linear approximations to the state equation and the observation equation. In the case of Δ... t Convert ′ to position p EKF,t =EKF(Δ t In the process of EKF (Electronic Kinematics Function), it first makes predictions based on a dynamic change model (state equation) of "range → position," using the position state at the previous moment to estimate the position at the current moment, while considering the impact of process noise (such as environmental interference and model errors). Then, it combines the observation equation, using Kalman gain to balance the uncertainty of prediction and the uncertainty of observation, and uses observation information to correct the prediction results, ultimately obtaining a more accurate position. In this way, EKF maps abstract range information into calculable and processable position coordinates, paving the way for subsequent processes and realizing an effective conversion from range to position.

[0113] After that, position p EKF,t Smoothing is performed on a circular buffer C of length N (N is an odd number, affecting the smoothing effect and adaptability to different motions; a large buffer provides high accuracy on straight paths but is prone to lag when handling maneuvers). When the buffer is full, new data (including p) is processed. EKF,t and related Δ t ′、CIR t a, CIR t (Auxiliary data) is added and the oldest data is removed, then the average position of the middle position in the buffer is calculated. Middle position index. To ensure correlation with data before and after the buffer, the average position is calculated using the following formula:

[0114] x i y i These are location coordinates, averaged to reduce noise impact.

[0115] Then take the average position p avg,m Inverse conversion to corrected range estimate Δ avg,m (by calculating to anchor point a) n (Euclidean distance implementation), reward function Quantitative correction effect, Δ′ m The range estimate for the intermediate position m after correction by the Actor network, Δ m ′ and Δ avg,m The closer the range, the higher the reward, guiding the agent to optimize the range correction (i.e., adjust 'a'). In reinforcement learning, this reward provides feedback for the agent's actions (range correction 'a'). Because updating the network with single-step samples is inefficient, batch sampling containing CIRs is performed from the experience replay buffer. m a m R m (CIR m Training with empirical data (such as training data) improves stability and avoids overfitting. The entire process forms a self-supervised closed loop of "range correction → position transformation → smoothing and noise reduction → reward generation → policy optimization", which helps the agent learn a better range correction strategy.

[0116] like Figure 4 , 5 As shown, a variety of typical indoor environmental scenarios were set up for comprehensive experimental verification and analysis to evaluate the accuracy, stability and environmental adaptability of the developed method under different environmental conditions, providing reliable technical basis and experience reference for practical applications.

[0117] The laboratory is equipped with the following experimental equipment:

[0118] Several Link Track-P UWB sensors; one RS-Helios-5515 32-line LiDAR; one intelligent robot; one Nvidia Jetson Orin Nano and one NVIDIA GeForce RTX 4090 graphics card. The lab is capable of conducting experiments, building datasets, and training models. Furthermore, it possesses comprehensive data analysis and processing software, facilitating in-depth analysis of the collected data.

[0119] Experiments show that, compared with traditional single-modal feature extraction schemes, this architecture improves feature discrimination by 57% in dynamic industrial scenarios, laying a solid foundation for millimeter-level error correction.

[0120] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A self-supervised ultrawideband localization method based on an improved dual-delay depth deterministic strategy, characterized in that, Includes the following steps: Acquire channel impulse response (CIR) data and preprocess it; An improved dual-delay deep deterministic strategy model is constructed to correct UWB ranging errors, and the ranging information is obtained after reinforcement learning correction. The improved dual-delay deep deterministic policy model is based on the dual-delay deep deterministic policy gradient model. It designs a dual-commentator network structure, introduces a spatiotemporal feature encoding layer based on a self-attention mechanism, captures long-range dependencies through a dynamic weight allocation mechanism, and establishes a spatiotemporal correlation feature mapping across timestamps. It also introduces a target network delayed update mechanism, making the update frequency of the target network lower than that of the commentator network and the actor network, regularizing the action noise generated by the policy, and preventing the policy from overestimating the region of the Q function. The improved dual-delay deep deterministic strategy model includes an Actor network and a Critic network. The Actor network includes a current Actor network and a target Actor network. The Critic network module includes two current Critic networks, Q1 and Q2, and two target Critic networks. When calculating the target Q value, the minimum value output by the two current Critic networks is selected. The Actor network takes the preprocessed CIR data as input, and the Critic network takes the action 'a' output by the Actor network and the preprocessed CIR data as input. It then inputs a feature extraction layer composed of convolutional layers to capture the local features of the data, and a transformer encoder layer to capture the global dependencies between features. Finally, through a fully connected layer, the Actor network and the Critic network output the action 'a' and the action value 'Q', respectively. The Ctiric network module obtains the action value y through the mathematical expression: y=r+γ·min(Q1(s',π(s')), Q2(s',π(s'))); Where r is the immediate reward, γ is the discount factor, s' is the next state, π is the policy network, and Q1 and Q2 are the outputs of the two current Critic networks; The ranging information corrected by reinforcement learning is used to generate pseudo-labels using extended Kalman filtering. The model parameters and learning strategy are iteratively optimized based on the pseudo-label data to obtain the localization results.

2. The self-supervised ultrawideband positioning method based on an improved dual-delay depth deterministic strategy according to claim 1, characterized in that, Preprocessing includes: Complex IQ sampling is performed on the channel impulse response (CIR) data, and the complex IQ sampled array is converted into an RSSI sampled array: RSSI = I is the real part, Q is the imaginary part, and the RSSI array is cropped to a certain number of samples to focus on signal features near the first diameter; Minimum-maximum normalization is applied to the cropped CIR: CIRnorm = (CIR - min(CIR)) / (max(CIR) - min(CIR)); CIRnorm represents the normalized CIR data, and min(CIR) and max(CIR) represent the minimum and maximum values ​​of the CIR data, respectively. The normalized CIR data is subjected to noise filtering and dimensionality reduction to eliminate the influence of hardware transmit power differences and environmental distance variations on absolute signal strength.

3. The self-supervised ultrawideband localization method based on an improved dual-delay depth deterministic strategy according to claim 1, characterized in that, The Actor network acts as a policy generator, analyzing the time-domain characteristics of the channel impulse response and dynamically outputting error correction amounts to adapt to channel distortion. The Critic network, on the other hand, plays the role of a value evaluator, constructing an implicit quality evaluation system by analyzing the historical correction effects and the physical consistency of the motion trajectory. The two networks achieve coupled optimization through self-supervised signals formed by environmental feedback: the Actor network generates correction policies based on the current channel state, while the Critic network evaluates the effectiveness of the policies through the reward function derived by the trajectory smoothing module, thereby guiding the direction of parameter updates.

4. The self-supervised ultrawideband localization method based on an improved dual-delay depth deterministic strategy according to claim 1, characterized in that, Regularization of action noise generated by the policy includes: Add noise reduction to the output 'a' of the Actor network to obtain a': a'=clip(π(s')+clip(ε,-c, c),a min ,a max ) Where clip() is the cutoff function, ε is random noise, c is the noise amplitude limit, and a min ,a max These are the upper bound and the lower bound of the action, respectively.

5. The self-supervised ultrawideband localization method based on an improved dual-delay depth deterministic strategy according to claim 1, characterized in that, Iterative optimization includes: The ranging information after reinforcement learning correction is processed by extended Kalman filtering, combined with historical observation data for state estimation and prediction, and the filtered data is smoothed by a circular buffer to generate pseudo-labels. The pseudo-label data is input into the smoothing buffer for processing. After the Euclidean distance to ranging value conversion process, the ranging benchmark used for reward calculation is obtained. Based on the correlation between the converted ranging value and the actual ranging, the reward is calculated through the reward function and fed back to the experience replay buffer to provide optimization signals. This enables the improved dual-delay deep deterministic strategy model to continuously iterate and form a closed-loop optimization.

6. The self-supervised ultrawideband localization method based on an improved dual-delay depth deterministic strategy according to claim 5, characterized in that, Generating pseudo-tags includes: Determine the range correction for the current Actor network. a and the current moment t Measured distance Combined with the corrected range estimate ; Process with extended Kalman filter In the future Convert to position During the process, To extend Kalman filtering, prediction is first made based on a dynamic change model of "range → position". The position at the current moment is estimated using the position state at the previous moment, while taking into account the influence of process noise. Then, combined with the observation equation, the uncertainty of prediction and the uncertainty of observation are balanced by Kalman gain. The prediction result is corrected with observation information to obtain a more accurate position. The abstract range information is mapped into position coordinates to realize the conversion from range to position.

7. The self-supervised ultrawideband localization method based on an improved dual-delay depth deterministic strategy according to claim 6, characterized in that, This enables the improved dual-delay deep deterministic strategy model to continuously iterate and form a closed-loop optimization, including: Location The data is stored in a circular buffer C of length N and smoothed. When the buffer is full, new data is added and the oldest data is removed. Then, the average position of the middle position of the buffer is calculated, and the index of the middle position is determined. To ensure correlation with data before and after the buffer, the average position formula is used. These are position coordinates; Average position based on known anchor point locations Inverse conversion to corrected range estimation The effect of the correction is quantified through a reward function. The middle position m Range estimation corrected by Actor network The closer the target is, the higher the reward, which guides the agent to optimize range correction. This reward provides feedback for the agent's actions. Data training is performed by batch sampling of experience data from the experience replay buffer, enabling the agent to learn a better range correction strategy.

8. A self-supervised ultra-wideband positioning system based on an improved dual-delay depth deterministic strategy, used to implement the self-supervised ultra-wideband positioning method based on an improved dual-delay depth deterministic strategy as described in any one of claims 1-7, characterized in that, include: The data acquisition and preprocessing module acquires and preprocesses the channel impulse response (CIR) data. The reinforcement learning module constructs an improved dual-delay deep deterministic policy model for UWB ranging error correction. This improved model is based on a dual-delay deep deterministic policy gradient model, employs a dual-commentator network structure, introduces a spatiotemporal feature encoding layer based on a self-attention mechanism, captures long-range dependencies through a dynamic weight allocation mechanism, and establishes a cross-timestamp spatiotemporal correlation feature mapping. A target network delayed update mechanism is introduced, making the target network's update frequency lower than that of the commentator and actor networks, regularizing the action noise generated by the policy, and preventing the policy from overestimating the region using the Q-function. The self-supervised learning module uses extended Kalman filtering to generate pseudo-labels, dynamically adjusts model parameters and learning strategies for iterative optimization, and obtains localization results.

9. A computer storage medium having a computer program stored thereon, characterized in that, When the computer executes the computer program, it implements the self-supervised ultrawideband positioning method based on the improved dual-delay depth deterministic strategy as described in any one of claims 1-7.

Images