Deep learning-based multi-target data association method, trajectory tracking method and system

Abstract

This invention belongs to the field of communication and computer technology, and relates to a deep learning-based multi-target data association method, trajectory tracking method, and system. The invention proposes a deep learning-based multi-target data association method that generates simulation features of target motion based on a simulation model of target motion; by improving the input layer of the traditional neural network, the network is made adaptable to AoA and Doppler velocity inputs; and a TCN with temporal modeling capabilities is used as the core processing unit to output the associated motion parameter estimation results. The invention also provides a deep learning-based multi-target data association system, including a motion model construction unit, a global state estimation and global covariance estimation determination unit, a two-dimensional data matrix construction unit, a three-dimensional data matrix transformation unit, a neural network, and a TCN processor. The invention further provides a trajectory tracking method that jointly constructs a tracking trajectory model using an elliptical constraint model and a ray parameter model.

Description

Technical Field

[0001] This invention relates to the fields of communication and computer technology, and in particular to a multi-target data association method, trajectory tracking method, and system based on deep learning. Background Technology

[0002] Data association, a core technology in multi-target perception and tracking systems, primarily aims to accurately identify and match measurements belonging to the same target from observation data collected at different times or from different sensors. This process is crucial for ensuring the accuracy of subsequent state estimation and trajectory recovery. Especially in dynamic and complex environments, multiple targets may coexist and continuously move, leading to highly mixed observation information and significantly increasing the difficulty of data association. However, in the field of wireless sensing, despite improvements in perception accuracy and resolution in recent years, data association methods remain relatively weak compared to mature fields such as computer vision and radar.

[0003] Most systems are designed with the assumption that sensor data can be directly attributed to a single target, ignoring the confusion caused by occlusion, signal reflection, or interference between multiple targets. This becomes a key bottleneck limiting the accuracy and robustness of wireless sensing systems. Traditional wireless sensing methods mainly rely on a few low-level physical features, such as the signal's angle of arrival, Doppler shift, or channel state information, for target state estimation. However, in real-world multi-target scenarios, these features are often redundant or uncertain, especially when targets occlude each other, reflection paths overlap, or environmental multipath effects are significant. A single feature can easily lead to association ambiguity.

[0004] Current mainstream association algorithms, such as Nearest Neighbor (NN) and Joint Probabilistic Data Association (JPDA), while having some theoretical basis, essentially rely on static geometric constraints or simple statistical models, lacking the ability to represent complex dynamic environments. In high-density scenarios, the performance of these algorithms degrades rapidly, making it difficult to effectively distinguish similar observations and severely impacting the overall perception effect of the system.

[0005] In contrast, deep learning technology has made groundbreaking progress in data association in recent years, particularly in tasks such as multi-object tracking (MOT) and radar point cloud registration in computer vision. End-to-end learning frameworks have enabled the modeling of complex spatiotemporal relationships between targets. Its powerful feature abstraction capabilities, temporal modeling capabilities (e.g., through RNNs and Transformers), and ability to approximate nonlinear mappings provide novel approaches to resolving ambiguity issues in data association. Especially in multi-object temporal modeling, deep models can automatically learn more discriminative semantic representations from raw perceptual data, effectively enhancing the distinguishability between observations.

[0006] However, in the field of wireless sensing, related research is still in its early exploratory stages. Current methods mostly focus on feature extraction itself, lacking a systematic time series modeling and joint correlation framework. Furthermore, due to data scarcity, labeling difficulties, and the multi-source channel interference unique to wireless sensing, the application scenarios and stability of neural network models have not yet been fully validated.

[0007] How to closely integrate deep learning methods with wireless signal sensing and explore its potential in multi-target temporal correlation remains an important issue that urgently needs to be addressed. Summary of the Invention

[0008] In traditional wireless sensing systems, the data association process faces the problem of observation ambiguity in dynamic multi-target scenarios. Specifically, when multiple moving entities move in complex environments, multipath effects and mutual occlusion between targets lead to redundancy and uncertainty in physical characteristics such as angle of arrival and Doppler velocity, making it difficult for traditional association algorithms to effectively distinguish observation data belonging to different targets. This problem stems from the over-reliance of existing methods on static geometric constraints or simple statistical models, lacking the ability to dynamically model the temporal evolution of motion behavior, thus causing association ambiguity and directly affecting the accuracy of state estimation and system robustness. These issues lead to cumulative biases in motion state estimation, making it impossible to maintain the continuity of target identity during trajectory recovery. In scenarios with high target density, this problem significantly reduces the system's ability to identify the behavior patterns of moving entities, thereby affecting the reliability of subsequent sensing tasks. Especially when moving entities perform non-stationary movements such as acceleration and turning, traditional methods, lacking adaptive modeling for the diversity of motion behavior, cannot effectively suppress association ambiguity, ultimately leading to the failure of the overall system's sensing function.

[0009] The purpose of this invention is to address the shortcomings of existing technologies by proposing a deep learning-based multi-target data association method. This method generates simulation features of target motion based on a simulation model of the target motion. By improving the input layer of the traditional neural network, the network is made adaptable to AoA and Doppler velocity inputs. A TCN with temporal modeling capabilities is used as the core processing unit to output the associated motion parameter estimation results. This invention also provides a deep learning-based multi-target data association system, including a motion model construction unit, a global state estimation and global covariance estimation unit, a two-dimensional data matrix construction unit, a three-dimensional data matrix transformation unit, a neural network, and a TCN processor. Furthermore, this invention provides a trajectory tracking method that combines an elliptic constraint model and a ray parameter model to construct a tracking trajectory model.

[0010] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a multi-objective data association method based on deep learning, comprising the following steps: Construct multiple motion models that can represent different motion behaviors. The motion models include at least an acceleration motion model, a steady motion model, a deceleration motion model, and a turning motion model. The updated motion model probability is used as a weight to sum the state estimates of each motion model to obtain the global state estimate of the moving subject. At the same time, the covariance estimates of each motion model are summed in a weighted manner to obtain the global covariance estimate of the moving subject. The global state estimates of each moving subject at each time step are constructed into a two-dimensional data matrix including the number of time steps and the number of moving subjects; Calculate the angle of arrival and Doppler velocity between the sensor and the moving subject, and add the angle of arrival and Doppler velocity dimensions to the two-dimensional matrix to construct a three-dimensional data matrix; The angle of arrival in the 3D data matrix is ​​discretized, the Doppler velocity in the 3D data matrix is ​​normalized, and then the 3D data matrix is ​​transformed into a 2D feature vector through tensor reshaping; then feature embedding is performed to map the high-dimensional sparse discretized input to a low-dimensional dense semantic vector space. A TCN processor with temporal modeling capabilities is employed, and the multi-channel convolution in the TCN processor is replaced with complex convolution to maintain the coupling property between the angle of arrival and Doppler velocity. At the same time, the associated motion parameter estimation results are output through an end-to-end learning method.

[0011] As one possible implementation, for each motion model of each moving subject, the updated motion model probability is obtained in the following way: With each other motion model in The probability of each time step is used as a weight, and the transition probabilities from each other motion model to the motion model to be updated are weighted and summed to obtain the prediction probability of the motion model to be updated. The transition probability from any other motion model to the motion model to be updated is related to the motion model in... The product of the probabilities at each time step is divided by the predicted probability to obtain the mixed transition probability from any other motion model to the motion model to be updated. Using the mixed transition probability from any other motion model to the motion model to be updated as the weight, the state estimate and covariance estimate of the motion model to be updated at the previous time step are weighted and summed to obtain the mixed state estimate and mixed covariance estimate of the motion model to be updated. Kalman filtering is applied to both the mixed state estimate and the mixed covariance estimate to obtain the updated state estimate and the updated covariance estimate, respectively. Construct and solve the likelihood function of the motion model to be updated; The product of the likelihood function and the predicted probability is divided by the normalization constant to obtain the probability of the motion model to be updated.

[0012] As one possible implementation, the angle of arrival in the three-dimensional data matrix is ​​discretized, specifically by quantizing the angle of arrival into a discrete range of 0° to 90°. Normalization is performed on the Doppler velocities in the three-dimensional data matrix, specifically including: normalizing the Doppler velocities, with values ​​ranging from [value range missing]. .

[0013] As one possible implementation, by replacing multichannel convolutions in the TCN processor with complex convolutions, the coupling properties between the angle of arrival and Doppler velocity are maintained by the following method: The temporal characteristics of the angle of arrival and Doppler velocity are encoded into complex input signals, where the angle of arrival corresponds to the real part of the complex number and the Doppler velocity corresponds to the imaginary part of the complex number, forming a complex sequence as a whole.

[0014] Secondly, the present invention provides a multi-objective data association system based on deep learning, comprising: The motion model construction unit constructs multiple motion models representing different motion behaviors for each moving subject. The motion models include at least an acceleration motion model, a steady motion model, a deceleration motion model, and a turning motion model. The global state estimation and global covariance estimation determination unit is used to take the probability of each updated motion model as a weight, and sum the state estimates of each motion model to obtain the global state estimate of the moving subject. At the same time, it sums the covariance estimates of each motion model to obtain the global covariance estimate of the moving subject. The two-dimensional data matrix construction unit is used to construct a two-dimensional data matrix that includes the number of time steps and the number of moving subjects from the global state estimate of each moving subject at each time step. The three-dimensional data matrix conversion unit calculates the angle of arrival and Doppler velocity between the sensor and the moving subject, adding the angle of arrival and Doppler velocity dimensions to the two-dimensional matrix to construct a three-dimensional data matrix; The neural network discretizes and normalizes the angle of arrival and Doppler velocity in the three-dimensional data matrix, respectively, and then transforms the three-dimensional data matrix into a two-dimensional feature vector through tensor reshaping; then feature embedding is performed to map the high-dimensional sparse discretized input to a low-dimensional dense semantic vector space. The TCN processor has temporal modeling capabilities. It replaces the multi-channel convolution in the TCN processor with complex convolution to maintain the coupling properties between the angle of arrival and Doppler velocity. At the same time, it outputs the correlated motion parameter estimation results through an end-to-end learning method.

[0015] As one possible implementation, the input layer of the neural network is improved, and the improved input layer includes quantization blocks, normalization blocks, tensor reshaping blocks, and feature embedding blocks; Among them, the quantization block is used to quantize the angle of arrival into a discrete range of 0° to 90°; Normalized blocks are used to normalize Doppler velocities, and their values ​​range from [value range missing]. ; Tensor reshaping blocks are used to convert three-dimensional data matrices into two-dimensional feature vectors; Feature embedding blocks are used to map high-dimensional, sparse, discretized inputs to a low-dimensional, dense semantic vector space through embedding.

[0016] Thirdly, the present invention provides a trajectory tracking method, comprising the following steps: Based on the physical relationship between Doppler velocity and carrier frequency, an elliptic constraint model with the transmitter and receiver as foci is constructed. A ray parameter model is constructed with the receiver position as the origin; By combining the elliptical constraint model and the ray parameter model, and integrating the correlated motion parameter estimation results from the previous time step, the positioning result at this time step can be obtained. By connecting the location results from multiple moments in chronological order, the tracking trajectory can be obtained.

[0017] As one possible implementation, a ray parameter model is constructed with the receiver location as the origin, including: A direction vector is established based on the measured angle of arrival; A ray parameter model is constructed based on the receiver's position and direction vector, specifically as follows: ;in, The coordinates of the receiver's location. It is a direction vector. , This is the measured value of the angle of arrival.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention proposes a multi-target data association method based on deep learning, which can effectively improve the association accuracy and robustness of multi-target motion features in complex environments by leveraging the powerful feature extraction and modeling capabilities of deep learning, thereby significantly improving the trajectory tracking performance of multi-targets. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 A flowchart illustrating the construction of a deep learning-based multi-objective data association method provided in this embodiment of the invention; Figure 2 The present invention provides a structure for a TCN processor based on a deep learning-based multi-objective data association method. Figure 3 This is a schematic diagram of complex convolution operation for a deep learning-based multi-objective data association method provided in an embodiment of the present invention; Figure 4 This is a demonstration of the association results of a multi-objective data association method based on deep learning provided in an embodiment of the present invention. Detailed Implementation

[0020] To facilitate a clear description of the technical solutions in the embodiments of the present invention, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. For example, the first threshold and the second threshold are merely used to distinguish different thresholds and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" are not necessarily different.

[0021] It should be noted that in this invention, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0022] In this invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one" or similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, "at least one of a, b, or c" can represent: a, b, c, a combination of a and b, a combination of a and c, a combination of b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0023] This invention aims to provide a deep learning-based multi-target data association method. Based on a simulation model of target motion, it generates simulation features of target motion. By improving the input layer of traditional neural networks, the network is made adaptable to AoA and Doppler velocity inputs. A TCN with temporal modeling capabilities is used as the core processing unit to output the associated motion parameter estimation results. This invention also provides a deep learning-based multi-target data association system, including a motion model construction unit, a global state estimation and global covariance estimation unit, a two-dimensional data matrix construction unit, a three-dimensional data matrix transformation unit, a neural network, and a TCN processor. This invention further provides a trajectory tracking method that combines an elliptic constraint model and a ray parameter model to construct a tracking trajectory model.

[0024] In a first aspect, the present invention provides a multi-objective data association method based on deep learning, comprising the following steps: Construct multiple motion models that can represent different motion behaviors. The motion models include at least an acceleration motion model, a steady motion model, a deceleration motion model, and a turning motion model. The updated motion model probability is used as a weight to sum the state estimates of each motion model to obtain the global state estimate of the moving subject. At the same time, the covariance estimates of each motion model are summed in a weighted manner to obtain the global covariance estimate of the moving subject. The global state estimates of each moving subject at each time step are constructed into a two-dimensional data matrix including the number of time steps and the number of moving subjects; Calculate the angle of arrival and Doppler velocity between the sensor and the moving subject, and add the angle of arrival and Doppler velocity dimensions to the two-dimensional matrix to construct a three-dimensional data matrix; The angle of arrival in the 3D data matrix is ​​discretized, the Doppler velocity in the 3D data matrix is ​​normalized, and then the 3D data matrix is ​​transformed into a 2D feature vector through tensor reshaping; then feature embedding is performed to map the high-dimensional sparse discretized input to a low-dimensional dense semantic vector space. A TCN processor with temporal modeling capabilities is employed, and the multi-channel convolution in the TCN processor is replaced with complex convolution to maintain the coupling property between the angle of arrival and Doppler velocity. At the same time, the associated motion parameter estimation results are output through an end-to-end learning method.

[0025] Deep learning is a machine learning method that uses multi-layered neural networks to abstract and learn from data, automatically extracting high-level features from raw data to achieve pattern recognition and decision-making tasks.

[0026] Among them, multi-target data association refers to matching observation data acquired by sensors at different times or locations in a multi-target tracking system to the corresponding moving subject in order to solve the correspondence between observation data and moving subject.

[0027] Among them, the motion model is a model used to describe the evolution of a moving subject over time, which can predict the future state of the moving subject. In this embodiment, the motion model includes an acceleration motion model, a steady motion model, a deceleration motion model, and a turning motion model, which respectively characterize different possible motion behavior patterns of the moving subject.

[0028] Among them, the motion model probability represents the likelihood that a certain moving subject will follow a specific motion model at the current moment, which reflects the degree of fit of the motion model to the current observation data.

[0029] State estimation refers to the prediction or inference of kinematic parameters such as the current or future position and velocity of a moving subject. Covariance estimation, on the other hand, represents the uncertainty of state estimation.

[0030] Among them, global state estimation and global covariance estimation refer to the assessment of the overall motion state and its uncertainty of the moving subject based on considering all possible motion models and their probabilities.

[0031] The two-dimensional data matrix is ​​a data structure that organizes data in rows and columns. In this embodiment, the matrix is ​​used to store the global state estimate of each moving subject at each time step, where rows can represent time steps and columns can represent the number of moving subjects.

[0032] The angle of arrival (AoA) refers to the angle of the signal received by the sensor relative to a certain reference direction. Doppler velocity refers to the velocity component corresponding to the change in signal frequency caused by the relative motion between the moving object and the sensor.

[0033] Among them, the three-dimensional data matrix is ​​a data structure that adds the dimensions of angle of arrival and Doppler velocity to the two-dimensional data matrix, which can characterize the temporal, spatial and physical characteristics of the moving subject.

[0034] Discretization is the process of dividing a continuous numerical range into discrete intervals and mapping the original numerical values ​​to these intervals. Normalization is the process of transforming data of different dimensions or ranges to a uniform scale to eliminate the influence of dimensions and improve the stability of data processing.

[0035] Tensor reshaping refers to changing the dimensional arrangement or shape of a multidimensional array without altering its constituent data elements. Two-dimensional feature vectors are a flattened representation formed after tensor reshaping, converting high-dimensional data into sequences for easier processing by subsequent neural networks.

[0036] Feature embedding is the process of mapping high-dimensional, sparse input data to a low-dimensional, dense vector space. In this low-dimensional space, points with similar semantic features are closer together, thus enabling the capture of the inherent relationships between data.

[0037] Among them, the TCN processor is a temporal convolutional network processor that has the ability to process temporal data. By using techniques such as causal convolution and dilated convolution, it can capture long-distance dependencies.

[0038] Complex convolution is a convolution operation whose input and output are both complex numbers. Through complex convolution, real and imaginary information can be processed, thus preserving the coupling properties between physical quantities.

[0039] The coupling property refers to the interrelationship between two or more physical quantities. In this embodiment, there is a physical correlation between the angle of arrival and the Doppler velocity, and complex convolution helps to maintain this correlation.

[0040] End-to-end learning refers to the entire process from raw input data to final output results being learned and optimized by a neural network model, without the need for manual design of intermediate features or modules.

[0041] Among them, the motion parameter estimation results refer to the kinematic parameters of the moving subject obtained after data association and deep learning processing.

[0042] Specifically, this embodiment provides a deep learning-based multi-object data association method, which estimates motion parameters in multi-object scenarios through a series of steps. See [link to relevant documentation]. Figure 1 The specific steps for establishing a multi-objective data association method in deep learning are as follows: First, simulation models based on target motion are proposed. These motion models can include acceleration, steady-state, deceleration, and turning motion models. As an example, a constant velocity model, a constant acceleration model, and a constant turning rate model can be preset for each moving subject. These models can describe the motion laws of the moving subject in scenarios such as linear acceleration, uniform linear motion, linear deceleration, and curvilinear motion. By employing multiple motion models, various dynamic behaviors that the moving subject may exhibit can be covered, thereby improving the accuracy of motion state description.

[0043] To generate angle of arrival (AoA) and Doppler velocity that conform to realistic motion patterns, this embodiment of the invention employs an interactive multi-model (IMM) algorithm for feature simulation. This algorithm integrates typical motion features and probabilities, runs in parallel across multiple motion models (acceleration, steady motion, deceleration, and turning models, etc.), and dynamically adjusts the weights of each model. This effectively simulates the diverse motion behaviors of moving targets in complex environments, thereby generating dynamically changing feature parameters. This method is particularly suitable for scenarios where the sensor's motion state is uncertain or frequently changes.

[0044] As an example, suppose the moving target has Types of motion states, corresponding to A type of motion model. The first... The state equations for each model are: The observation equation is: in It is a model The state transition matrix, It is a model The observation matrix and The covariances are respectively and Zero-mean Gaussian white noise.

[0045] Secondly, a global covariance estimate of the moving subjects is obtained. The likelihood of each motion model to the observed data at the current time is calculated and converted into a probability. These probabilities are then used as weighting coefficients to linearly combine the state estimates independently generated by each motion model, thus obtaining a global state estimate. Similarly, the covariance estimates of each motion model are also weighted and summed using their corresponding probabilities to obtain a global covariance estimate, which reflects the uncertainty of the global state estimate. This weighted summation method can integrate the advantages of different motion models and avoid the limitations of a single model.

[0046] Specifically, the global state estimates of each moving subject at each time step are constructed into a two-dimensional data matrix that includes the number of time steps and the number of moving subjects. (T,M) Then, a mathematical physics model is constructed based on the relative positions of the sensor and the moving target to calculate the angle of arrival (AoA) and Doppler velocity (element N), thus forming a three-dimensional data matrix based on the two-dimensional data matrix. Specifically, firstly, the global state estimates of each moving subject at each time step can be constructed into a two-dimensional data matrix including the number of time steps and the number of moving subjects to quantify and standardize the data. Then, the angle of arrival and Doppler velocity between the sensor and the moving subject are calculated, adding the angle of arrival and Doppler velocity dimensions to the two-dimensional matrix to construct a three-dimensional data matrix. That is, each element of the two-dimensional matrix is ​​appended with its corresponding angle of arrival and Doppler velocity information, thereby extending the data structure from two-dimensional to three-dimensional, forming a comprehensive data representation that includes time, moving subjects, and observation characteristics.

[0047] In this scheme, the output of the proposed simulation model is represented by the dimension of the velocity vector, denoted as N, where N represents the dimension of the velocity vector. At each moment, the velocity vector has one x-axis component and one y-axis component, i.e., N = 2. This is then combined with the global state estimation of each target's data at each moment. The constructed two-dimensional matrix This will yield a three-dimensional data matrix. .

[0048] in, M represents the number of time steps; N represents the number of moving objects; and N is the dimension of the velocity vector.

[0049] Next, since traditional neural networks are not well-suited for the generated 3D matrix, improvements are needed to the input layer. The angular quantization is performed to a predetermined discrete range, while the Doppler velocity can be linearly scaled to a standard range. This processed data is then transformed into a flattened 2D feature vector through a tensor reshaping operation, facilitating its input to the neural network. This 2D feature vector is then fed into a feature embedding layer, which converts the high-dimensional sparse discretized input into a low-dimensional dense vector representation. This embedding operation helps capture the nonlinear relationships between features and reduces the computational complexity of subsequent processing.

[0050] Finally, a TCN processor with temporal modeling capabilities is used, and the multi-channel convolution in the TCN processor is replaced with complex convolution to maintain the coupling property between the angle of arrival and Doppler velocity. At the same time, the associated motion parameter estimation results are output through an end-to-end learning method.

[0051] This step uses a temporal convolutional network (TCN) as the core feature processing architecture. See the TCN network structure for details. Figure 2 This network structure employs dilated causal convolution, which expands the temporal receptive field while preventing the leakage of future information, making it suitable for the time-series data from the previous step. Simultaneously, AoA and Doppler velocity jointly characterize the target's dynamic behavior in the spatial and velocity dimensions, exhibiting a natural co-evolutionary relationship. This inherent coupling structure between features contains rich high-order semantic information, which is crucial for enhancing the model's ability to recognize target behavior patterns.

[0052] Multi-channel convolution for AoA and Doppler velocity is a relatively intuitive operation, and this method can capture the evolution of features over time to some extent. However, since the two branches are independent of each other during the modeling process, they cannot explicitly express the potential physical coupling relationship between AoA and Doppler velocity, leading to information loss or inconsistency in feature fusion. Therefore, this step introduces a complex convolutional neural network to model the correlation between AoA and Doppler velocity in a more natural way.

[0053] Through the above technical solution, this embodiment first constructs multiple motion models capable of representing different motion behaviors, and then weights and sums the state estimates of each motion model using the updated motion model probabilities to obtain a global state estimate of the moving subject. This method can capture the dynamic behavior of the moving subject, such as vehicle acceleration, deceleration, smooth driving, and turning, avoiding the limitations that motion models may have in the scene. Existing technologies often rely on constant speed or constant acceleration models, which are difficult to describe the switching of the moving subject between different motion modes, leading to a decrease in correlation accuracy.

[0054] Secondly, this embodiment constructs a two-dimensional data matrix by estimating the global state of each moving subject at each moment, and then adds the dimensions of angle of arrival and Doppler velocity to form a three-dimensional data matrix, thus realizing the representation of the temporal, spatial, and physical characteristics of multiple targets. This data organization method provides clear input for subsequent deep learning models to process multi-source information. In existing technologies, physical features such as angle of arrival and Doppler velocity are often processed independently or simply pieced together, failing to fully utilize their correlation, resulting in information redundancy or loss.

[0055] Furthermore, the angle of arrival and Doppler velocity in the three-dimensional data matrix are discretized and normalized, then transformed into two-dimensional feature vectors through tensor reshaping, followed by feature embedding. This maps the high-dimensional, sparse, discretized input to a low-dimensional, dense semantic vector space. This series of preprocessing steps addresses the potential scale inconsistencies, distribution differences, and high-dimensional sparsity issues in the original sensor data, providing input features for deep learning models. Existing techniques are typically simple in feature processing, making it difficult to extract discriminative semantic features from raw data.

[0056] Crucially, this embodiment employs a TCN processor with temporal modeling capabilities and replaces the multi-channel convolution in the TCN processor with complex convolution. By encoding the temporal features of angle of arrival and Doppler velocity as complex input signals, complex convolution can preserve the coupling properties between these two physical quantities. For example, when a vehicle turns, the changes in its angle of arrival and Doppler velocity are correlated. Existing multi-channel convolution techniques typically treat these features as independent channels, potentially ignoring or disrupting the physical coupling relationship, leading to compromised accuracy. This embodiment, through complex convolution, ensures that these physical quantities maintain their intrinsic connection during deep learning processing, improving the model's understanding of motion patterns and the accuracy of data association.

[0057] Finally, through an end-to-end learning approach, this embodiment can directly output the correlated motion parameter estimation results. This end-to-end method avoids the error accumulation problem in traditional multi-stage correlation algorithms, enabling the entire system to be optimized, thereby achieving data correlation and motion parameter estimation in multi-target wireless sensing environments.

[0058] Based on the above examples, the deep learning-based multi-target data association method proposed in this embodiment demonstrates a technical contribution to solving the problem of multi-target data association in the field of wireless sensing.

[0059] As one possible implementation, for each motion model of each moving subject, the updated motion model probability is obtained in the following way: With each other motion model in Using the probability of each time step as a weight, the transition probabilities from other motion models to the motion model to be updated are weighted and summed to obtain the prediction probability of the motion model to be updated. ; The transition probability from any other motion model to the motion model to be updated is related to the motion model in... The product of the probabilities at each time step, divided by the prediction probability. To obtain the mixed transition probabilities from any other motion model to the motion model to be updated. ; Using the mixed transition probability from any other motion model to the motion model to be updated as the weight, the state estimate and covariance estimate of the motion model to be updated at the previous time step are weighted and summed to obtain the mixed state estimate and mixed covariance estimate of the motion model to be updated. Kalman filtering is applied to both the mixed state estimate and the mixed covariance estimate to obtain the updated state estimate and the updated covariance estimate, respectively. Construct and solve the likelihood function of the motion model to be updated; The product of the likelihood function and the predicted probability is divided by the normalization constant to obtain the probability of the motion model to be updated.

[0060] Kalman filtering is an optimal linear filter that can effectively process noisy measurement data and provide optimal state estimates.

[0061] Specifically, in calculating the predicted probability of the motion model to be updated At that time, this step aims to take into account the previous time step. This involves determining the probabilities of all other motion models and their likelihood of transitioning to the current motion model to be updated. As an example, for each motion model j to be updated, iterate through all possible motion models i, and then assign model i to the current motion model to be updated. probability at time step The transition probability from model i to model j Multiply them, then sum all the products to get the predicted probability. .

[0062] Obtain the hybrid transition probability from any other arbitrary motion model to the motion model to be updated. This step calculates the posterior probability that the previous motion model was i, given that the motion model to be updated at the current time step is j. This is essentially an application of Bayes' theorem to determine which previous motion model contributes most to the current model j under the current observations. As an example, for each model pair (i, j), the following steps are performed: Then divide it by the predicted probability. To obtain the mixed transition probability from model i to model j .

[0063] When using the mixed transition probabilities from other arbitrary motion models to the motion model to be updated as weights, and weighting and summing the state estimate and covariance estimate of the previous time step of the motion model to be updated to obtain the mixed state estimate and mixed covariance estimate of the motion model to be updated, the purpose of this step is to obtain the mixed state estimate and mixed covariance estimate of the motion model to be updated based on the mixed transition probabilities. Compared with the state estimates X of each motion model at the previous moment i Covariance Estimation A weighted combination is performed to generate a "mixed" initial state and covariance for the motion model j to be updated. This provides more accurate prior information for subsequent Kalman filtering. As an example, for the motion model j to be updated, its mixed state estimate is the sum of the state estimates of all models i at time k-1 and their corresponding mixed transition probabilities. The sum of the products of these factors. Similarly, the mixture covariance estimate is the sum of the covariance estimates of all models i at time k-1 and their corresponding mixture transition probabilities. The sum of the products must also take into account the propagation of state estimation errors.

[0064] When Kalman filtering is applied to both the mixed state estimate and the mixed covariance estimate to obtain updated state and covariance estimates respectively, this step utilizes Kalman filtering to optimize the mixed state and covariance, thus fusing the sensor observation data at the current moment. As an example, the obtained mixed state estimate and mixed covariance estimate are used as inputs to the prediction step of the Kalman filter, and then, combined with the sensor measurements at the current moment, more accurate state and covariance estimates are updated through Kalman gain calculation.

[0065] In constructing and solving the likelihood function of the motion model to be updated, this step aims to evaluate the degree of fit between the current sensor observations and the motion model j to be updated. The likelihood function represents the probability of the observed data occurring given model j and its updated state estimate. As an example, based on the updated state estimate and covariance obtained from Kalman filtering, and the measurement at the current time, the probability density function of the measurement residual is calculated. Typically, it is assumed that the measurement noise follows a Gaussian distribution, thus constructing a Gaussian likelihood function.

[0066] When obtaining the probability of the motion model to be updated by dividing the product of the likelihood function and the predicted probability by the normalization constant, this step, based on Bayes' theorem, combines the likelihood function and the predicted probability to calculate the posterior probability of the motion model j at the current time. This probability reflects the likelihood that the motion model is correct after considering the current observation data. As an example, the calculated likelihood function value is shown below. With predicted probability Multiply them, and then use the sum of this product of all motion models as a normalization constant to normalize the product of each motion model, thereby ensuring that the sum of the probabilities of all motion models is 1.

[0067] As an example, the global covariance estimate of any motion model can be obtained by the following method: (1) Input interaction By combining the state estimates and covariances of each model from the previous time step, as well as the mixture weights calculated based on the model transition probabilities, a mixture initial state and covariance matrix corresponding to each model is generated, which serves as the input for the current time step. As an example, the first... The predicted probabilities of each model are: in It is a model To model The transition probability, It is a model In the The probability at each time step. Model To model The mixed transition probability is: Model The mixed state estimate is as follows: Model The mixed covariance estimate is: in, and These are the state estimate and covariance estimate from the previous time step, respectively.

[0068] (2) Kalman filtering Given a mixed state estimate and mixed covariance Given the input, the Kalman filter performs a measurement update, generating an updated state estimate. and its covariance .

[0069] The state estimate after Kalman filtering is: The covariance estimate after Kalman filtering is: in It is the Kalman gain. The observation equation for the (k-1)th time step; Let be the observation matrix of model j at time step k-1.

[0070] (3) Model probability update The model probability is updated through the likelihood function, where the model... The likelihood function expression is: in, It is the covariance matrix of the observation noise. It is a parameter related to the mixture covariance, which is used to update... , Used for updating (i.e., in (1)) ), Used to update the mixed state estimate for this moment.

[0071] Model probability for: in It is a normalization constant.

[0072] (4) Output interaction.

[0073] Using the updated model probabilities as weights, the posterior state estimates and covariances of each model are weighted and fused to obtain the final global estimate. The expression for the global state estimate is: The global covariance estimate is: in, Is with the model probability The relevant parameters are used for updating , It represents the probability of a certain model at a certain time step. Each iteration requires the covariance estimate from the previous time step, and the covariance from the current time step is used to estimate the probability. Update. That is, the global covariance estimation is used to iterate the relevant parameters at the next time step, the three-dimensional data matrix. (T, M, N）The elements in the table represent: At time step T, a total of T-1 iterations were performed; M is the number of moving targets. For example, if there are 3 moving targets, then M is 3. For each moving target, its own state estimation is performed at T time steps. A total of M targets' state estimations were performed at T time steps, i.e., M moving targets, and the parameters (P and ...) of each moving target. The system underwent T-1 iterations of updates.

[0074] Through the above technical solution, this application enables dynamic and accurate updating of motion model probabilities. This update mechanism fully considers the transition characteristics between different motion models and the probability distribution of the previous moment, allowing the weight of each motion model at the current moment to more accurately reflect its matching degree with the actual target motion behavior. This effectively solves the problem of degraded data association performance caused by untimely or inaccurate motion model probability updates in complex multi-target scenarios. By fusing multi-model information and combining Kalman filtering for state estimation, this application significantly improves the accuracy and robustness of global state estimation of the moving subject, thereby providing high-quality input for subsequent deep learning processing and ultimately improving the overall performance of multi-target data association.

[0075] As one possible implementation, the angle of arrival in the three-dimensional data matrix is ​​discretized, specifically by quantizing the angle of arrival into a discrete range of 0° to 90°. Normalization is performed on the Doppler velocities in the three-dimensional data matrix, specifically including: normalizing the Doppler velocities, with values ​​ranging from [value range missing]. .

[0076] The arrival angle is quantized into a discrete range of 0° to 90°, which aims to focus on the movement of the target in front of the sensor or within a specific sector. This is the main area of ​​concern in many practical applications, such as in vehicle radar or drone detection, where the focus is usually on targets in front or to the side. Quantization is the process of mapping continuous angle-of-arrival values ​​to a finite discrete set. It can effectively reduce data dimensionality, lower computational complexity, and help eliminate measurement noise. As an example, the continuous angle range of 0° to 90° can be divided into a predetermined number of equally or unequally spaced discrete intervals, and each measurement value can be classified into its respective interval; alternatively, a clustering-based approach can be used to dynamically determine the quantization interval based on the distribution of historical data.

[0077] The Doppler velocity is normalized, and its value range is [value range missing]. This aims to filter out outliers or irrelevant extreme velocities while ensuring numerical consistency in the input data, preventing certain features from dominating deep learning model training due to their excessively large numerical ranges. Doppler velocity is the radial velocity of a target relative to the sensor, reflecting the rate at which the target approaches or moves away from the sensor. Normalization is the process of scaling the raw Doppler velocity measurements to a predefined standard range, which helps accelerate model convergence and improve the model's generalization ability. Implementation methods may include: scaling values ​​beyond... The Doppler velocity range is truncated, and the truncated values ​​are then mapped to a smaller range, such as [-1,1] or [0,1], through a linear transformation; alternatively, the Z-score standardization method can be used to adjust the mean and standard deviation within a defined range to make the data distribution more consistent with a standard normal distribution.

[0078] As an example, to improve the input layer of a traditional neural network, methods such as quantization and normalization, tensor reshaping, and feature embedding can be used to adapt the network to inputs with AoA and Doppler velocity. The specific methods are as follows: ① Quantification and standardization The AoA measurement value was quantized into a discrete interval of 0-90 degrees, and the Doppler velocity was normalized to limit its range. ; ② Tensor Reshaping The original three-dimensional data matrix (in Indicates the time step. Represents the target quantity. Convert the feature dimension to a two-dimensional feature vector. (in This operation, while preserving the joint semantics of time and target, compresses the spatiotemporal dimension into the sample index dimension, so that each "spatiotemporal unit" (i.e. a specific target at a specific time) is represented as an independent but semantically complete feature vector; ③ Feature embedding By projecting high-dimensional, sparse, discretized inputs onto a low-dimensional, dense semantic vector space through an embedding map, the embedding process not only significantly compresses the input dimensionality, effectively alleviating the curse of dimensionality caused by sparse representation, but more importantly, by uniformly encoding each spatiotemporal unit in a variable-length time series into a dense vector of fixed dimensions, the model can seamlessly handle arbitrary time steps. The input sequence significantly improves the format compatibility and generalization ability of training samples of different lengths, and further enhances the robustness of the system in dynamic observation scenarios.

[0079] Through the above technical solutions, the angle of arrival and Doppler velocity in the 3D data matrix were discretized and normalized in a targeted manner, effectively solving the problems of excessively wide range of original data, large noise interference, and inconsistent numerical scales. Quantizing the angle of arrival into a discrete interval of 0° to 90° allows the system to focus on the motion information of the target in key areas, significantly reducing data redundancy and computational burden. Simultaneously, normalizing the Doppler velocity and limiting it to the range of [-2, 2] m / s effectively suppresses the influence of outliers, ensuring the stability and consistency of the input data quality. These refined preprocessing steps provide high-quality, highly relevant input for subsequent feature embedding and TCN processors, greatly improving the accuracy and robustness of the deep learning model's motion parameter estimation results in multi-target data association tasks, enabling the model to more effectively learn and utilize the coupling relationship between the angle of arrival and Doppler velocity.

[0080] As one possible implementation, by replacing multichannel convolutions in the TCN processor with complex convolutions, the coupling properties between the angle of arrival and Doppler velocity are maintained by the following method: The temporal characteristics of the angle of arrival and Doppler velocity are encoded into complex input signals, where the angle of arrival corresponds to the real part of the complex number and the Doppler velocity corresponds to the imaginary part of the complex number, forming a complex sequence as a whole.

[0081] In some embodiments described above in this application, a TCN processor is proposed, and complex convolution is used instead of multi-channel convolution to process angle of arrival and Doppler velocity. However, in its implementation, how to effectively input these two intrinsically related physical quantities—angle of arrival and Doppler velocity—into the complex convolutional network and ensure that their coupling properties are maintained during processing is a technical problem that needs to be solved. To address this, this application further proposes that by replacing multi-channel convolution in the TCN processor with complex convolution, the coupling properties between angle of arrival and Doppler velocity are maintained through complex convolution operations.

[0082] This step employs a Temporal Convolutional Network (TCN) as the core feature processing architecture. This network structure utilizes dilated causal convolution, which expands the temporal receptive field while preventing the leakage of future information, making it suitable for the time-series data from the previous step. Simultaneously, AoA and Doppler velocity jointly characterize the target's dynamic behavior in the spatial and velocity dimensions. There is a natural co-evolutionary relationship between the two, and this inherent coupling structure between features contains rich high-order semantic information, which is crucial for improving the model's ability to recognize target behavior patterns.

[0083] Multi-channel convolution for AoA and Doppler velocity is a relatively intuitive operation, and this method can capture the evolution of features over time to some extent. However, since the two branches are independent of each other during the modeling process, they cannot explicitly express the potential physical coupling relationship between AoA and Doppler velocity, leading to information loss or inconsistency in feature fusion. Therefore, this step introduces a complex convolutional neural network to model the correlation between AoA and Doppler velocity in a more natural way.

[0084] Specifically, the temporal features of AoA and Doppler velocity are encoded into complex input signals, where AoA corresponds to the real part of the complex number and Doppler velocity corresponds to the imaginary part. The entire input constitutes a complex sequence. Since complex neural networks inherently possess the ability to model the coupling between the real and imaginary parts, the network can automatically learn and capture the complex nonlinear interaction between these two types of features during forward propagation and gradient optimization.

[0085] For complex convolution operations, see [link to complex convolution operation]. Figure 3 The time-series characteristic pair of AoA and Doppler velocity is encoded into a complex input signal, which has two characteristic spectra: a real characteristic spectrum and a real characteristic spectrum. and Imaginary Number Characteristic Spectrum There are two convolution kernels, a real convolution kernel. and virtual convolution kernel .

[0086] The two feature spectra and two convolution kernels are convolved separately: Right now That is, the real feature spectrum after convolution. Imaginary number characteristic spectrum .

[0087] Complex neural networks inherently possess the ability to model the coupling between real and imaginary parts, enabling them to automatically learn and capture the complex nonlinear interactions between these two types of features during forward propagation and gradient optimization. Compared to traditional real-valued models, complex models not only improve the compactness and consistency of feature representations but also extract cross-dimensional collaborative information more efficiently without introducing additional structural complexity. In specific experiments, this method demonstrates significant performance advantages. Its ability to model complex temporal dynamics is significantly superior to real-valued separation modeling strategies, while exhibiting better robustness and generalization capabilities in perception scenarios with multiple dense targets or frequent occlusion. This strategy not only provides a new feature fusion paradigm but also offers theoretical support and a technical path for constructing perception neural networks with physical consistency constraints.

[0088] The engineering implementation of this step is based on the PyTorch deep learning framework, which constructs an end-to-end neural network model. This end-to-end model takes simulation data as input, and through preprocessing of the input layer, stacks multiple layers of complex dilated causal convolutional modules; finally, it is decoded by a complex fully connected layer to output the estimated associated motion parameters corresponding one-to-one with the target, namely AoA and Doppler velocity.

[0089] Compared to traditional real-valued models, complex models not only improve the compactness and consistency of feature representations but also extract cross-dimensional collaborative information more efficiently without introducing additional structural complexity. In specific experiments, this method demonstrates significant performance advantages. Its ability to model complex temporal dynamics is significantly superior to real-valued-based separation modeling strategies, while exhibiting better robustness and generalization ability in perception scenarios with multiple dense targets or frequent occlusion. This strategy not only provides a new feature fusion paradigm but also offers theoretical support and a technical path for constructing perception neural networks with physical consistency constraints. The engineering implementation in this step is based on the PyTorch deep learning framework, constructing an end-to-end neural network model. This end-to-end model takes simulation data as input, preprocesses the input layer, stacks multiple layers of complex dilated causal convolutional modules, and finally decodes through a complex fully connected layer, outputting the estimated associated motion parameters corresponding one-to-one with the target, namely AoA and Doppler velocity. The entire process eliminates the need for explicit design of association rules or post-processing matching algorithms, achieving a direct mapping from raw simulation data to association results, significantly improving the system's real-time performance and accuracy in highly dynamic, multi-objective scenarios.

[0090] Secondly, embodiments of the present invention provide a multi-objective data association system based on deep learning, comprising: The motion model construction unit constructs multiple motion models representing different motion behaviors for each moving subject. The motion models include at least an acceleration motion model, a steady motion model, a deceleration motion model, and a turning motion model. The global state estimation and global covariance estimation determination unit is used to take the probability of each updated motion model as a weight, and sum the state estimates of each motion model to obtain the global state estimate of the moving subject. At the same time, it sums the covariance estimates of each motion model to obtain the global covariance estimate of the moving subject. The two-dimensional data matrix construction unit is used to construct a two-dimensional data matrix that includes the number of time steps and the number of moving subjects from the global state estimate of each moving subject at each time step. The three-dimensional data matrix conversion unit calculates the angle of arrival and Doppler velocity between the sensor and the moving subject, adding the angle of arrival and Doppler velocity dimensions to the two-dimensional matrix to construct a three-dimensional data matrix; The neural network discretizes and normalizes the angle of arrival and Doppler velocity in the three-dimensional data matrix, respectively, and then transforms the three-dimensional data matrix into a two-dimensional feature vector through tensor reshaping; then feature embedding is performed to map the high-dimensional sparse discretized input to a low-dimensional dense semantic vector space. The TCN processor has temporal modeling capabilities. It replaces the multi-channel convolution in the TCN processor with complex convolution to maintain the coupling properties between the angle of arrival and Doppler velocity. At the same time, it outputs the correlated motion parameter estimation results through an end-to-end learning method.

[0091] The core innovation of this embodiment lies in combining complex convolution with a multi-motion model temporal modeling framework, specifically encoding the angle of arrival and Doppler velocity as the real and imaginary parts of complex signals. This effectively maintains the inherent coupling characteristics of physical quantities in wireless sensing, solves the correlation ambiguity problem caused by independent feature processing in dynamic and complex environments, and improves the accuracy and robustness of multi-target data correlation. Specifically, traditional methods typically treat the angle of arrival and Doppler velocity as independent features, ignoring their physical correlation, such as the synchronicity of angle and velocity changes during target turning, leading to feature redundancy and information loss. The system in this application quantizes the angle of arrival into a discrete range of 0° to 90° through complex convolution operations, while normalizing the Doppler velocity to the range of [-2,2] m / s, and encodes both as a complex sequence for input to the TCN processor. Since complex convolution can process both real and imaginary information simultaneously, it fully preserves the physical coupling between the angle of arrival and Doppler velocity during the convolution operation, avoiding the defect of traditional multi-channel convolution that destroys feature correlation due to independent channel processing. Furthermore, the motion model construction unit constructs acceleration motion models, steady motion models, deceleration motion models, and turning motion models for each moving subject, covering the possible dynamic behavior patterns of the target. The global state estimation and global covariance estimation determination unit uses the updated motion model probabilities as weights to perform a weighted summation of the state estimates and covariance estimates of each motion model, thereby obtaining a more accurate global state estimate. The two-dimensional data matrix construction unit organizes the global state estimates of each moving subject at each time step into a two-dimensional matrix composed of the number of time steps and the number of moving subjects. The three-dimensional data matrix transformation unit then integrates the angle of arrival and Doppler velocity dimensions to form a three-dimensional data structure that comprehensively represents temporal, spatial, and physical characteristics. The neural network preprocesses the three-dimensional data through quantization blocks, normalization blocks, tensor reshaping blocks, and feature embedding blocks. The quantization block performs angle of arrival discretization, the normalization block normalizes the Doppler velocity, the tensor reshaping block converts the data into two-dimensional feature vectors, and the feature embedding block maps the high-dimensional sparse input to a low-dimensional dense semantic space. The TCN processor utilizes the temporal modeling capabilities of complex convolution to extract the temporal dependencies of moving subjects from the semantic space, and finally outputs the associated motion parameter estimation results. Through the above technical solution, the embodiments of the present invention effectively overcome the bottleneck problem of multi-target data association in the field of wireless sensing. Compared with the basic solution, this system not only adapts to the dynamic behavior switching of targets through multiple motion models, avoiding the limitations of a single model, but more importantly, it uses complex convolution to maintain the coupling property of angle of arrival and Doppler velocity, significantly enhancing the feature discrimination capability. In high-density scenes or multipath interference environments, such as dense traffic in urban areas, traditional methods suffer from a sharp performance drop due to their inability to distinguish similar observations, while the system of this application can accurately identify the association features of turning vehicles, thereby outputting reliable motion parameter estimates. As a specific implementation method, when the sensor detects multiple moving subjects, the system first constructs a set of motion models for each subject, dynamically updates the model probabilities, and generates a global state estimate; then, the angle of arrival and Doppler velocity are encoded into complex sequences, preprocessed by a neural network, and input into a TCN processor; finally, the association result is directly output through end-to-end learning, avoiding error accumulation in multi-stage processing, and improving the overall accuracy and stability of data association in complex dynamic environments for wireless sensing systems.

[0092] As one possible implementation, the input layer of the neural network is improved, and the improved input layer includes a quantization block, a normalization block, a tensor reshaping block, and a feature embedding block; Among them, the quantization block is used to quantize the angle of arrival into a discrete range of 0° to 90°; Normalized blocks are used to normalize Doppler velocities, and their values ​​range from [value range missing]. ; Tensor reshaping blocks are used to convert three-dimensional data matrices into two-dimensional feature vectors; Feature embedding blocks are used to map high-dimensional, sparse, discretized inputs to a low-dimensional, dense semantic vector space through embedding.

[0093] The aforementioned scheme proposes a multi-target data association system based on deep learning. This system uses a neural network to discretize, normalize, tensor reshape, and embed features of the angle of arrival and Doppler velocity in the three-dimensional data matrix. However, in practical applications, how to efficiently and structurally process these raw sensor data and transform them into feature representations that can be effectively learned by the neural network is a key issue affecting system performance.

[0094] To address this, this application further proposes an improved input layer for the neural network. The improved input layer includes a quantization block, a normalization block, a tensor reshaping block, and a feature embedding block. Specifically, the quantization block quantizes the angle of arrival into a discrete range of 0° to 90°; the normalization block normalizes the Doppler velocity, with a value range of [-2, 2] m / s; the tensor reshaping block converts the three-dimensional data matrix into a two-dimensional feature vector; and the feature embedding block maps the high-dimensional, sparse, discrete input to a low-dimensional, dense semantic vector space through embedding.

[0095] The input layer of a neural network is the first stage where data enters the model for processing. Improving it aims to optimize the preprocessing flow of raw data, making it more suitable for neural network training and inference, thereby improving the model's learning efficiency, convergence speed, and final performance. This improvement can be achieved through modular design, encapsulating different preprocessing functions into independent sub-modules and chaining them together according to the data flow sequence; or by employing a configurable input layer architecture, allowing for flexible adjustment of the combination and parameters of preprocessing steps based on different data characteristics or task requirements.

[0096] Quantization blocks are used to convert continuous angle-of-arrival (AOA) data into discrete interval values. This discretization helps reduce data dimensionality and noise, simplifies the model's learning task, and may improve sensitivity to specific angle ranges. Quantizing the AOA into discrete intervals of 0° to 90° is optimized based on common angle ranges of target motion in real-world applications to focus on key information. Quantization blocks can use uniform quantization, dividing the 0°–90° range into several discrete intervals, each corresponding to an integer code; or non-uniform quantization, dividing the 0°–90° range into intervals of varying widths based on the distribution characteristics of the AOA data to better preserve important information.

[0097] Standardization blocks are used to normalize Doppler velocities. Normalization eliminates dimensional differences between features, bringing all features within similar numerical ranges. This prevents certain features from dominating the model's learning process due to excessively large values, accelerating model convergence and improving generalization ability. Limiting the Doppler velocity range to [-2, 2] m / s is based on the actual physical constraints of the target's motion velocity and the model's processing requirements. Standardization blocks can use linear normalization (Min-Max Scaling) to scale the data to the specified [-2, 2] interval; or Z-score normalization to convert the data into a distribution with a mean of 0 and a standard deviation of 1, then mapping it to the [-2, 2] interval through a linear transformation.

[0098] Tensor reshaping blocks are used to convert three-dimensional data matrices into two-dimensional feature vectors. In deep learning, different types of neural network layers have specific requirements for the dimensionality of the input data. Reshaping three-dimensional data into two-dimensional vectors is usually to adapt to the input format of fully connected layers or certain specific convolutional layers, while flattening multi-dimensional information into a one-dimensional sequence to facilitate subsequent feature extraction. Tensor reshaping blocks can flatten each dimension of a three-dimensional matrix row by row or column by column, sequentially unfolding and concatenating them into a long vector; more complex reshaping strategies can also be used, such as merging some dimensions first and then flattening, to preserve specific local structural information.

[0099] Feature embedding blocks are used to map high-dimensional, sparse, discretized inputs to a low-dimensional, dense semantic vector space through embedding. Discretized data is typically high-dimensional and sparse (e.g., one-hot encoded), and directly inputting it into a neural network can lead to the curse of dimensionality and computational inefficiency. Feature embedding can learn low-dimensional continuous representations of these discrete features, making semantically similar features closer together in the vector space, thereby improving the model's learning efficiency and expressive power. Feature embedding blocks can use an embedding lookup table to assign a learnable low-dimensional vector to each discrete value; alternatively, they can use a small, fully connected network to map the one-hot encoded high-dimensional sparse vector to a low-dimensional, dense vector space.

[0100] The system provided in this invention, particularly its neural network component, is designed to process sensor data to achieve multi-target data correlation. After the angle of arrival and Doppler velocity between the sensor and the moving subject are calculated and combined with the global state estimate of the moving subject to construct a three-dimensional data matrix, this three-dimensional data matrix first enters the improved neural network input layer. In this input layer, a quantization block first quantizes the angle of arrival in the three-dimensional data matrix, converting it into a discrete range of 0° to 90°. This helps to focus on key angle information of the target motion and reduces the complexity of subsequent processing. Next, a normalization block normalizes the Doppler velocity, limiting its value range to [-2, 2] m / s, thereby eliminating the influence of different physical dimensions and ensuring that the Doppler velocity is processed fairly and effectively in the neural network. After the initial preprocessing of the angle of arrival and Doppler velocity, a tensor reshaping block converts the processed three-dimensional data matrix into a two-dimensional feature vector. This conversion step is necessary because it flattens the multi-dimensional sensor data into a linear structure that is easier for the neural network to process, laying the foundation for subsequent feature extraction and learning. Finally, the feature embedding block receives these two-dimensional feature vectors and maps them from a high-dimensional, sparse, discretized input space to a low-dimensional, dense semantic vector space. Through this embedding, the neural network can learn more expressive feature representations, bringing semantically related inputs closer together in the vector space, thus significantly improving the efficiency and accuracy of the TCN processor in temporal modeling and association learning. This modular and refined input layer design ensures that the raw sensor data undergoes sufficient and optimized preprocessing before entering the TCN processor, providing high-quality input for subsequent complex convolutions and end-to-end learning, ultimately improving the accuracy and robustness of multi-object data association.

[0101] Through the above technical solutions, the system of this application can perform refined and structured preprocessing of raw sensor data. The introduction of quantization blocks and normalization blocks effectively unifies the two different physical quantities, angle of arrival and Doppler velocity, into the range that the neural network can process, avoiding model training difficulties and performance degradation caused by differences in data dimensions or excessive noise in the raw data. The synergistic effect of tensor reshaping blocks and feature embedding blocks further efficiently transforms the preprocessed multidimensional data into low-dimensional and semantically rich feature representations, greatly reducing the computational burden on the subsequent TCN processor and improving its ability to capture target motion patterns and correlations from complex time-series data. This not only improves the accuracy and real-time performance of multi-target data correlation but also enhances the system's adaptability and robustness to different sensor data inputs.

[0102] Thirdly, embodiments of the present invention provide a trajectory tracking method, comprising the following steps: Based on the physical relationship between Doppler velocity and carrier frequency, an elliptic constraint model with the transmitter and receiver as foci is constructed. A ray parameter model is constructed with the receiver position as the origin; By combining the elliptical constraint model and the ray parameter model, and integrating the correlated motion parameter estimation results from the previous time step, the positioning result at this time step can be obtained. By connecting the location results from multiple moments in chronological order, the tracking trajectory can be obtained.

[0103] Specifically, this method combines an elliptical constraint model and a ray parameter model in a joint solution approach, utilizing the physical relationship between Doppler velocity and angle of arrival to constrain the localization solution space, thereby reducing localization ambiguity in multi-object scenarios. Specifically, the elliptical constraint model, based on the Doppler effect, restricts the possible positions of the moving subject to an elliptical curve with the transmitter and receiver as foci, stemming from the physical correlation between Doppler frequency shift and relative velocity. Simultaneously, the ray parameter model uses the angle of arrival measurement to define a direction vector originating from the receiver, forming a ray; the intersection of these two models represents the localization solution at the current moment, effectively narrowing the search space. By combining the correlated motion parameter estimation results from the previous moment, this method can smoothly handle motion continuity, avoiding trajectory jumps or breaks.

[0104] Through the above technical solution, this application significantly improves the accuracy and robustness of trajectory tracking. Compared with traditional methods that rely on only a single feature, this solution fully utilizes the coupled physical characteristics of Doppler velocity and angle of arrival, effectively addressing multipath and occlusion problems in wireless sensing, and providing reliable technical support for multi-target tracking. In practical applications, this method can reduce trajectory crossing errors and ensure accurate motion trajectory generation even in high-density scenarios, thereby solving the key bottlenecks of low trajectory recovery accuracy and poor robustness mentioned in the background technology.

[0105] As one possible implementation, a ray parameter model is constructed with the receiver location as the origin, including: A direction vector is established based on the measured angle of arrival; A ray parameter model is constructed based on the receiver's position and direction vector, specifically as follows: ;in, The coordinates of the receiver's location. It is a direction vector. , This is the measured value of the angle of arrival.

[0106] The purpose of establishing a direction vector based on the measured angle of arrival is to transform the angle between the signal propagation direction and the sensor reference direction (i.e., the angle of arrival) when the sensor receives the signal into a mathematical representation with direction and magnitude. This allows for geometric calculations in a coordinate system, directly indicating the target's position and orientation relative to the receiver. This direction vector can be generated as a unit direction vector in two-dimensional or three-dimensional space by directly substituting the measured angle of arrival into trigonometric functions (such as sine and cosine).

[0107] Specifically, this method establishes a mathematical-physical model and achieves high-precision and robust trajectory tracking based on prior position information and the correlation results of motion characteristics. Figure 1 The method for establishing mathematical physics models is as follows: This step uses the correlation results output from the preceding steps (see...). Figure 4 The input consists of the correlated AoA and Doppler velocity. This multi-target trajectory tracking step is based on a rigorous geometric constraint modeling and optimization process. First, Doppler geometric constraint modeling is performed, based on the Doppler velocity... With carrier frequency The physical relationship, constructing a system based on the transmitting end and receiving end Elliptical constraint model with focus: The center of the ellipse , , , .

[0108] Subsequently, AoA ray constraint modeling was performed, based on AoA measurements obtained from array signal processing. Establish direction vectors: Based on the location of the receiving end Using the origin as the reference point, construct the ray parametric equations: Where R is the coordinate of the receiver's location. It is a direction vector. This is the measured value of the angle of arrival.

[0109] The ray parameter model is a mathematical model that describes a straight line extending in a specific direction from the receiving end. This model uses the position R of the receiving end and the direction vector determined by the angle of arrival. Let's define a ray, where k is a positive scalar parameter representing the distance between the target and the receiver along that direction. The purpose of this model is to combine the received angle of arrival information with the receiver's own spatial position to determine the possible trajectory of the target in space. The receiver position R can be a fixed, known coordinate point, such as the sensor's installation location. Direction vector. The ray parameter is calculated from the measured angle of arrival (θ). By substituting these parameters into the ray parameter equation, a ray originating from the receiver (R) and pointing towards the target can be obtained. In another implementation, the receiver position R can also be dynamically changing, such as a sensor mounted on a mobile platform. In this case, R needs to be acquired in real time, for example, via GPS or other positioning systems. This model can flexibly adapt to changes in the receiver position, ensuring the accuracy of the ray parameter model.

[0110] By simultaneously solving the elliptic constraint equations and the ray constraint equations, and combining this with the prior position information from the previous moment, the positioning result for the current moment can be obtained. Once the positioning results for each moment are obtained, connecting these positioning points in chronological order yields the sensor's positioning and tracking results. Through this continuous positioning process, the sensor's trajectory over a period of time can be effectively tracked.

[0111] The method of this invention establishes a direction vector based on the measured angle of arrival and combines this direction vector with the coordinates R of the receiver position. Using the mathematical expression of the ray parameter equation, a ray originating from the receiver R and pointing towards the target is explicitly defined. This construction method allows the ray parameter model to accurately reflect the target's direction information relative to the receiver, thus providing accurate geometric constraints for subsequent joint solutions with the elliptic constraint model. In this way, the abstract step of "constructing a ray parameter model with the receiver position as the origin" is concretized into an operable mathematical model, ensuring the geometric accuracy and reliability of trajectory tracking. The combination of this ray parameter model and the elliptic constraint model effectively confines the target position to a specific intersection point in space, thereby improving the accuracy of positioning.

[0112] The above technical solution concretizes and refines the steps of constructing a ray parameter model in trajectory tracking methods. It establishes a direction vector based on the measured angle of arrival and combines this with the coordinates of the receiver's position using mathematical formulas. The ray parameter model is constructed in a specific form, enabling accurate quantification and representation of the target's orientation information. This effectively solves the uncertainty in accurately building a ray parameter model based on measurement data to precisely reflect the target's orientation information during trajectory tracking. This precisely constructed ray parameter model, when solved in conjunction with an elliptical constraint model, provides more accurate geometric constraints, thereby significantly improving the accuracy of the positioning results and the reliability of trajectory tracking.

[0113] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, the disclosure, and the description of the drawings, in carrying out the claimed invention. In this specification, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple components. A single processor or other unit can implement several of the functions listed in the specification. While certain measures are described in different embodiments, this does not mean that these measures cannot be combined to produce good results.

[0114] Although the invention has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made therein without departing from the spirit and scope of the invention. Accordingly, this specification and drawings are merely illustrative of the invention and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if such modifications and modifications fall within the scope of the invention and its equivalents, the invention is also intended to include such modifications and modifications.

Claims

1. A multi-objective data association method based on deep learning, characterized in that, Includes the following steps: Construct multiple motion models that can represent different motion behaviors. The motion models include at least an acceleration motion model, a steady motion model, a deceleration motion model, and a turning motion model. The updated motion model probability is used as a weight to sum the state estimates of each motion model to obtain the global state estimate of the moving subject. At the same time, the covariance estimates of each motion model are summed in a weighted manner to obtain the global covariance estimate of the moving subject. The global state estimates of each moving subject at each time step are constructed into a two-dimensional data matrix including the number of time steps and the number of moving subjects; Calculate the angle of arrival and Doppler velocity between the sensor and the moving subject, and add the angle of arrival and Doppler velocity dimensions to the two-dimensional matrix to construct a three-dimensional data matrix; The angle of arrival in the 3D data matrix is ​​discretized, the Doppler velocity in the 3D data matrix is ​​normalized, and then the 3D data matrix is ​​transformed into a 2D feature vector through tensor reshaping; then feature embedding is performed to map the high-dimensional sparse discretized input to a low-dimensional dense semantic vector space. A TCN processor with temporal modeling capabilities is employed, and the multi-channel convolution in the TCN processor is replaced with complex convolution to maintain the coupling property between the angle of arrival and Doppler velocity. At the same time, the associated motion parameter estimation results are output through an end-to-end learning method.

2. The multi-objective data association method based on deep learning according to claim 1, characterized in that, For each motion model of each moving subject, the updated motion model probability is obtained in the following way: With each other motion model in The probability of each time step is used as a weight, and the transition probabilities from each other motion model to the motion model to be updated are weighted and summed to obtain the prediction probability of the motion model to be updated. The transition probability from any other motion model to the motion model to be updated is related to the motion model in... The product of the probabilities at each time step is divided by the predicted probability to obtain the mixed transition probability from any other motion model to the motion model to be updated. Using the mixed transition probability from any other motion model to the motion model to be updated as the weight, the state estimate and covariance estimate of the motion model to be updated at the previous time step are weighted and summed to obtain the mixed state estimate and mixed covariance estimate of the motion model to be updated. Kalman filtering is applied to both the mixed state estimate and the mixed covariance estimate to obtain the updated state estimate and the updated covariance estimate, respectively. Construct and solve the likelihood function of the motion model to be updated; The product of the likelihood function and the predicted probability is divided by the normalization constant to obtain the probability of the motion model to be updated.

3. The multi-objective data association method based on deep learning according to claim 1, characterized in that, Discretize the angle of arrival in the three-dimensional data matrix, specifically by quantizing the angle of arrival into a discrete range of 0° to 90°; Normalization is performed on the Doppler velocities in the three-dimensional data matrix, specifically including: normalizing the Doppler velocities, with values ​​ranging from [value range missing]. .

4. The multi-objective data association method based on deep learning according to claim 1, characterized in that, After replacing multichannel convolutions with complex convolutions in the TCN processor, the coupling properties between the angle of arrival and Doppler velocity are preserved by the following method: The temporal characteristics of the angle of arrival and Doppler velocity are encoded into complex input signals, where the angle of arrival corresponds to the real part of the complex number and the Doppler velocity corresponds to the imaginary part of the complex number, forming a complex sequence as a whole.

5. A multi-objective data association system based on deep learning, characterized in that, include: The motion model construction unit constructs multiple motion models representing different motion behaviors for each moving subject. The motion models include at least an acceleration motion model, a steady motion model, a deceleration motion model, and a turning motion model. The global state estimation and global covariance estimation determination unit is used to take the probability of each updated motion model as a weight, and sum the state estimates of each motion model to obtain the global state estimate of the moving subject. At the same time, it sums the covariance estimates of each motion model to obtain the global covariance estimate of the moving subject. The two-dimensional data matrix construction unit is used to construct a two-dimensional data matrix that includes the number of time steps and the number of moving subjects from the global state estimate of each moving subject at each time step. The three-dimensional data matrix conversion unit calculates the angle of arrival and Doppler velocity between the sensor and the moving subject, adding the angle of arrival and Doppler velocity dimensions to the two-dimensional matrix to construct a three-dimensional data matrix; The neural network discretizes and normalizes the angle of arrival and Doppler velocity in the three-dimensional data matrix, respectively, and then transforms the three-dimensional data matrix into a two-dimensional feature vector through tensor reshaping; then feature embedding is performed to map the high-dimensional sparse discretized input to a low-dimensional dense semantic vector space. The TCN processor has temporal modeling capabilities. It replaces the multi-channel convolution in the TCN processor with complex convolution to maintain the coupling properties between the angle of arrival and Doppler velocity. At the same time, it outputs the correlated motion parameter estimation results through an end-to-end learning method.

6. The multi-objective data association system based on deep learning according to claim 5, characterized in that, An improved input layer for neural networks is proposed, comprising a quantization block, a normalization block, a tensor reshaping block, and a feature embedding block. Among them, the quantization block is used to quantize the angle of arrival into a discrete range of 0° to 90°; Normalized blocks are used to normalize Doppler velocities, and their values ​​range from [value range missing]. ; Tensor reshaping blocks are used to convert three-dimensional data matrices into two-dimensional feature vectors; Feature embedding blocks are used to map high-dimensional, sparse, discretized inputs to a low-dimensional, dense semantic vector space through embedding.

7. A trajectory tracking method, characterized in that, Includes the following steps: Based on the physical relationship between Doppler velocity and carrier frequency, an elliptic constraint model with the transmitter and receiver as foci is constructed. A ray parameter model is constructed with the receiver position as the origin; By combining the elliptical constraint model and the ray parameter model, and integrating the correlated motion parameter estimation results from the previous time step, the positioning result at this time step can be obtained. By connecting the location results from multiple moments in chronological order, the tracking trajectory can be obtained.

8. The trajectory tracking method according to claim 7, characterized in that, A ray parameter model is constructed with the receiver location as the origin, including: A direction vector is established based on the measured angle of arrival; A ray parameter model is constructed based on the receiver's position and direction vector, specifically as follows: ;in, The coordinates of the receiver's location. It is a direction vector. , This is the measured value of the angle of arrival.

Images