Data association method and system based on cross Transform network architecture

Abstract

The invention provides a data association method and system based on a cross Transform network architecture, and belongs to the technical field of artificial intelligence. The problem that the accuracy of a data association result is reduced due to the fact that existing multi-target tracking has false alarm and missing detection conditions on a network framework is solved. According to the method, a relative distance feature matrix is defined, and the matrix allows data standardization by adopting a data filling and partitioning technology; then constructing a data association network based on a cross Transform, wherein the network effectively extracts features from a distance matrix corresponding to the continuous time instance; in addition, an association result obtained from the CTDA network is refined through a data post-processing algorithm, thereby ensuring accurate target tracking.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a data association method and system based on a cross Transformer network architecture. Background Art

[0002] Multi-target tracking (MTT) or multiple object tracking (MOT) plays a vital role in various fields such as surveillance, autonomous driving, and robotics. The MTT problem refers to associating relative measurements obtained by sensors over time to multiple moving objects (targets) and estimating their trajectories and other properties (velocity, acceleration, etc.). A key issue in MTT is to resolve the unknown correspondences between targets and measurements, a task known as data association. The MTT task is highly challenging in scenarios with occlusions, sensor noise, clutter, and targets that may enter or leave the scene.

[0003] In order to address these challenges, many algorithms and methods for multi-target tracking have been proposed, which can generally be divided into maximum likelihood data association methods and Bayesian data association methods. The maximum likelihood data association method is based on the likelihood ratio of the observed data for association. Classical algorithms include joint maximum likelihood algorithm, integer programming, generalized association method, etc. The maximum likelihood data association method is not suitable for situations with intersecting targets, maneuvering targets and dense multi-target scenes, which limits its application in practice. Bayesian data association methods are based on the Bayesian criterion and mainly include the nearest neighbor (NN) algorithm, probabilistic data association (PDA) algorithm, joint probabilistic data association (JPDA) algorithm, and multiple hypothesis tracking (MHT) method. These methods, such as JPDA, need to assume that the number of targets is constant. The computational complexity of the MHT method increases rapidly with the increase in the number of targets. Another type of MTT method is based on random finite sets (RFS), which uses multi-object conjugate priors to derive closed expressions for multiple object posteriors and obtain Bayesian optimal estimates. Typical algorithms include probability hypothesis density (PHD) filters, potential probability hypothesis density (CPHD) filters, and generalized label multi-Bernoulli (GLMB) trackers. RFS-based methods are able to transfer more information between different time steps, thereby improving tracking results. However, these methods usually rely on prior information about the true posterior density. The approximation of the true posterior density may lead to inaccurate data association. In addition, due to the use of high-dimensional sampling processes, the computational and storage requirements of these methods are increased. Summary of the invention

[0004] The technical problems to be solved by the present invention are:

[0005] In order to solve the problem that the existing multi-target tracking relies on the prior information of the true posterior density in the network architecture, resulting in inaccurate data association and high computing and storage requirements.

[0006] The present invention adopts the following technical solutions to solve the above technical problems:

[0007] The present invention provides a data association method based on a cross Transformer network architecture, comprising the following steps:

[0008] S100, defining a relative distance feature matrix about the x, y and z directions to ensure the generalization of normalized features of data at different positions;

[0009] S200, data preprocessing, introducing a block method and a filling method, filling the data in the to-be-associated data set D1 and the data set D2 with additional data of appropriate dimensions and performing block processing, and taking the enhanced matrix containing relative distance features as the input of the network;

[0010] S300, constructing a data association network based on a cross Transformer, including constructing a Transformer encoder and a Transformer decoder, and then using a row-column cross Transformer network and a 3-channel cross Transformer network to obtain an output matrix;

[0011] S400, each element of the matrix obtained from the cross-Transformer-based data association network in step S300 represents the association probability of a given input data pair, and the Kuhn-Munkres algorithm is combined to find the best allocation, and then the network output is converted into an association matrix.

[0012] Furthermore, in step S100, it includes:

[0013] set up and D1 and D2 represent two sets of position data respectively;

[0014] Define m×n matrices of relative distances in the x, y, and z directions, namely, relative distance feature matrices, respectively, to characterize the association relationship characteristics between data set D1 and data set D2;

[0015] Among them, for the x-axis direction,

[0016] set up The relative distance feature matrix associated with the x direction is as follows,

[0017]

[0018] in, and are the x-direction data from dataset D1 and dataset D2 respectively; bias is a small amount used to avoid data errors caused by division by zero; x mid is the geometric median of the absolute values ​​of all relative distances between two data sets,

[0019]

[0020] For the relative distance feature matrices in the y-axis direction and the z-axis direction, they are expressed as and

[0021] Furthermore, in step S200, it includes:

[0022] Let N pad is a fixed value, and N pad Less than or equal to m and n; let and are integers for the rows and columns of the scaled padding matrix, is the round-up operation;

[0023] make is the fill matrix associated with x after partitioning,

[0024]

[0025] Division as follows,

[0026]

[0027] in, and are the number of partitions for rows and columns respectively; each partition submatrix is an N pad ×N pad matrix;

[0028] Using the partitioning defined in Equation (4), the scaled padding matrix is ​​reshaped into a batch matrix as follows,

[0029]

[0030] The padding matrices associated with directions y and z are defined in the same way as for direction x; subsequently, the augmented matrix containing the relative distance features is used as the network input.

[0031] Furthermore, in step S300, it includes:

[0032] S310, build the Transformer encoder, including three integrated modules, namely multi-head attention, feedforward network and normalization network;

[0033] The self-attention neural network uses Query to retrieve the attention values ​​corresponding to Keys and assign them to the corresponding Values; then redistributes Value according to the attention weights; let Attention represent the attention processing,

[0034]

[0035] in, and Represent Query and Key matrices respectively; represents the y matrix value, d is the dimension of Q, K and V in the self-attention neural network, Q, K and V represent the same feature encoding;

[0036] Using the matrix W Q , W K and W V Combined with a multi-head attention network, different types of relationships can be extracted from the data.

[0037] MultiHead(Q,K,V)=Concat(head1,…,head n )W O (7)

[0038] in, is each attention network head, and W O It is a trainable parameter, which is equivalent to the Q, K and V tensors having the same shape;

[0039] S320. Build a Transformer decoder, including two multi-head attention networks, set a mask layer in the first multi-head attention network, and use Q, K, and V in the second multi-head attention network.

[0040] The decoder input is used as the query feature and the encoder input is used as the key and value. The resulting weighted input encoding is combined with the decoder input and passed through normalization and a feed-forward neural network to produce the decoder output.

[0041] S330, combining the Transformer encoder of step S310 and the Transformer decoder of step S320 to construct a cross-Transformer-based data association network, including:

[0042] S331. Construct a row-column crossover Transformer network, which is applied to the three channels of the input matrix; each channel is processed separately by a row Transformer encoder and a column Transformer encoder, which treat the rows and columns of the input as feature vectors respectively; the output of the encoder is input to the row and column Transformer decoders in a crossover mode;

[0043] Introducing the element-by-element method,

[0044]

[0045] Where he is the output of the mixing matrix, Reshape(·) means converting the dimension of the matrix to a specified number, [·] represents the concatenation operation, W and b are trainable weights and biases, and h1 and h2 are decoders;

[0046] S332, construct a 3-channel cross Transformer network, the output of the element-by-element mixing processing module consisting of 3-channel data is guided to three Transformer encoders to obtain the encoding features; then, the output of each encoder corresponding to each channel is simultaneously fed to the first MHA module of the decoder of other channels; each output is simultaneously used as the key and value input of the second MHA module in the corresponding decoder of the channel; then the output matrix is ​​obtained by element-by-element mixing,

[0047]

[0048] Among them, O1, O2 and O3 are the outputs of the decoders corresponding to the three channels.

[0049] Furthermore, in step S300, it also includes using binary cross entropy loss as a cost to calculate the relative association probability of the network output,

[0050]

[0051] in, and The incidence matrix O pr and the true incidence matrix O gt Predict the element at row i and column j.

[0052] Furthermore, it can be used for multi-target tracking or multi-target tracing of monitoring, autonomous driving and robots to associate measurement values ​​with multiple targets, or to resolve unknown correspondences between targets and measurement values.

[0053] A data association system based on a cross Transformer network architecture, the system has a program module corresponding to the above steps, and executes the steps in the above data association method based on the cross Transformer network architecture when running.

[0054] A computer-readable storage medium stores a computer program, wherein the computer program is configured to implement the steps of a data association method based on a cross Transformer network architecture when called by a processor.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] The present invention discloses a data association method and system based on a cross Transformer network architecture, which are used to process multiple targets in the presence of false positives and missed detections. In order to solve the challenge of different data quantities and sizes in different application tasks, a relative distance feature matrix is ​​defined, which allows data standardization by adopting data filling and blocking techniques. Then, a Cross Transformer-based data association network is constructed, which effectively extracts features from the distance matrix corresponding to continuous time instances. In addition, a data post-processing algorithm is used to refine the association results obtained from the CTDA network, thereby ensuring accurate target tracking.

[0057] The data filling technology of the present invention has a clearer physical meaning, because it is equivalent to generating multiple false alarms at a place infinitely far away from the original data; therefore, in the entire filling and partitioning process, no additional information is introduced into the RDF matrix, ensuring that the basic characteristics of the matrix after the filling process are not affected. Compared with the existing row-column Transformer module, the cross Transformer module constructed by the present invention is easy to operate, and the number of Transformers in the cross structure applied between each pair of channels will change with the dimension of the data, and the number of encoders and decoders can also be different, so the scope of application is wider. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of a data association method based on a cross Transformer network architecture in an embodiment of the present invention;

[0059] Figure 2 Flow chart of data filling and blocking in the data preprocessing process in an embodiment of the present invention;

[0060] Figure 3 : is a diagram of a Transformer-based encoder network structure in an embodiment of the present invention;

[0061] Figure 4: is a decoder network structure diagram based on Transformer in an embodiment of the present invention;

[0062] Figure 5 This is a data association network diagram based on a cross Transformer in an embodiment of the present invention;

[0063] Figure 6 A diagram of a processing method of the Kuhn-Munkres (KM) algorithm in an embodiment of the present invention;

[0064] Figure 7 Graphs showing test results of the algorithm in an embodiment of the present invention, wherein (a), (b), (c) and (d) correspond to test result graphs of four situations respectively.

[0065] Description of reference numerals: DETAILED DESCRIPTION

[0066] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0067] The data association process aims to solve the problem of determining whether a set of data (which can be measurement data or estimated data) originates from the same target. In the MTT process, new target detection requires "measurement-measurement" data association across multiple sampling periods. In order to update the track and maintain the tracking estimate, "measurement-track" association is required to identify new measurement data for track correction. The correlation between these associations, including "measurement-measurement" and "measurement-track" data associations, is described by the association matrix. Given two sets of data: and To be associated, the association matrix of the two sets of data is expressed as,

[0069]

[0070] in,

[0071]

[0072] It should be noted that each column and each row of the incidence matrix is ​​allowed to have at most one non-zero element, i.e. 1, while all other elements are 0. The purpose of this algorithm is to estimate the state of a different number of targets by using position measurements based on noisy radar.

[0073] Specific implementation plan 1: Combine Figures 1 to 5 As shown, the present invention provides a data association method based on a cross Transformer network architecture, comprising the following steps:

[0074] S100, define a relative distance feature (RelativeDistanceFeature, RDF) matrix to ensure the generalization of normalized features of data at different locations, including:

[0075] set up and D1 and D2 represent two sets of position data, which can be measurements, estimates, and predictions from real targets or false alarms;

[0076] In order to characterize the association relationship between data sets D1 and D2, an m×n matrix of relative distances in the x, y, and z directions is defined respectively, which is called the Relative Distance Feature (RDF) matrix.

[0077] Among them, for the x-axis direction,

[0078] set up The RDF matrix associated with the x direction is as follows,

[0079]

[0080] in, and are the x-direction data from dataset D1 and dataset D2 respectively; bias is a small quantity of appropriate size to avoid data errors caused by division by zero; x mid is the geometric median of the absolute values ​​of all relative distances between two data sets,

[0081]

[0082] The RDF matrices for the y-axis and z-axis directions are expressed as and Can be constructed in the same way;

[0083] In the RDF matrix, a value close to 1 indicates that the corresponding data points in datasets D1 and D2 are similar and are likely to be related to each other; conversely, a value close to 0 indicates that the data points are significantly different and do not belong to the same target;

[0084] S200, data preprocessing, introducing a block method and a filling method, filling the data in the to-be-associated data set D1 and the data set D2 with additional data of appropriate dimensions and performing block processing, including:

[0085] In many tasks, the number and dimensionality of measurements taken vary significantly; furthermore, even within a single task, the number of measurements taken at different times may vary;

[0086] In the case of tracking a limited number of targets, an appropriate maximum padding value can be pre-allocated; however, when faced with a large number of targets and clutter, an excessively large padding value becomes computationally infeasible; in order to make the data association algorithm proposed in the present invention applicable to data sets of different sizes, it is necessary to introduce a better data partitioning technology.

[0087] Let N pad is a constant value, and N pad Less than or equal to m and n; let and are integers for the rows and columns of the scaled padding matrix, is the round-up operation;

[0088] make is the fill matrix associated with x after partitioning,

[0089]

[0090] In order to adapt to the input dimension of the CTDA network, it is further divided as follows,

[0091]

[0092] in, and are the number of partitions for rows and columns respectively; each partition submatrix is an N pad ×N pad matrix;

[0093] Using the partitioning defined in Equation (6), the scaled padding matrix will be reshaped into the batch matrix as follows,

[0094]

[0095] The padding matrices associated with directions y and z can be defined in the same way as direction x; subsequently, the augmented matrix containing the relative distance features is used as the input of the CTDA network;

[0096] Compared with the prior art, the data filling technology of the present invention has a clearer physical meaning, because it is equivalent to generating multiple false alarms at a place infinitely far away from the original data; therefore, in the entire filling and partitioning process, no additional information is introduced into the RDF matrix; it is ensured that the basic characteristics of the matrix after the filling process are not affected;

[0097] S300, build a data association network based on cross Transformer, including,

[0098] S310, build the Transformer encoder, including three integrated modules, namely Multi-Head Attention (MHA), Feed Forward Network (FFN) and Normalization Network, such as Figure 3 As shown, the most important one is the multi-head attention, i.e. the attention module, which is the cornerstone of the MHA backbone;

[0099] The main goal of the self-attention neural network is to use Query to retrieve the attention values ​​corresponding to Keys and assign them to the corresponding Values; then redistribute the Value according to the attention weight; let Attention represent the attention processing, which is as follows,

[0100]

[0101] in, and Represent Query and Key matrices respectively; represents the y matrix value, d is the dimension of Q, K and V in the self-attention neural network, Q, K and V represent the same feature encoding;

[0102] Furthermore, by using the matrix W Q , W K and W V Combined with the multi-head attention network, different types of relationships can be extracted from the data.

[0103] MultiHead(Q,K,V)=Concat(head1,…,head n )W O (9)

[0104] in, is each attention network head, and W O It is a trainable parameter, which is equivalent to the Q, K and V tensors having the same shape;

[0105] S320, constructing a Transformer decoder, including two multi-head attention networks (MHA), setting a mask layer in the first multi-head attention network, wherein the mask layer can be configured as a matrix with all elements equal to 1, so as to be the same as the standard multi-head attention network; using Q, K and V in the second multi-head attention network;

[0106] Specifically, the decoder input is used as the query feature, while the encoder input is used as the key and value; this arrangement enables the decoder to retrieve information from the encoder output using the encoded features obtained through multi-head attention as a query; subsequently, the resulting weighted input encoding is combined with the decoder input and normalized and fed-forwarded to produce the decoder output;

[0107] S330, combining the Transformer encoder of step S310 and the Transformer decoder of step S320 to construct a cross-Transformer-based data association network (CTDA), including:

[0108] S331. Construct a row-column crossover Transformer network, which is applied to the three channels of the input matrix. Each channel is processed separately by a row Transformer encoder and a column Transformer encoder, which treat the rows and columns of the input as feature vectors respectively. It should be noted that the input of each channel is processed independently. Then, the output of the encoder is input to the row and column Transformer decoders in a crossover mode.

[0109] Since the Transformer decoder includes two MHA modules, in the context of the row-column cross Transformer network, the outputs of the row and column encoders serve as the Key and Value inputs of the second MHA module in the row and column Transformer decoders, respectively; at the same time, the output of each encoder is cross-fed into another Transformer decoder as the input of the first MHA module layer; the outputs of the row and column decoders (each channel consists of two matrices) are combined using a processing module; in order to meet different requirements, the element-by-element method is introduced to improve the efficiency of this algorithm.

[0110]

[0111] Among them, h e is the output of the mixing matrix, Reshape(·) means converting the dimension of the matrix to a specified number, [·] means the concatenation operation, W and b are trainable weights and biases, and h1 and h2 are decoders;

[0112] S332, construct a 3-channel cross Transformer network, and implement a cross Transformer network structure for the 3-channel encoded data obtained by bidirectional processing; the output of the element-by-element mixing processing module consisting of the 3-channel data is initially directed to three Transformer encoders to obtain the encoded features; then, the output of each encoder corresponding to each channel is simultaneously fed to the first MHA module of the decoder of other channels; in addition, each output is simultaneously used as the key and value input of the second MHA module in the corresponding decoder of the channel; then the output matrix is ​​obtained by element-by-element mixing,

[0113]

[0114] Among them, O1, O2 and O3 are the outputs of the decoders corresponding to the three channels;

[0115] The cross transformer used in the 3-channel cross transformer network is different from the corresponding cross transformer in the row-column cross transformer; first, the 3-channel cross transformer network contains a reduced number of layers compared to the row-column transformer module. This adjustment is made by observing that the information fusion between different channels is often relatively less complex than the operations required by the row-column cross transformer module. Using a deeper network in this particular case may lead to overfitting; second, the number of transformers in the cross structure applied between each pair of channels varies with the dimension of the data, rather than being fixed; in addition, the number of encoders and decoders can be different;

[0116] S340. Since the data association matrix exhibits sparsity and is composed of binary values ​​(0-1 matrix), the binary cross entropy loss is used as the cost to calculate the relative association probability of the network output.

[0117]

[0118] in, and The incidence matrix O pr and the true incidence matrix O gt The predicted element in row i and column j;

[0119] S400, each element of the matrix obtained from the cross-Transformer-based data association network represents the association probability corresponding to the given input data; in order to further convert the network output into an association matrix, the Kuhn-Munkres (KM) algorithm, also known as the Hungarian matching algorithm, is combined to find the best allocation. The specific steps are combined Figure 6 As shown;

[0120] It is worth noting that the KM algorithm is specifically designed to solve complex situations and is very useful when dealing with more challenging scenarios; in most cases, it is sufficient to directly threshold the network output to generate the association matrix.

[0121] Specific implementation scheme 2: The present invention provides a data association system based on a cross Transformer network architecture, which has a program module corresponding to the above steps and executes the steps in the above data association method based on the cross Transformer network architecture during operation.

[0122] The other combinations and connection relationships of this embodiment are the same as those of the first embodiment.

[0123] Specific implementation scheme three: The present invention provides a computer-readable storage medium, which stores a computer program, and the computer program is configured to implement the steps of a data association method based on a cross Transformer network architecture when called by a processor.

[0124] The other combinations and connection relationships of this embodiment are the same as those of the first embodiment.

[0125] Simulation experiment

[0126] A custom 3D dataset was utilized to evaluate the association accuracy of the CTDA algorithm.

[0127] Evaluation Metrics,When evaluating the association between two data sets, accuracy is usually used as an,evaluation metric.,However, since there is not always a one-to-one correspondence between the two,sets of data taking into account false alarms and missed detections,,a more comprehensive accuracy evaluation metric is adopted, as shown below:

[0128]

[0129] Among them, a ij and The correlation matrix A is m×n and the true correlation matrix The elements in row i and column j of ;

[0130]

[0131] Setting up the dataset and training, the algorithm is able to associate 2D or 3D trajectory data; the advantage is that no prior quantitative information is required, such as false alarm probability and missed detection probability. In order to train the network in the algorithm, a large amount of data is required to enhance the generalization ability of the network and prevent overfitting. However, obtaining real trajectory data can be challenging, and the available data is often limited. To overcome these limitations, we developed a comprehensive multi-objective simulation platform to generate a large amount of simulated data of trajectory measurements for network training.

[0132] First, build a 3D dynamic model and a measurement model to generate training data. Select the state vector as x = [x, y, z, v, ζ, ψ] T and measure z = [r, θ, φ],

[0133]

[0134] Where x, y and z represent the position of the target; v, ζ and ψ represent the velocity, pitch angle and yaw angle respectively; θ and φ represent the azimuth and elevation angle respectively; u D 、u L and u G are the control of speed, pitch angle and yaw angle respectively; η is the process noise and measurement noise vector.

[0135] The parameters of the customized multi-target dataset are shown in Table 1.

[0136] Table 1

[0137]

[0138] By using these parameters, a total of 2000 trajectories were generated. Among them, 1800 trajectories were used for training and the remaining 200 trajectories were reserved for testing the network performance.

[0139] The network was trained for 8 epochs on the training dataset with a batch size of 256. The Adam optimizer was used with a learning rate of 10 -4 Since the training process is relatively short, no learning rate decay is used.

[0140] In the CTDA network of this algorithm, the data dimension is selected as 40, which corresponds to the size of the input and output matrices. The multi-head attention (MHA) module in CTDA uses 4 heads. A dropout rate of 0.3 is selected for regularization. During testing and filtering, a threshold of 0.1 (λ a =0.1).

[0141] To demonstrate the accuracy of the CTDA network, the following four cases are considered:

[0142] Case 1: The number of clutter is zero, the number of targets ranges from 0 to 100, and the measurement noise is zero;

[0143] Case 2: The number of targets is set to 20, the number of clutter ranges from 0 to 100, and the measurement noise is zero;

[0144] Case 3: The number of clutter is 20, and the number of targets ranges from 0 to 100;

[0145] Case 4: The number of clutter is 20, the number of targets is 10, and the missed detection rate ranges from 0% to 100%.

[0146] The test results of the CTDA network are as follows: Figure 7 First, the correlation accuracy between the method proposed by the present invention and the KM allocation algorithm in case 1 is compared; Figure 7 As shown in (a) of Figure 1, it can be seen that the relative distance accuracy of the KM algorithm alone decreases rapidly with respect to the number of targets. By combining the KM algorithm with the normalization technique given in formula (3), the accuracy becomes very close to 1 in most cases. For the CTDA method, the accuracy exceeds 90% in the test scenario and reaches 95% in most cases, but is slightly lower than the normalized KM algorithm. This is because the training data set of the network contains missed detections and false detections. In the scenario where there are only targets but no false detections and missed detections, the recognition accuracy will be reduced.

[0147] like Figure 7 (b) shows that the KM algorithm becomes ineffective as the number of clutter (i.e., false positives) increases. In contrast, the CTDA method demonstrates reliable correlation performance, achieving over 75% accuracy even in the presence of a large number of false positives.

[0148] The correlation results of case 3 and case 4 are as follows: Figure 7 (c) and Figure 7 As shown in (d), the CTDA network proposed in the present invention shows adaptability in handling unknown and different numbers of targets to be tracked as well as scenarios involving missed detections.

[0149] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A data association method based on a cross Transformer network architecture, characterized in that: The following steps are involved: S100, defining a relative distance feature matrix about the x, y and z directions to ensure the generalization of normalized features of data at different positions; S200, data preprocessing, introducing a block method and a filling method, filling the data in the to-be-associated data set D1 and the data set D2 with additional data of appropriate dimensions and performing block processing, and taking the enhanced matrix containing relative distance features as the input of the network; S300, constructing a data association network based on a cross Transformer, including constructing a Transformer encoder and a Transformer decoder, and then using a row-column cross Transformer network and a 3-channel cross Transformer network to obtain an output matrix; S400, each element of the matrix obtained from the cross-Transformer-based data association network of step S300 represents the association probability of a given input data pair, and the Kuhn-Munkres algorithm is combined to find the best allocation, and then the network output is converted into an association matrix.

2. According to the data association method based on the cross Transformer network architecture of claim 1, it is characterized in that: In step S100, it includes: set up and D1 and D2 represent two sets of position data respectively; Define m×n matrices of relative distances in the x, y, and z directions, namely, relative distance feature matrices, respectively, to characterize the association relationship characteristics between data set D1 and data set D2; Among them, for the x-axis direction, set up The relative distance feature matrix associated with the x direction is as follows, in, and are the x-direction data from dataset D1 and dataset D2 respectively; bias is a small amount used to avoid data errors caused by division by zero; x mid is the geometric median of the absolute values ​​of all relative distances between two data sets, For the relative distance feature matrices in the y-axis direction and the z-axis direction, they are expressed as and 3. According to claim 2, a data association method based on a cross Transformer network architecture is characterized in that: In step S200, it includes: Let N pad is a fixed value, and N pad Less than or equal to m and n; let and are integers for the rows and columns of the scaled padding matrix, is the round-up operation; make is the fill matrix associated with x after partitioning, Division as follows, in, and are the number of partitions for rows and columns respectively; each partition submatrix is an N pad ×N pad matrix; Using the partitioning defined in Equation (4), the scaled padding matrix is ​​reshaped into a batch matrix as follows, The padding matrices associated with directions y and z are defined in the same way as for direction x; subsequently, the augmented matrix containing the relative distance features is used as the network input.

4. According to claim 3, a data association method based on a cross Transformer network architecture is characterized in that: In step S300, it includes: S310, build the Transformer encoder, including three integrated modules, namely multi-head attention, feedforward network and normalization network; The self-attention neural network uses Query to retrieve the attention values ​​corresponding to Keys and assign them to the corresponding Values; then redistributes Value according to the attention weights; let Attention represent the attention processing, in, and Represent Query and Key matrices respectively; represents the y matrix value, d is the dimension of Q, K and V in the self-attention neural network, Q, K and V represent the same feature encoding; Using the matrix W Q , W K and W V Combined with a multi-head attention network, different types of relationships can be extracted from the data. MultiHead(Q,K,V)=Concat(head1,…,head n )W O (7) in, is each attention network head, and W O It is a trainable parameter, which is equivalent to the Q, K and V tensors having the same shape; S320. Build a Transformer decoder, including two multi-head attention networks, set a mask layer in the first multi-head attention network, and use Q, K, and V in the second multi-head attention network. The decoder input is used as the query feature and the encoder input is used as the key and value. The resulting weighted input encoding is combined with the decoder input and passed through normalization and a feed-forward neural network to produce the decoder output. S330, combining the Transformer encoder of step S310 and the Transformer decoder of step S320 to construct a cross-Transformer-based data association network, including: S331. Construct a row-column crossover Transformer network, which is applied to the three channels of the input matrix; each channel is processed separately by a row Transformer encoder and a column Transformer encoder, which treat the rows and columns of the input as feature vectors respectively; the output of the encoder is input to the row and column Transformer decoders in a crossover mode; Introducing the element-by-element method, Among them, h e is the output of the mixing matrix, Reshape(·) means converting the dimension of the matrix to a specified number, [·] means the concatenation operation, W and b are trainable weights and biases, and h1 and h2 are decoders; S332, construct a 3-channel cross Transformer network, the output of the element-by-element mixing processing module consisting of 3-channel data is guided to three Transformer encoders to obtain the encoding features; then, the output of each encoder corresponding to each channel is simultaneously fed to the first MHA module of the decoder of other channels; each output is simultaneously used as the key and value input of the second MHA module in the corresponding decoder of the channel; then the output matrix is ​​obtained by element-by-element mixing, Among them, O1, O2 and O3 are the outputs of the decoders corresponding to the three channels.

5. The data association method based on the cross Transformer network architecture according to claim 4 is characterized in that: In step S300, it also includes using binary cross entropy loss as a cost to calculate the relative association probability of the network output, in, and The incidence matrix O pr and the true incidence matrix O gt Predict the element at row i and column j.

6. The data association method based on the cross Transformer network architecture according to claim 5, characterized in that: It can be used for multi-target tracking or multi-target tracking of monitoring, autonomous driving and robots to associate measurement values ​​with multiple targets, or to resolve unknown correspondences between targets and measurement values.

7. A data association system based on a cross Transformer network architecture, characterized in that: The system has a program module corresponding to the steps of any one of claims 1 to 6 above, and executes the steps in the above-mentioned data association method based on the cross Transformer network architecture when running.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of the data association method based on the cross Transformer network architecture described in any one of claims 1 to 6 when called by a processor.

Images