Intelligent track fusion method based on cross attention

Abstract

The embodiment of the invention relates to the technical field of track data fusion, in particular to an intelligent track fusion method based on cross attention, and the method comprises the steps: carrying out the linear fitting of two pieces of to-be-fused track data through a fitting function, obtaining a track approximation result, and carrying out the subtraction between the two pieces of to-be-fused track data and the track approximation result, so as to obtain two sets of error estimation values; inputting the two groups of error estimation values into a weight discrimination network, deducing the stability and reliability of the two tracks, and obtaining a weight matrix; inputting the two pieces of track data and the weight matrix into a fusion network, performing feature extraction on the two pieces of track data to obtain a feature matrix of the two pieces of track data, and performing weighted fusion to obtain track fusion features; and inputting the flight path fusion features into an output network, and reconstructing the flight path fusion features to obtain a flight path fusion result. The method can well solve the problem that the track fusion precision is insufficient due to the fact that data source credibility and global information are not considered in a traditional method.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of track data fusion, and in particular to a track intelligent fusion method based on cross-attention. Background Art

[0002] In the field of data fusion technology, with the continuous advancement of technology, the types, functions and application scenarios of sensors are becoming increasingly diversified, which makes the multi-sensor systems in modern military and civilian fields more complete and efficient. By increasing the number and types of sensors, these systems can capture target information from multiple angles and different levels, thereby enhancing the stability of the system and making the captured information more accurate and specific. Using appropriate fusion algorithms to effectively integrate the track data from different sensors can obtain more comprehensive and effective air situation information, remove redundant data, and reduce the interference of environmental factors on the system, which is crucial to improving the performance of the system.

[0003] Multi-sensor track fusion technology involves collecting measurement information from various sensors, including homologous information, heterogeneous information, and other auxiliary information. Through specific fusion methods, more accurate track fusion results can be obtained than with a single sensor. In practical applications, we need to consider factors such as the sensor's own measurement accuracy, data expression, and credibility, all of which may affect the fusion process. Therefore, how to effectively fuse data from heterogeneous multi-source sensors, extract key information, and make optimal estimates and decisions based on this information is an urgent problem to be solved in current research.

[0004] In the field of track fusion technology, many classic algorithms have been proposed, providing a strong theoretical basis and practical guidance for multi-sensor data fusion. For example, the simple convex combination track fusion algorithm has been widely used in application scenarios with high real-time requirements. However, this algorithm has limitations. It assumes that the state estimation errors between tracks are independent of each other, which is often not true in practical applications. In order to overcome this limitation, the mutual covariance combination track fusion algorithm considers the correlation between tracks, thereby improving the fusion accuracy. In addition, the hierarchical fusion algorithm further enriches the methodology of track fusion by analyzing similar prior information in the data set. The covariance cross fusion algorithm effectively solves the problem of unknown error covariance matrix of local track state estimation. While the fast sequential covariance cross fusion algorithm improves computational efficiency, its accuracy is not significantly correlated with the sensor fusion order. The distributed fusion structure with global information feedback further improves the performance of the fusion algorithm. The maximum allocation covariance method effectively solves the fusion problem when the track correlation is unknown by constructing the relevant parameters between multiple sensor tracks.

[0005] Deep learning technology is gaining increasing attention in the field of data fusion technology due to its excellent feature extraction and learning capabilities, and has been widely used in the research and practice of track fusion. Data-driven track fusion methods construct deep neural networks and use a large number of labeled local tracks and real track data sets to dynamically update and learn network parameters to achieve accurate track prediction and track fusion. These methods use carefully constructed data sets and advanced network structures to autonomously extract features from track data and deeply explore the potential useful information of track data. Compared with traditional algorithms, deep learning-driven track fusion technology can more effectively integrate multi-sensor information and improve the accuracy of fusion results. Due to the adaptability and powerful data processing capabilities of deep learning models, they show great development potential in the field of track fusion.

[0006] However, the deep learning-driven track fusion technology currently proposed still has certain defects.

[0007] First, due to technical differences, environmental interference, target characteristics and other factors, the accuracy and reliability of track data collected by different radar systems vary significantly. The track data fusion methods currently proposed fail to fully consider the quality and credibility of the data sources when processing these different data sources, thus affecting the stability and reliability of the fusion results.

[0008] Second, the currently proposed track data fusion methods are insufficient in considering the global characteristics and long-term dependencies of the data, which limits the in-depth understanding of the overall patterns and trends of the tracks by the relevant models, and thus affects the accuracy of the track data fusion results. Summary of the invention

[0009] In view of this, an embodiment of the present application proposes a track intelligent fusion method based on cross-attention, which can perform track intelligent fusion processing based on cross-attention, solving the problem of insufficient track fusion accuracy caused by traditional methods that do not consider the credibility of data sources and the global information of data, thereby improving the speed and accuracy of target tracking and monitoring.

[0010] In the first aspect, an embodiment of the present application proposes a method for intelligent track fusion based on cross-attention, which includes the following steps: linearly fitting the two track data to be fused through a fitting function to obtain a track approximation result, and then subtracting the two track data from the track approximation result to obtain two sets of error estimates; inputting the two sets of error estimates into a weighted discriminant network, and the weighted discriminant network infers the stability and reliability of the two tracks based on the two sets of error estimates to obtain a weight matrix matching the time series length of the two track data; inputting the two track data and the weight matrix into a fusion network, and the fusion network extracts features from the two track data to obtain a feature matrix of the two track data, and performs weighted fusion based on the weight matrix and the feature matrix of the two track data to obtain a track fusion feature; inputting the track fusion feature into an output network, and the output network reconstructs the track fusion feature to obtain a track fusion result of the two track data.

[0011] Optionally, the two track data to be fused are respectively recorded as the first track data and the second track data, and the two track data to be fused are linearly fitted through a fitting function to obtain a track approximation result, and then the two track data are respectively subtracted from the track approximation result to obtain two sets of error estimation values, including:

[0012] The first track data and the second track data are linearly fitted using the least square linear fitting function to obtain the track approximation result, which is expressed by the formula:

[0013]

[0014] in, Indicates the first track data, represents the second track data, LSLF(·) represents the least squares linear fitting function, X fit Indicates the track approximation result;

[0015] The first track data and the second track data are respectively subtracted from the track approximation result to obtain a first error estimate and a second error estimate. The first error estimate and the second error estimate are expressed by the formula:

[0016]

[0017] in, represents the first error estimate, represents the second error estimate.

[0018] Optionally, the weighted discriminant network is composed of a normalization unit, a feature extraction subnetwork, and a weighted decision subnetwork;

[0019] The normalization unit is used to normalize the first error estimate and the second error estimate to obtain a normalized first error estimate sequence and a normalized second error estimate sequence; the feature extraction subnetwork is composed of a plurality of stacked Transformer encoders, and the Transformer encoder converts the first error estimate sequence and the second error estimate sequence into a first embedding vector and a second embedding vector through an embedding layer, and then adds position coding information to the first embedding vector and the second embedding vector to obtain a position-coded first embedding vector and a position-coded second embedding vector, and then multiplies them by corresponding feature extraction weights to obtain a first intermediate matrix and a second intermediate matrix, and finally performs nonlinear transformation through a feedforward neural network, and then obtains a first feature matrix and a second feature matrix after residual connection and layer normalization processing;

[0020] The weight decision subnetwork consists of a fully connected layer, which is used to map the first feature matrix and the second feature matrix to the weight space to integrate the features extracted by convolutional layers of different scales, form a comprehensive judgment on the weights of the first error estimate and the second error estimate, and finally output a weight matrix that matches the time series length of the two track data.

[0021] Optionally, the weight matrix output by the weighted discriminant network that matches the time series length of the two track data is expressed by the formula:

[0022] α=FC(Q·K T );

[0023] Q=LayerNorm[FFN(Q′)+Q′], K=LayerNorm[FFN(K′)+K′];

[0024]

[0025] Wherein, S1 represents the first error estimation value sequence, S2 represents the second error estimation value sequence, Norm(·) represents the normalization process, represents the first embedding vector, represents the second embedding vector, Embed(·) represents the embedding layer, represents the positional encoding of the first embedding vector, represents the positional encoding of the second embedding vector, represents the first embedding vector after position encoding, represents the second embedding vector after position encoding, W Q and W Kdenote the feature extraction weights corresponding to the first embedding vector after position encoding and the second embedding vector after position encoding, respectively. Q′ denotes the first intermediate matrix, K′ denotes the second intermediate matrix, FFN(·) denotes a feedforward neural network, LayerNorm(·) denotes layer normalization, Q denotes the first feature matrix, K denotes the second feature matrix, FC(·) denotes a fully connected layer, and α denotes the weight matrix output by the weighted discriminant network that matches the time series length of the two track data.

[0026] Optionally, the fusion network is composed of a normalization unit, a feature extraction subnetwork and a fusion unit, and the normalization unit and the feature extraction subnetwork in the fusion network may be the same as or different from the normalization unit and the feature extraction subnetwork in the weighted discriminant network;

[0027] The fusion unit is used to multiply the first feature matrix and the second feature matrix output by the feature extraction subnetwork of the fusion network by α and 1-α respectively, and then perform feature fusion through 1D multi-scale convolution technology, and finally output the track fusion feature.

[0028] Optionally, the track fusion feature output by the fusion network is expressed by the formula:

[0029] X nf =Fuse[α·V1,(1-α)·V2];

[0030] Where V1 represents the first feature matrix output by the feature extraction subnetwork of the fusion network, V2 represents the second feature matrix output by the feature extraction subnetwork of the fusion network, Fuse(·) represents the fusion unit, and X nf Represents the track fusion characteristics output by the fusion network.

[0031] Optionally, the output network is composed of multiple stacked Transformer decoders, which perform nonlinear transformation on the track fusion features through a feedforward neural network to obtain an output feature matrix, then apply residual connection and layer normalization processing, and finally generate the track fusion result through a linear layer.

[0032] Optionally, the track fusion result output by the output network is expressed by the formula:

[0033] X out =Linear(X′ out );

[0034] X o ′ ut =LinearNorm[FFN(X nf ),X nf ];

[0035] Among them, Linear(·) represents the linear layer, X out Represents the track fusion result output by the output network.

[0036] Optionally, the weighted discriminant network, the fusion network and the output network together constitute a track intelligent fusion model. The loss function used when training the track intelligent fusion model is expressed by the following formula:

[0037] loss = L1[Net fuse (X new ,X fuse ),X r ];

[0038] Among them, X r represents the true label, X new represents the track fusion result output by the track intelligent fusion model, X fuse Represents the original track data, Net fuse (·) represents the forward propagation function, L1(·) represents the absolute difference loss function, and loss represents the calculated loss value.

[0039] The cross-attention-based intelligent track fusion method proposed in this application has the following technical effects.

[0040] First, the present application uses the characteristics of error data to infer the stability and reliability of the original track data of the two bands fused. This method can more accurately evaluate the quality of the track data. By combining the characteristics of the error data with the track data, the accuracy of the track can be quantitatively evaluated, thereby taking into account the credibility of each data source in the track fusion process. This detailed analysis and utilization of errors makes the track fusion result more accurate because it not only considers the location information of the data, but also the uncertainty and reliability of the data.

[0041] Second, this application proposes a dynamic weight allocation mechanism. The weight matrix generated by the weighted discriminant network matches the time series length of the original track data, which means that each track data will be assigned a corresponding weight according to its quality and credibility. By multiplying the features obtained after processing the two track data with their respective weight matrices, the fusion network can better handle data sources of different quality and credibility. This personalized weight allocation strategy is adjusted according to the characteristics and reliability of the error, thereby more accurately reflecting the contribution of each data source to the final track fusion result, effectively improving the fusion accuracy.

[0042] Third, the Transformer model is used in the weighted discriminant network, fusion network, and output network. The Transformer model has significant advantages in processing long time series data such as track data. Because the Transformer model uses a self-attention mechanism, it can process each element in the sequence in parallel, rather than one by one like a traditional recurrent neural network. This mechanism allows the model to simultaneously consider the relationship between any two time points in the sequence when analyzing track data. Therefore, the Transformer model can better understand and predict the global patterns and trends of the track, which is crucial for track prediction tasks that need to consider long-term dependencies and complex interactions. In addition, due to the characteristics of parallel processing, the Transformer is also more efficient when processing long sequences, which can significantly reduce computing time and resource consumption.

[0043] On the second aspect, an embodiment of the present application proposes a track intelligent fusion system based on cross-attention, the system comprising: an error estimation module and a track intelligent fusion model, the track intelligent fusion model consisting of a weight discriminant network, a fusion network and an output network; the error estimation module is used to linearly fit the two track data to be fused through a fitting function to obtain a track approximation result, and then subtract the two track data from the track approximation result to obtain two sets of error estimation values; the weight discriminant network is used to infer the stability and reliability of the two tracks based on the two sets of error estimation values, and obtain a weight matrix matching the time series length of the two track data; the fusion network is used to extract features from the two track data to obtain a feature matrix of the two track data, and perform weighted fusion based on the weight matrix and the feature matrix of the two track data to obtain a track fusion feature; the output network is used to reconstruct the track fusion feature to obtain a track fusion result of the two track data.

[0044] It can be understood that the beneficial effects of the second aspect mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 is a flow chart of a track intelligent fusion method based on cross-attention provided in one embodiment of the present application;

[0047] Figure 2is a schematic diagram of the working principle of a weighted discriminant network provided in one embodiment of the present application;

[0048] Figure 3 is a schematic diagram of the working principle of a fusion network provided in one embodiment of the present application;

[0049] Figure 4 is a structural schematic diagram of a track intelligent fusion system based on cross-attention provided in one embodiment of the present application;

[0050] Figure 5 It is a structural diagram of an electronic device deployed in another embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. In the various embodiments of the present application, many technical details are proposed in order to make the reader better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is only for the convenience of description, and the specific implementation mode of the present application should not constitute any limitation. The various embodiments can be combined with each other and referenced to each other under the premise of no contradiction.

[0052] An embodiment of the present application proposes a method for intelligent fusion of tracks based on cross-attention, which is applied to an electronic device, wherein the electronic device can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described using a server as an example. The implementation details of the method for intelligent fusion of tracks based on cross-attention proposed in this embodiment are described in detail below. The following content is only the implementation details provided for ease of understanding and is not necessary for the implementation of this solution. The specific process of the method for intelligent fusion of tracks based on cross-attention proposed in this embodiment can be as follows: Figure 1 As shown, including:

[0053] Step 101, linearly fit the two track data to be fused through a fitting function to obtain a track approximation result, and then subtract the two track data from the track approximation result to obtain two sets of error estimation values.

[0054] In the specific implementation, when the server performs intelligent track fusion, it first needs to perform linear fitting on the two track data to be fused through a fitting function to obtain the track approximation result, and then subtract the two track data from the track approximation result to obtain two corresponding sets of error estimates.

[0055] In one example, the two track data to be fused are respectively recorded as the first track data and the second track data. When the server performs error extraction, it first uses the least squares linear fitting function to perform linear fitting on the two local track data, the first track data and the second track data, to obtain a track approximation result (i.e., an approximation of the real track).

[0056] The track approximation result can be expressed by the formula:

[0057]

[0058] in, Indicates the first track data, represents the second track data, LSLF(·) represents the least squares linear fitting function, X fit Represents the track approximation result.

[0059] Subsequently, the server subtracts the track approximation result from the first track data and the second track data respectively to obtain a first error estimation value and a second error estimation value.

[0060] The first error estimate and the second error estimate can be expressed by the formula:

[0061]

[0062] in, represents the first error estimate, represents the second error estimate.

[0063] Step 102, input the two sets of error estimation values ​​into the weighted discriminant network, and the weighted discriminant network infers the stability and reliability of the two tracks based on the two sets of error estimation values ​​to obtain a weight matrix that matches the time series length of the two track data.

[0064] In the specific implementation, after obtaining two sets of error estimates, the server can use the track intelligent fusion model composed of a weighted discriminant network, a fusion network, and an output network for fusion. The server inputs the two sets of error estimates into the weighted discriminant network, which infers the stability and reliability of the two tracks based on the two sets of error estimates and obtains a weight matrix that matches the time series length of the two track data.

[0065] In an example, the weighted discriminant network works as follows Figure 2 As shown in the figure, the weighted discriminant network is specifically composed of a normalization unit, a feature extraction subnetwork, and a weighted decision subnetwork.

[0066] The normalization unit is responsible for data preprocessing, which is used to normalize the first error estimate and the second error estimate before weight determination to obtain a normalized first error estimate sequence and a normalized second error estimate sequence. The normalization process can be expressed by the formula: Wherein, S1 represents the first error estimation value sequence, S2 represents the second error estimation value sequence, and Norm(·) represents normalization processing.

[0067] The feature extraction subnetwork consists of multiple stacked Transformer encoders and is responsible for in-depth analysis of the error estimates of the track data.

[0068] The Transformer encoder first converts the first error estimate value sequence and the second error estimate value sequence into a first embedding vector and a second embedding vector through an embedding layer. The embedding layer maps each element to a high-dimensional space to capture richer features.

[0069] The processing of the embedding layer can be expressed by the formula: represents the first embedding vector, represents the second embedding vector and Embed(·) represents the embedding layer.

[0070] Next, the Transformer encoder needs to add the position encoding information to the first embedding vector and the second embedding vector to obtain the first embedding vector after position encoding and the second embedding vector after position encoding. This step ensures that the weighted discriminant network can utilize the order information of the elements in the sequence.

[0071] The position encoding process can be expressed by the formula:

[0072]

[0073] in, represents the positional encoding of the first embedding vector, represents the positional encoding of the second embedding vector, represents the first embedding vector after position encoding, Represents the second embedding vector after position encoding.

[0074] For each position p and each dimension i, the position encoding PE can be calculated as follows:

[0075]

[0076] Among them, d model is the dimension of the weighted discriminant network, i.e. the dimension of the embedding vector.

[0077] Subsequently, the server multiplies the first embedding vector and the second embedding vector after position encoding by the corresponding feature extraction weights to obtain the first intermediate matrix and the second intermediate matrix, and finally performs nonlinear transformation through the feedforward neural network, and then obtains the first feature matrix and the second feature matrix after residual connection and layer normalization. Among them, the feedforward neural network contains two linear transformations and an activation function for further feature extraction. The residual connection and layer normalization can stabilize the training process and alleviate the gradient vanishing problem.

[0078] Stacked multiple Transformer encoders are able to extract deeper features well.

[0079] The weight decision subnetwork consists of a fully connected layer, which is used to map the first feature matrix and the second feature matrix to the weight space to integrate the features extracted by the convolutional layers of different scales, form a comprehensive judgment on the weights of the first error estimate and the second error estimate, and finally output a weight matrix that matches the time series length of the two track data. The weight matrix output by the weight discriminant network that matches the time series length of the two track data can be expressed by the formula:

[0080] α=FC(Q·K T );

[0081] Q=LayerNorm[FFN(Q′)+Q′], K=LayerNorm[FFN(K′)+K′];

[0082]

[0083] Among them, W Q and W K They represent the feature extraction weights corresponding to the first embedding vector after position encoding and the second embedding vector after position encoding. In the initialization stage, W Q and W K is randomly generated, Q′ represents the first intermediate matrix, K′ represents the second intermediate matrix, FFN(·) represents the feedforward neural network, LaterNorm(·) represents the layer normalization process, Q represents the first feature matrix, K represents the second feature matrix, FC(·) represents the fully connected layer, and α represents the weight matrix output by the weighted discriminant network that matches the time series length of the two track data. The weight matrix will be used in the subsequent track data fusion process to optimize the accuracy and reliability of the track.

[0084] Step 103, input the two track data and the weight matrix into the fusion network, the fusion network extracts features from the two track data to obtain feature matrices of the two track data, and performs weighted fusion based on the weight matrix and the feature matrix of the two track data to obtain track fusion features.

[0085] In the specific implementation, after the weight discriminant network completes the output of the weight matrix, the two track data and the weight matrix will be input into the fusion network. The fusion network will extract features from the two track data to obtain the feature matrix of the two track data, and perform weighted fusion based on the weight matrix and the feature matrix of the two track data to obtain the track fusion feature.

[0086] In one example, the fusion network can work like this: Figure 3 As shown. The fusion network consists of a normalization unit, a feature extraction subnetwork and a fusion unit. The normalization unit and feature extraction subnetwork in the fusion network can be the same as or different from the normalization unit and feature extraction subnetwork in the weighted discriminant network. However, the working principles of the normalization unit and feature extraction subnetwork in the fusion network and the normalization unit and feature extraction subnetwork in the weighted discriminant network are the same, which will not be repeated here.

[0087] The fusion unit is used to multiply the first feature matrix and the second feature matrix output by the feature extraction subnetwork of the fusion network by α and 1-α respectively, and then perform feature fusion through 1D multi-scale convolution technology, and finally output the track fusion feature. 1D multi-scale convolution technology can well realize the efficient fusion of deep features.

[0088] The track fusion characteristics output by the fusion network can be expressed by the formula:

[0089] X nf =Fuse[α·V1,(1-α)·V2];

[0090] Where V1 represents the first feature matrix output by the feature extraction subnetwork of the fusion network, V2 represents the second feature matrix output by the feature extraction subnetwork of the fusion network, Fuse(·) represents the fusion unit, and X nf Represents the track fusion characteristics output by the fusion network.

[0091] Step 104, inputting the track fusion feature into the output network, and the output network reconstructs the track fusion feature to obtain the track fusion result of the two track data.

[0092] In a specific implementation, after the fusion network completes the output of the track fusion features, the track fusion features can be input into the output network, and the output network reconstructs the track fusion features to obtain the track fusion results of the two track data.

[0093] In one example, the output network consists of multiple stacked Transformer decoders, which perform nonlinear transformations on track fusion features through a feedforward neural network to obtain an output feature matrix, then apply residual connections and layer normalization, and finally generate track fusion results through a linear layer.

[0094] The track fusion result output by the output network can be expressed by the formula:

[0095] X out =Linear(X′ out );

[0096] X′ out =LayerNorm[FFN(X nf ),X nf ];

[0097] Among them, Linear(·) represents the linear layer, X out Represents the track fusion result output by the output network.

[0098] In an example, the weighted discriminant network, the fusion network and the output network together constitute a track intelligent fusion model, which can be deployed and executed on the Kylin v10 operating system.

[0099] In one example, the loss function used by the server when training the track intelligent fusion model is expressed by the following formula:

[0100] loss = L1[Net fuse (X new ,X fuse ),X r ];

[0101] Among them, X r represents the true label, X new represents the track fusion result output by the track intelligent fusion model, X fuse Represents the original track data, Net fuse (·) represents the forward propagation function, L1(·) represents the absolute difference loss function, and loss represents the calculated loss value.

[0102] In this embodiment, a track intelligent fusion method based on cross-attention is proposed, which has the following technical effects.

[0103] First, the present application uses the characteristics of error data to infer the stability and reliability of the original track data of the two bands fused. This method can more accurately evaluate the quality of the track data. By combining the characteristics of the error data with the track data, the accuracy of the track can be quantitatively evaluated, thereby taking into account the credibility of each data source in the track fusion process. This detailed analysis and utilization of errors makes the track fusion result more accurate because it not only considers the location information of the data, but also the uncertainty and reliability of the data.

[0104] Second, this application proposes a dynamic weight allocation mechanism. The weight matrix generated by the weighted discriminant network matches the time series length of the original track data, which means that each track data will be assigned a corresponding weight according to its quality and credibility. By multiplying the features obtained after processing the two track data with their respective weight matrices, the fusion network can better handle data sources of different quality and credibility. This personalized weight allocation strategy is adjusted according to the characteristics and reliability of the error, thereby more accurately reflecting the contribution of each data source to the final track fusion result, effectively improving the fusion accuracy.

[0105] Third, the Transformer model is used in the weighted discriminant network, fusion network, and output network. The Transformer model has significant advantages in processing long time series data such as track data. Because the Transformer model uses a self-attention mechanism, it can process each element in the sequence in parallel, rather than one by one like a traditional recurrent neural network. This mechanism allows the model to simultaneously consider the relationship between any two time points in the sequence when analyzing track data. Therefore, the Transformer model can better understand and predict the global patterns and trends of the track, which is crucial for track prediction tasks that need to consider long-term dependencies and complex interactions. In addition, due to the characteristics of parallel processing, the Transformer is also more efficient when processing long sequences, which can significantly reduce computing time and resource consumption.

[0106] The step division of the above methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0107] Another embodiment of the present application proposes a track intelligent fusion system based on cross-attention. The implementation details of the track intelligent fusion system based on cross-attention proposed in this embodiment are specifically described below. The following content is only for the convenience of understanding the implementation details provided, and is not necessary for the implementation of this example. Figure 4 It is a structural diagram of a track intelligent fusion system based on cross-attention proposed in this embodiment. The system includes: an error estimation module 200 and a track intelligent fusion model 300. The track intelligent fusion model 300 is composed of a weighted discriminant network 301, a fusion network 302 and an output network 303.

[0108] The error estimation module 200 is used to perform linear fitting on the two track data to be fused through a fitting function to obtain a track approximation result, and then respectively subtract the two track data from the track approximation result to obtain two sets of error estimation values.

[0109] The weighted discriminant network 301 is used to infer the stability and reliability of the two tracks based on the two sets of error estimation values, and obtain a weight matrix that matches the time series length of the two track data.

[0110] The fusion network 302 is used to extract features from the two track data to obtain feature matrices of the two track data, and perform weighted fusion based on the weight matrix and the feature matrices of the two track data to obtain track fusion features.

[0111] The output network 303 is used to reconstruct the track fusion feature to obtain the track fusion result of the two track data.

[0112] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0113] It is not difficult to find that this embodiment is a system embodiment corresponding to the above method embodiment, and this embodiment can be implemented in conjunction with the above method embodiment. The relevant technical details and technical effects mentioned in the above embodiments are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiments.

[0114] Another embodiment of the present application deploys an electronic device, whose specific structure is as follows Figure 5As shown, it includes: at least one processor 401; and a memory 402 that is communicatively connected to the at least one processor 401; wherein the memory 402 stores instructions that can be executed by the at least one processor 401, and the instructions are executed by the at least one processor 401 so that the at least one processor 401 can execute a track intelligent fusion method based on cross-attention as described in the above embodiments.

[0115] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be further described in this article. The bus interface is responsible for providing an interface between the bus and the transceiver. The transceiver can be one component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium through an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0116] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0117] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM (Read-Only Memory), RAM (Random Access Memory), disk or optical disk and other media that can store program codes.

[0118] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A track intelligent fusion method based on cross attention, characterized in that: include: The two track data to be fused are linearly fitted through the fitting function to obtain the track approximation result, and then the two track data are subtracted from the track approximation result to obtain two sets of error estimation values; The two sets of error estimation values ​​are input into the weighted discriminant network, and the weighted discriminant network infers the stability and reliability of the two tracks based on the two sets of error estimation values, and obtains a weight matrix that matches the time series length of the two track data; The two track data and the weight matrix are input into the fusion network, and the fusion network extracts the features of the two track data to obtain the feature matrix of the two track data, and performs weighted fusion based on the weight matrix and the feature matrix of the two track data to obtain the track fusion feature; The track fusion features are input into the output network, and the output network reconstructs the track fusion features to obtain the track fusion results of two track data.

2. The track intelligent fusion method based on cross attention according to claim 1 is characterized in that: The two track data to be fused are recorded as the first track data and the second track data respectively. The two track data to be fused are linearly fitted through the fitting function to obtain the track approximation result. Then, the two track data are subtracted from the track approximation result to obtain two sets of error estimates, including: The first track data and the second track data are linearly fitted using the least square linear fitting function to obtain the track approximation result, which is expressed by the formula: in, Indicates the first track data, represents the second track data, LSLF(·) represents the least squares linear fitting function, X fit Indicates the track approximation result; The first track data and the second track data are respectively subtracted from the track approximation result to obtain a first error estimate and a second error estimate. The first error estimate and the second error estimate are expressed by the formula: in, represents the first error estimate, represents the second error estimate.

3. The track intelligent fusion method based on cross attention according to claim 2 is characterized in that: The weighted discriminant network consists of a normalization unit, a feature extraction subnetwork, and a weighted decision subnetwork; The normalization unit is used to perform normalization processing on the first error estimation value and the second error estimation value to obtain a normalized first error estimation value sequence and a normalized second error estimation value sequence; The feature extraction subnetwork is composed of multiple stacked Transformer encoders. The Transformer encoder converts the first error estimate value sequence and the second error estimate value sequence into the first embedding vector and the second embedding vector through the embedding layer, then adds the position encoding information to the first embedding vector and the second embedding vector to obtain the first embedding vector after position encoding and the second embedding vector after position encoding, and then multiplies them by the corresponding feature extraction weights to obtain the first intermediate matrix and the second intermediate matrix, and finally performs nonlinear transformation through a feedforward neural network, and then obtains the first feature matrix and the second feature matrix after residual connection and layer normalization processing; The weight decision subnetwork consists of a fully connected layer, which is used to map the first feature matrix and the second feature matrix to the weight space to integrate the features extracted by convolutional layers of different scales, form a comprehensive judgment on the weights of the first error estimate and the second error estimate, and finally output a weight matrix that matches the time series length of the two track data.

4. The track intelligent fusion method based on cross attention according to claim 3 is characterized in that: The weight matrix output by the weighted discriminant network matches the time series length of the two track data and is expressed by the formula: α=FC(Q·K T ); Q=LayerNorm[FFN(Q′)+Q′], K=LayerNorm[FFN(K′)+K′]; Wherein, S1 represents the first error estimation value sequence, S2 represents the second error estimation value sequence, Norm(·) represents the normalization process, represents the first embedding vector, represents the second embedding vector, Embed(·) represents the embedding layer, represents the positional encoding of the first embedding vector, represents the positional encoding of the second embedding vector, represents the first embedding vector after position encoding, represents the second embedding vector after position encoding, W Q and W K denote the feature extraction weights corresponding to the first embedding vector after position encoding and the second embedding vector after position encoding, respectively. Q′ denotes the first intermediate matrix, K′ denotes the second intermediate matrix, FFN(·) denotes a feedforward neural network, LayerNorm(·) denotes layer normalization, Q denotes the first feature matrix, K denotes the second feature matrix, FC(·) denotes a fully connected layer, and α denotes the weight matrix output by the weighted discriminant network that matches the time series length of the two track data.

5. The track intelligent fusion method based on cross attention according to claim 4 is characterized in that: The fusion network is composed of a normalization unit, a feature extraction subnetwork and a fusion unit. The normalization unit and the feature extraction subnetwork in the fusion network may be the same as or different from the normalization unit and the feature extraction subnetwork in the weighted discriminant network. The fusion unit is used to multiply the first feature matrix and the second feature matrix output by the feature extraction subnetwork of the fusion network by α and 1-α respectively, and then perform feature fusion through 1D multi-scale convolution technology, and finally output the track fusion feature.

6. The track intelligent fusion method based on cross attention according to claim 5 is characterized in that: The track fusion characteristics output by the fusion network are expressed by the formula: X nf =Fuse[α·V1,(1-α)·V2]; Where V1 represents the first feature matrix output by the feature extraction subnetwork of the fusion network, V2 represents the second feature matrix output by the feature extraction subnetwork of the fusion network, Fuse(·) represents the fusion unit, and X nf Represents the track fusion characteristics output by the fusion network.

7. The track intelligent fusion method based on cross attention according to claim 6 is characterized in that: The output network consists of multiple stacked Transformer decoders. The Transformer decoder performs nonlinear transformation on the track fusion features through a feedforward neural network to obtain the output feature matrix, then applies residual connection and layer normalization processing, and finally generates the track fusion result through a linear layer.

8. The track intelligent fusion method based on cross attention according to claim 7 is characterized in that: The track fusion result output by the output network is expressed by the formula: X out =Linear(X′ out ); X′ out =LinearNorm[FFN(X nf ),X nf ]; Among them, Linear(·) represents the linear layer, X out Represents the track fusion result output by the output network.

9. A track intelligent fusion method based on cross attention according to any one of claims 1 to 8, characterized in that: The weighted discriminant network, fusion network and output network together constitute the track intelligent fusion model. The loss function used in training the track intelligent fusion model is expressed by the following formula: loss=L1[Net fuse (X new ,X fuse ),X r ]; Among them, X r represents the true label, X new represents the track fusion result output by the track intelligent fusion model, X fuse Represents the original track data, Net fuse (·) represents the forward propagation function, L1(·) represents the absolute difference loss function, and loss represents the calculated loss value.

10. A track intelligent fusion system based on cross attention, characterized in that: include: Error estimation module and track intelligent fusion model. The track intelligent fusion model consists of a weighted discriminant network, a fusion network and an output network; The error estimation module is used to perform linear fitting on the two track data to be fused through the fitting function to obtain the track approximation result, and then subtract the two track data from the track approximation result to obtain two sets of error estimation values; A weighted discriminant network is used to infer the stability and reliability of the two tracks based on two sets of error estimates, and obtain a weight matrix that matches the time series length of the two track data; A fusion network is used to extract features from the two track data to obtain feature matrices of the two track data, and to perform weighted fusion based on a weight matrix and the feature matrices of the two track data to obtain track fusion features; The output network is used to reconstruct the track fusion features and obtain the track fusion result of the two track data.

Images