A track intelligent fusion method based on cross attention

Abstract

The embodiments of the present application relate to the field of track data fusion technology, and in particular to a track intelligent fusion method based on cross-attention, comprising: performing linear fitting on the two track data to be fused through a fitting function to obtain a track approximation result, and then subtracting the two track approximation results to obtain two sets of error estimates; inputting the two sets of error estimates into a weighted discriminant network to infer the stability and reliability of the two tracks and obtain a weight matrix; inputting the two track data and the weight matrix into a fusion network, performing feature extraction on the two track data to obtain a feature matrix of the two track data, and performing weighted fusion to obtain a track fusion feature; inputting the track fusion feature into an output network, reconstructing the track fusion feature, and obtaining a track fusion result. This method can well solve the problem of insufficient track fusion accuracy caused by traditional methods that do not consider the credibility of data sources and global information.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of track data fusion, and in particular to a track intelligent fusion method based on cross-attention. Background Art

[0002] In the field of data fusion, technological advancements are leading to an increasing diversity in sensor types, functions, and application scenarios. This has led to more sophisticated and efficient multi-sensor systems in both the military and civilian sectors. By increasing the number and types of sensors, these systems can capture target information from multiple angles and levels, enhancing system stability and making the captured information more accurate and specific. Using appropriate fusion algorithms to effectively integrate track data from various sensors can yield more comprehensive and effective air situation information, eliminate redundant data, and reduce environmental interference, which is crucial for improving system performance.

[0003] Multi-sensor track fusion technology involves collecting measurement information from various sensors, including homogeneous information, heterogeneous information, and other auxiliary information. Through specific fusion methods, more accurate track fusion results can be obtained than those obtained from a single sensor. In practical applications, factors such as the sensor's own measurement accuracy, data representation, and reliability must be considered, all of which may affect the fusion process. Therefore, how to effectively fuse heterogeneous multi-source sensor data, extract key information, and make optimal estimates and decisions based on this information is a pressing issue in current research.

[0004] In the field of track fusion technology, numerous classic algorithms have been proposed, providing a strong theoretical foundation and practical guidance for multi-sensor data fusion. For example, the simple convex combination track fusion algorithm has been widely used in applications requiring high real-time performance. However, this algorithm has limitations. It assumes that the state estimation errors between tracks are independent of each other, an assumption that often does not hold true in practical applications. To overcome this limitation, the cross-covariance combination track fusion algorithm considers the correlation between tracks, thereby improving fusion accuracy. In addition, the hierarchical fusion algorithm further enriches track fusion methodology by analyzing similar prior information in the dataset. The covariance cross fusion algorithm effectively addresses the problem of unknown error covariance matrices for local track state estimation. The fast sequential covariance cross fusion algorithm improves computational efficiency while its accuracy is not significantly dependent on the sensor fusion order. Distributed fusion structures with global information feedback further enhance the performance of fusion algorithms. The maximum distribution covariance method effectively addresses the fusion problem when track correlation is unknown by constructing correlation parameters between multiple sensor tracks.

[0005] Deep learning technology, with its superior feature extraction and learning capabilities, is gaining increasing attention in the field of data fusion technology and has been widely used in the research and practice of track fusion. Data-driven track fusion methods construct deep neural networks and use a large number of labeled local tracks and real track data sets to dynamically update and learn network parameters to achieve accurate track prediction and track fusion. These methods utilize carefully constructed data sets and advanced network structures to autonomously extract features from track data and deeply explore the potential useful information in track data. Compared with traditional algorithms, deep learning-driven track fusion technology can more effectively integrate multi-sensor information and improve the accuracy of fusion results. Due to the adaptability and powerful data processing capabilities of deep learning models, they show great development potential in the field of track fusion.

[0006] However, the deep learning-driven track fusion technology currently proposed still has certain defects.

[0007] First, the accuracy and reliability of track data collected by different radar systems vary significantly due to factors such as technological differences, environmental interference, and target characteristics. Existing track data fusion methods fail to fully consider the quality and credibility of these different data sources, thus affecting the stability and reliability of the fusion results.

[0008] Second, the currently proposed track data fusion methods are insufficient in considering the global characteristics and long-term dependencies of the data, which limits the relevant models' in-depth understanding of the overall patterns and trends of the tracks, and thus affects the accuracy of the track data fusion results. Summary of the Invention

[0009] In view of this, an embodiment of the present application proposes a track intelligent fusion method based on cross-attention, which can perform track intelligent fusion processing based on cross-attention, solving the problem of insufficient track fusion accuracy caused by the traditional method not considering the credibility of the data source and the global information of the data, and improving the speed and accuracy of target tracking and monitoring.

[0010] In the first aspect, an embodiment of the present application proposes a method for intelligent track fusion based on cross-attention, which includes the following steps: linearly fitting the two track data to be fused through a fitting function to obtain a track approximation result, and then subtracting the two track data from the track approximation result to obtain two sets of error estimation values; inputting the two sets of error estimation values ​​into a weighted discriminant network, and the weighted discriminant network infers the stability and reliability of the two tracks based on the two sets of error estimation values ​​to obtain a weight matrix matching the time series length of the two track data; inputting the two track data and the weight matrix into a fusion network, and the fusion network extracts features from the two track data to obtain a feature matrix of the two track data, and performs weighted fusion based on the weight matrix and the feature matrix of the two track data to obtain a track fusion feature; inputting the track fusion feature into an output network, and the output network reconstructs the track fusion feature to obtain a track fusion result of the two track data.

[0011] Optionally, the two track data to be fused are respectively recorded as the first track data and the second track data. The two track data to be fused are linearly fitted by a fitting function to obtain a track approximation result. Then, the two track data are respectively subtracted from the track approximation result to obtain two sets of error estimates, including:

[0012] The first track data and the second track data are linearly fitted using the least squares linear fitting function to obtain the track approximation result. The track approximation result is expressed by the formula:

[0013]

[0014] in, Indicates the first track data, represents the second track data, LSLF(·) represents the least squares linear fitting function, X fit Indicates the track approximation result;

[0015] The first track data and the second track data are respectively subtracted from the track approximation result to obtain a first error estimate and a second error estimate. The first error estimate and the second error estimate are expressed by the formula:

[0016]

[0017] in, represents the first error estimate, represents the second error estimate.

[0018] Optionally, the weighted discriminant network consists of a normalization unit, a feature extraction subnetwork, and a weighted decision subnetwork;

[0019] The normalization unit is used to normalize the first error estimate and the second error estimate to obtain a normalized first error estimate sequence and a normalized second error estimate sequence; the feature extraction subnetwork is composed of multiple stacked Transformer encoders, and the Transformer encoder converts the first error estimate sequence and the second error estimate sequence into a first embedding vector and a second embedding vector through an embedding layer, and then adds position coding information to the first embedding vector and the second embedding vector to obtain a position-coded first embedding vector and a position-coded second embedding vector, and then multiplies them by corresponding feature extraction weights to obtain a first intermediate matrix and a second intermediate matrix, and finally performs nonlinear transformation through a feedforward neural network, and then obtains a first feature matrix and a second feature matrix after residual connection and layer normalization.

[0020] The weight decision subnetwork consists of a fully connected layer, which is used to map the first feature matrix and the second feature matrix into the weight space to integrate the features extracted by the convolutional layers of different scales, form a comprehensive judgment on the weights of the first error estimate and the second error estimate, and finally output a weight matrix that matches the time series length of the two track data.

[0021] Optionally, the weight matrix output by the weighted discriminant network matches the time series length of the two track data and is expressed by the formula:

[0022] α=FC(Q·K T );

[0023] Q=LayerNorm[FFN(Q′)+Q′], K=LayerNorm[FFN(K′)+K′];

[0024]

[0025] Where S1 represents the first error estimation value sequence, S2 represents the second error estimation value sequence, Norm(·) represents the normalization process, represents the first embedding vector, represents the second embedding vector, Embed(·) represents the embedding layer, represents the positional encoding of the first embedding vector, represents the positional encoding of the second embedding vector, represents the first embedding vector after position encoding, represents the second embedding vector after position encoding, W Q and W Kdenote the feature extraction weights corresponding to the first embedding vector after position encoding and the second embedding vector after position encoding, respectively. Q′ denotes the first intermediate matrix, K′ denotes the second intermediate matrix, FFN(·) denotes the feedforward neural network, LayerNorm(·) denotes the layer normalization process, Q denotes the first feature matrix, K denotes the second feature matrix, FC(·) denotes the fully connected layer, and α denotes the weight matrix output by the weighted discriminant network that matches the time series length of the two track data.

[0026] Optionally, the fusion network is composed of a normalization unit, a feature extraction subnetwork and a fusion unit. The normalization unit and the feature extraction subnetwork in the fusion network may be the same as or different from the normalization unit and the feature extraction subnetwork in the weighted discriminant network.

[0027] The fusion unit is used to multiply the first feature matrix and the second feature matrix output by the feature extraction subnetwork of the fusion network by α and 1-α respectively, and then perform feature fusion through 1D multi-scale convolution technology to finally output the track fusion feature.

[0028] Optionally, the track fusion feature output by the fusion network is expressed by the formula:

[0029] X nf =Fuse[α·V1,(1-α)·V2];

[0030] Among them, V1 represents the first feature matrix output by the feature extraction sub-network of the fusion network, V2 represents the second feature matrix output by the feature extraction sub-network of the fusion network, Fuse(·) represents the fusion unit, X nf Represents the track fusion features output by the fusion network.

[0031] Optionally, the output network consists of multiple stacked Transformer decoders, which perform nonlinear transformation on the track fusion features through a feedforward neural network to obtain an output feature matrix, then apply residual connection and layer normalization processing, and finally generate the track fusion result through a linear layer.

[0032] Optionally, the track fusion result output by the output network is expressed by the formula:

[0033] X out =Linear(X′ out );

[0034] X o ′ ut =LinearNorm[FFN(X nf ),X nf ];

[0035] Among them, Linear(·) represents the linear layer, X out Represents the track fusion result output by the output network.

[0036] Optionally, the weighted discriminant network, the fusion network, and the output network together constitute a track intelligent fusion model. The loss function used when training the track intelligent fusion model is expressed by the following formula:

[0037] loss=L1[Net fuse (X new ,X fuse ),X r ];

[0038] Among them, X r represents the true label, X new represents the track fusion result output by the track intelligent fusion model, X fuse Represents the original track data, Net fuse (·) represents the forward propagation function, L1(·) represents the absolute difference loss function, and loss represents the calculated loss value.

[0039] The cross-attention-based intelligent track fusion method proposed in this application has the following technical effects.

[0040] First, this application uses the characteristics of error data to infer the stability and reliability of the two-band fused original track data. This method can more accurately evaluate the quality of the track data. By combining the characteristics of the error data with the track data, the accuracy of the track can be quantitatively evaluated, thereby taking into account the credibility of each data source in the track fusion process. This detailed analysis and utilization of errors makes the track fusion result more accurate because it not only considers the location information of the data, but also the uncertainty and reliability of the data.

[0041] Second, this application proposes a dynamic weight allocation mechanism. The weight matrix generated by the weighted discriminant network matches the time series length of the original track data, which means that each track data will be assigned a corresponding weight according to its quality and credibility. By multiplying the features obtained after processing the two track data with their respective weight matrices, the fusion network can better handle data sources of different quality and credibility. This personalized weight allocation strategy is adjusted according to the characteristics and reliability of the error, thereby more accurately reflecting the contribution of each data source to the final track fusion result, effectively improving the fusion accuracy.

[0042] Third, the Transformer model is used in the weighted discriminant network, fusion network, and output network. The Transformer model has significant advantages in processing long time series data such as track data. Because the Transformer model uses a self-attention mechanism, it can process each element in the sequence in parallel, rather than one by one like traditional recurrent neural networks. This mechanism allows the model to simultaneously consider the relationship between any two time points in the sequence when analyzing track data. Therefore, the Transformer model can better understand and predict the global patterns and trends of the track, which is crucial for track prediction tasks that need to consider long-term dependencies and complex interactions. In addition, due to its parallel processing characteristics, the Transformer is also more efficient when processing long sequences, which can significantly reduce computing time and resource consumption.

[0043] On the second aspect, an embodiment of the present application proposes a track intelligent fusion system based on cross-attention, which includes: an error estimation module and a track intelligent fusion model, which consists of a weighted discriminant network, a fusion network and an output network; the error estimation module is used to perform linear fitting on the two track data to be fused through a fitting function to obtain a track approximation result, and then subtract the two track data from the track approximation result to obtain two sets of error estimation values; the weighted discriminant network is used to infer the stability and reliability of the two tracks based on the two sets of error estimation values, and obtain a weight matrix that matches the time series length of the two track data; the fusion network is used to extract features from the two track data to obtain a feature matrix of the two track data, and perform weighted fusion based on the weight matrix and the feature matrix of the two track data to obtain a track fusion feature; the output network is used to reconstruct the track fusion feature to obtain a track fusion result of the two track data.

[0044] It can be understood that the beneficial effects of the second aspect mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related technologies, the following is a brief introduction to the drawings required for use in the embodiments of the present application or the description of the related technologies. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0046] Figure 1 is a flow chart of a track intelligent fusion method based on cross-attention provided in one embodiment of the present application;

[0047] Figure 2is a schematic diagram of the working principle of a weighted discriminant network provided in one embodiment of the present application;

[0048] Figure 3 is a schematic diagram of the working principle of the fusion network provided in one embodiment of the present application;

[0049] Figure 4 1 is a schematic structural diagram of a track intelligent fusion system based on cross-attention provided in one embodiment of the present application;

[0050] Figure 5 It is a structural diagram of an electronic device deployed in another embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. In the various embodiments of the present application, many technical details are proposed to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be implemented. The division of the following embodiments is only for the convenience of description and should not constitute any limitation on the specific implementation of the present application. The various embodiments can be combined with each other and referenced to each other under the premise of no contradiction.

[0052] An embodiment of the present application proposes a method for intelligent fusion of tracks based on cross-attention, which is applied to electronic devices, wherein the electronic device can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described using a server as an example. The following is a detailed description of the implementation details of the method for intelligent fusion of tracks based on cross-attention proposed in this embodiment. The following content is only the implementation details provided for easy understanding and is not necessary for the implementation of this solution. The specific process of the method for intelligent fusion of tracks based on cross-attention proposed in this embodiment can be as follows: Figure 1 As shown, including:

[0053] In step 101, the two track data to be fused are linearly fitted using a fitting function to obtain a track approximation result, and then the two track data are subtracted from the track approximation result to obtain two sets of error estimates.

[0054] In the specific implementation, when the server performs intelligent track fusion, it first needs to perform linear fitting on the two track data to be fused through a fitting function to obtain a track approximation result, and then subtract the two track data from the track approximation result to obtain two corresponding sets of error estimates.

[0055] In one example, the two track data to be fused are respectively recorded as the first track data and the second track data. When the server performs error extraction, it first uses the least squares linear fitting function to perform linear fitting on the two local track data, the first track data and the second track data, to obtain a track approximation result (i.e., an approximation of the true track).

[0056] The track approximation result can be expressed by the formula:

[0057]

[0058] in, Indicates the first track data, represents the second track data, LSLF(·) represents the least squares linear fitting function, X fit Indicates the track approximation result.

[0059] Subsequently, the server subtracts the track approximation result from the first track data and the second track data respectively to obtain a first error estimation value and a second error estimation value.

[0060] The first error estimate and the second error estimate can be expressed as:

[0061]

[0062] in, represents the first error estimate, represents the second error estimate.

[0063] In step 102, the two sets of error estimates are input into a weighted discriminant network, which infers the stability and reliability of the two tracks based on the two sets of error estimates to obtain a weight matrix that matches the time series length of the two track data.

[0064] In practice, after obtaining two sets of error estimates, the server performs fusion using a track intelligent fusion model consisting of a weighted discriminant network, a fusion network, and an output network. The server inputs the two sets of error estimates into the weighted discriminant network, which infers the stability and reliability of the two tracks based on the two error estimates, generating a weight matrix that matches the time series length of the two track data.

[0065] In an example, the working principle of the weighted discriminant network can be as follows Figure 2 As shown in the figure, the weighted discriminant network is specifically composed of a normalization unit, a feature extraction subnetwork, and a weighted decision subnetwork.

[0066] The normalization unit is responsible for data preprocessing, which is used to normalize the first error estimate and the second error estimate before weight determination to obtain a normalized first error estimate sequence and a normalized second error estimate sequence. The normalization process can be expressed by the formula: Wherein, S1 represents the first error estimation value sequence, S2 represents the second error estimation value sequence, and Norm(·) represents normalization processing.

[0067] The feature extraction subnetwork consists of multiple stacked Transformer encoders and is responsible for performing in-depth analysis of the error estimates of the track data.

[0068] The Transformer encoder first converts the first error estimate sequence and the second error estimate sequence into a first embedding vector and a second embedding vector through an embedding layer. The embedding layer maps each element to a high-dimensional space to capture richer features.

[0069] The processing of the embedding layer can be expressed by the formula: represents the first embedding vector, represents the second embedding vector, and Embed(·) represents the embedding layer.

[0070] Next, the Transformer encoder needs to add position encoding information to the first embedding vector and the second embedding vector to obtain the position-encoded first embedding vector and the position-encoded second embedding vector. This step ensures that the weighted discriminant network can utilize the order information of the elements in the sequence.

[0071] The position encoding process can be expressed by the formula:

[0072]

[0073] in, represents the positional encoding of the first embedding vector, represents the positional encoding of the second embedding vector, represents the first embedding vector after position encoding, Represents the second embedding vector after position encoding.

[0074] For each position p and each dimension i, the position encoding PE can be calculated as follows:

[0075]

[0076] Among them, d model is the dimension of the weighted discriminant network, that is, the dimension of the embedding vector.

[0077] The server then multiplies the position-encoded first and second embedding vectors by the corresponding feature extraction weights to obtain the first and second intermediate matrices. Finally, a nonlinear transformation is performed through a feedforward neural network. After residual connections and layer normalization, the first and second feature matrices are obtained. The feedforward neural network includes two linear transformations and an activation function for further feature extraction. Residual connections and layer normalization stabilize the training process and alleviate the vanishing gradient problem.

[0078] Stacked multiple Transformer encoders are able to extract deeper features well.

[0079] The weight decision subnetwork consists of a fully connected layer, which is used to map the first feature matrix and the second feature matrix into the weight space to integrate the features extracted by the convolutional layers of different scales, form a comprehensive judgment on the weights of the first error estimate and the second error estimate, and finally output a weight matrix that matches the time series length of the two track data. The weight matrix output by the weight discriminant network that matches the time series length of the two track data can be expressed by the formula:

[0080] α=FC(Q·K T );

[0081] Q=LayerNorm[FFN(Q′)+Q′], K=LayerNorm[FFN(K′)+K′];

[0082]

[0083] Among them, W Q and W K Respectively represent the feature extraction weights corresponding to the first embedding vector after position encoding and the second embedding vector after position encoding. In the initialization stage, W Q and W K is randomly generated, Q′ represents the first intermediate matrix, K′ represents the second intermediate matrix, FFN(·) represents the feedforward neural network, LaterNorm(·) represents layer normalization, Q represents the first feature matrix, K represents the second feature matrix, FC(·) represents the fully connected layer, and α represents the weight matrix output by the weighted discriminant network that matches the time series length of the two track data. The weight matrix will be used in the subsequent track data fusion process to optimize the accuracy and reliability of the track.

[0084] In step 103, the two track data and the weight matrix are input into the fusion network. The fusion network extracts features from the two track data to obtain feature matrices of the two track data, and performs weighted fusion based on the weight matrix and the feature matrices of the two track data to obtain track fusion features.

[0085] In the specific implementation, after the weighted discriminant network completes the output of the weight matrix, the two track data and the weight matrix will be input into the fusion network. The fusion network will extract features from the two track data to obtain the feature matrix of the two track data, and perform weighted fusion based on the weight matrix and the feature matrix of the two track data to obtain the track fusion feature.

[0086] In one example, the converged network can work like this: Figure 3 As shown. The fusion network consists of a normalization unit, a feature extraction subnetwork, and a fusion unit. The normalization unit and feature extraction subnetwork in the fusion network can be the same as or different from the normalization unit and feature extraction subnetwork in the weighted discriminant network. However, the working principles of the normalization unit and feature extraction subnetwork in the fusion network and the weighted discriminant network are the same, which will not be repeated here.

[0087] The fusion unit multiplies the first and second feature matrices output by the feature extraction subnetwork of the fusion network by α and 1-α, respectively. It then fuses the features using 1D multi-scale convolution technology, ultimately outputting the track fusion feature. 1D multi-scale convolution technology effectively achieves efficient fusion of deep features.

[0088] The track fusion characteristics output by the fusion network can be expressed by the formula:

[0089] X nf =Fuse[α·V1,(1-α)·V2];

[0090] Among them, V1 represents the first feature matrix output by the feature extraction sub-network of the fusion network, V2 represents the second feature matrix output by the feature extraction sub-network of the fusion network, Fuse(·) represents the fusion unit, X nf Represents the track fusion features output by the fusion network.

[0091] In step 104, the track fusion feature is input into the output network, and the output network reconstructs the track fusion feature to obtain a track fusion result of two track data.

[0092] In a specific implementation, after the fusion network completes the output of the track fusion features, the track fusion features can be input into the output network, and the output network reconstructs the track fusion features to obtain the track fusion results of the two track data.

[0093] In one example, the output network consists of multiple stacked Transformer decoders, which perform nonlinear transformations on track fusion features through a feedforward neural network to obtain an output feature matrix. Residual connections and layer normalization are then applied, and finally the track fusion result is generated through a linear layer.

[0094] The track fusion result output by the output network can be expressed by the formula:

[0095] X out =Linear(X′ out );

[0096] X′ out =LayerNorm[FFN(X nf ),X nf ];

[0097] Among them, Linear(·) represents the linear layer, X out Represents the track fusion result output by the output network.

[0098] In one example, the weighted discriminant network, fusion network, and output network together constitute a track intelligent fusion model, which can be deployed and executed on the Kylin v10 operating system.

[0099] In one example, the loss function used by the server when training the track intelligent fusion model is expressed by the following formula:

[0100] loss=L1[Net fuse (X new ,X fuse ),X r ];

[0101] Among them, X r represents the true label, X new represents the track fusion result output by the track intelligent fusion model, X fuse Represents the original track data, Net fuse (·) represents the forward propagation function, L1(·) represents the absolute difference loss function, and loss represents the calculated loss value.

[0102] In this embodiment, a track intelligent fusion method based on cross-attention is proposed, which has the following technical effects.

[0103] First, this application uses the characteristics of error data to infer the stability and reliability of the two-band fused original track data. This method can more accurately evaluate the quality of the track data. By combining the characteristics of the error data with the track data, the accuracy of the track can be quantitatively evaluated, thereby taking into account the credibility of each data source in the track fusion process. This detailed analysis and utilization of errors makes the track fusion result more accurate because it not only considers the location information of the data, but also the uncertainty and reliability of the data.

[0104] Second, this application proposes a dynamic weight allocation mechanism. The weight matrix generated by the weighted discriminant network matches the time series length of the original track data, which means that each track data will be assigned a corresponding weight according to its quality and credibility. By multiplying the features obtained after processing the two track data with their respective weight matrices, the fusion network can better handle data sources of different quality and credibility. This personalized weight allocation strategy is adjusted according to the characteristics and reliability of the error, thereby more accurately reflecting the contribution of each data source to the final track fusion result, effectively improving the fusion accuracy.

[0105] Third, the Transformer model is used in the weighted discriminant network, fusion network, and output network. The Transformer model has significant advantages in processing long time series data such as track data. Because the Transformer model uses a self-attention mechanism, it can process each element in the sequence in parallel, rather than one by one like traditional recurrent neural networks. This mechanism allows the model to simultaneously consider the relationship between any two time points in the sequence when analyzing track data. Therefore, the Transformer model can better understand and predict the global patterns and trends of the track, which is crucial for track prediction tasks that need to consider long-term dependencies and complex interactions. In addition, due to its parallel processing characteristics, the Transformer is also more efficient when processing long sequences, which can significantly reduce computing time and resource consumption.

[0106] The step division of the above various methods is only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0107] Another embodiment of the present application proposes a track intelligent fusion system based on cross attention. The following is a detailed description of the implementation details of the track intelligent fusion system based on cross attention proposed in this embodiment. The following content is only for the convenience of understanding the implementation details and is not necessary for the implementation of this example. Figure 4 This is a structural diagram of a track intelligent fusion system based on cross-attention proposed in this embodiment. The system includes: an error estimation module 200 and a track intelligent fusion model 300. The track intelligent fusion model 300 is composed of a weighted discriminant network 301, a fusion network 302 and an output network 303.

[0108] The error estimation module 200 is used to perform linear fitting on the two track data to be fused through a fitting function to obtain a track approximation result, and then subtract the two track data from the track approximation result to obtain two sets of error estimation values.

[0109] The weighted discriminant network 301 is used to infer the stability and reliability of the two tracks based on the two sets of error estimates, and obtain a weight matrix that matches the time series length of the two track data.

[0110] The fusion network 302 is used to extract features from the two track data to obtain feature matrices of the two track data, and perform weighted fusion based on the weight matrix and the feature matrices of the two track data to obtain track fusion features.

[0111] The output network 303 is used to reconstruct the track fusion feature to obtain the track fusion result of the two track data.

[0112] It is worth mentioning that all modules involved in this embodiment are logical modules. In actual applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, to highlight the innovation of this application, this embodiment does not include units that are not closely related to solving the technical problem proposed by this application. However, this does not mean that other units do not exist in this embodiment.

[0113] It is not difficult to find that this embodiment is a system embodiment corresponding to the above-mentioned method embodiment, and this embodiment can be implemented in conjunction with the above-mentioned method embodiment. The relevant technical details and technical effects mentioned in the above-mentioned embodiments are still valid in this embodiment, and to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above-mentioned embodiments.

[0114] Another embodiment of the present application deploys an electronic device, the specific structure of which is as follows Figure 5As shown, it includes: at least one processor 401; and a memory 402 communicatively connected to the at least one processor 401; wherein the memory 402 stores instructions that can be executed by the at least one processor 401, and the instructions are executed by the at least one processor 401 to enable the at least one processor 401 to execute a track intelligent fusion method based on cross-attention as described in the above embodiments.

[0115] The memory and processor are connected via a bus, which can include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors and memories. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and will not be described further in this article. The bus interface is responsible for providing an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. Data processed by the processor is transmitted via an antenna on a wireless medium. Furthermore, the antenna also receives data and transmits it to the processor.

[0116] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0117] Those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard drive, a ROM (Read-Only Memory), a RAM (Random Access Memory), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0118] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A track intelligent fusion method based on cross attention, characterized in that: include: The two track data to be fused are linearly fitted using a fitting function to obtain a track approximation result. The two track data are then subtracted from the track approximation result to obtain two sets of error estimates. The two sets of error estimates are input into a weighted discriminant network, which infers the stability and reliability of the two tracks based on the two sets of error estimates, and obtains a weight matrix that matches the time series length of the two track data. The two track data and the weight matrix are input into the fusion network, and the fusion network extracts the features of the two track data to obtain the feature matrix of the two track data, and performs weighted fusion based on the weight matrix and the feature matrix of the two track data to obtain the track fusion feature; The track fusion features are input into the output network, and the output network reconstructs the track fusion features to obtain the track fusion results of the two track data; The weighted discriminant network consists of a normalization unit, a feature extraction subnetwork, and a weighted decision subnetwork; The normalization unit is used to perform normalization processing on the first error estimation value and the second error estimation value to obtain a normalized first error estimation value sequence and a normalized second error estimation value sequence; The feature extraction subnetwork consists of multiple stacked Transformer encoders. The Transformer encoder converts the first error estimate value sequence and the second error estimate value sequence into the first embedding vector and the second embedding vector through the embedding layer. Then, position encoding information is added to the first embedding vector and the second embedding vector to obtain the position-encoded first embedding vector and the position-encoded second embedding vector. After multiplying them by the corresponding feature extraction weights, the first intermediate matrix and the second intermediate matrix are obtained. Finally, a nonlinear transformation is performed through a feedforward neural network, and after residual connection and layer normalization processing, the first feature matrix and the second feature matrix are obtained. The weight decision subnetwork consists of a fully connected layer, which is used to map the first feature matrix and the second feature matrix into the weight space to integrate the features extracted by the convolutional layers of different scales, form a comprehensive judgment on the weights of the first error estimate and the second error estimate, and finally output a weight matrix that matches the time series length of the two track data.

2. The track intelligent fusion method based on cross attention according to claim 1 is characterized in that: The two track data to be fused are recorded as the first track data and the second track data respectively. The two track data to be fused are linearly fitted through the fitting function to obtain the track approximation result. Then, the two track data are subtracted from the track approximation result to obtain two sets of error estimates, including: The first track data and the second track data are linearly fitted using the least squares linear fitting function to obtain the track approximation result. The track approximation result is expressed by the formula: ； in, Indicates the first track data, Indicates the second track data, represents the least squares linear fitting function, Indicates the track approximation result; The first track data and the second track data are respectively subtracted from the track approximation result to obtain a first error estimate and a second error estimate. The first error estimate and the second error estimate are expressed by the formula: ； ； in, represents the first error estimate, represents the second error estimate.

3. The track intelligent fusion method based on cross attention according to claim 1 is characterized in that: The weight matrix output by the weighted discriminant network matches the time series length of the two track data and is expressed by the formula: ； ， ； ， ； ， ； ， ； ， ； in, represents the first error estimate sequence, represents the second error estimate sequence, represents normalization processing, represents the first embedding vector, represents the second embedding vector, represents the embedding layer, represents the positional encoding of the first embedding vector, represents the positional encoding of the second embedding vector, represents the first embedding vector after position encoding, represents the second embedding vector after position encoding, and Respectively represent the feature extraction weights corresponding to the first embedding vector after position encoding and the second embedding vector after position encoding, represents the first intermediate matrix, represents the second intermediate matrix, represents a feedforward neural network, Representation layer normalization processing, represents the first characteristic matrix, represents the second characteristic matrix, represents the fully connected layer, Represents the weight matrix output by the weighted discriminant network that matches the time series length of the two track data.

4. The track intelligent fusion method based on cross attention according to claim 3 is characterized in that: The fusion network consists of a normalization unit, a feature extraction subnetwork, and a fusion unit. The normalization unit and feature extraction subnetwork in the fusion network can be the same as or different from the normalization unit and feature extraction subnetwork in the weighted discriminant network. The fusion unit is used to combine the first feature matrix and the second feature matrix output by the feature extraction sub-network of the fusion network with and Multiply them together, then use 1D multi-scale convolution technology to perform feature fusion, and finally output the track fusion feature.

5. The track intelligent fusion method based on cross attention according to claim 4 is characterized in that: The track fusion feature output by the fusion network is expressed by the formula: ； in, Represents the first feature matrix output by the feature extraction sub-network of the fusion network, Represents the second feature matrix output by the feature extraction sub-network of the fusion network, represents the fusion unit, Represents the track fusion features output by the fusion network.

6. The track intelligent fusion method based on cross attention according to claim 5 is characterized in that: The output network consists of multiple stacked Transformer decoders. The Transformer decoder performs nonlinear transformation on the track fusion features through a feedforward neural network to obtain the output feature matrix, then applies residual connection and layer normalization processing, and finally generates the track fusion result through a linear layer.

7. The track intelligent fusion method based on cross attention according to claim 6 is characterized in that: The track fusion result output by the output network is expressed by the formula: ； ； in, represents a linear layer, Represents the track fusion result output by the output network.

8. A track intelligent fusion method based on cross attention according to any one of claims 1 to 7, characterized in that: The weighted discriminant network, fusion network, and output network together constitute the track intelligent fusion model. The loss function used in training the track intelligent fusion model is expressed by the following formula: ； in, represents the true label, Indicates the track fusion result output by the track intelligent fusion model. Represents the original track data, represents the forward propagation function, represents the absolute difference loss function, Represents the calculated loss value.

9. A track intelligent fusion system based on cross attention, characterized in that: include: Error estimation module and track intelligent fusion model, the track intelligent fusion model consists of weighted discriminant network, fusion network and output network; The error estimation module is used to perform linear fitting on the two track data to be fused through the fitting function to obtain the track approximation result, and then subtract the two track data from the track approximation result to obtain two sets of error estimation values; A weighted discriminant network is used to infer the stability and reliability of the two tracks based on the two sets of error estimates, and obtain a weight matrix that matches the time series length of the two track data; A fusion network is used to extract features from the two track data to obtain feature matrices of the two track data, and perform weighted fusion based on the weight matrix and the feature matrices of the two track data to obtain track fusion features; The output network is used to reconstruct the track fusion features and obtain the track fusion results of the two track data; The weighted discriminant network consists of a normalization unit, a feature extraction subnetwork, and a weighted decision subnetwork; The normalization unit is used to perform normalization processing on the first error estimation value and the second error estimation value to obtain a normalized first error estimation value sequence and a normalized second error estimation value sequence; The feature extraction subnetwork consists of multiple stacked Transformer encoders. The Transformer encoder converts the first error estimate value sequence and the second error estimate value sequence into the first embedding vector and the second embedding vector through the embedding layer. Then, position encoding information is added to the first embedding vector and the second embedding vector to obtain the position-encoded first embedding vector and the position-encoded second embedding vector. After multiplying them by the corresponding feature extraction weights, the first intermediate matrix and the second intermediate matrix are obtained. Finally, a nonlinear transformation is performed through a feedforward neural network, and after residual connection and layer normalization processing, the first feature matrix and the second feature matrix are obtained. The weight decision subnetwork consists of a fully connected layer, which is used to map the first feature matrix and the second feature matrix into the weight space to integrate the features extracted by the convolutional layers of different scales, form a comprehensive judgment on the weights of the first error estimate and the second error estimate, and finally output a weight matrix that matches the time series length of the two track data.

Images