A vector track-based association method and device and readable storage medium

Abstract

The application discloses a kind of based on vector track association method, device and readable storage medium, belong to target identification field;The association method proposed in the application is coupled when processing sensor data, target space position relationship and the high-dimensional feature of depth based on sensor data extraction is strongly coupled.Form target vector feature point track, association is carried out based on vector feature point track, form vector track, finally based on vector point track and vector point track, vector point track and vector track association, greatly improve the associability under various complex conditions, improve situation awareness capability.

Description

Technical Field

[0001] This invention belongs to the field of target recognition and mainly involves methods such as feature extraction based on deep learning, association based on spatial location, and high-dimensional vector distance measurement. Specifically, it relates to an association method, device, and readable storage medium based on vector tracks. Background Technology

[0002] The statements in this section are provided only as background information in connection with this disclosure and may not constitute prior art.

[0003] Current track / point association methods are based on target spatial location. However, limitations such as sensor position perception accuracy and high-speed target maneuvering make location-based association methods prone to failing to associate with targets or incorrectly associating with targets. Furthermore, many passive sensors are constrained by whether the target radiates signals, meaning observation points may be spatially discontinuous, making track or point association impossible and severely restricting the platform's situational awareness generation capabilities.

[0004] While acquiring the spatial location of the target, various types of sensors on the battlefield can also acquire the target's characteristic data on a certain observation surface. These characteristic data themselves have time invariance or slowly changing characteristics, and through similarity comparison, they have the ability to be correlated across time and space.

[0005] Therefore, this invention proposes a method, apparatus, and readable storage medium for associating vector tracks. During sensor data processing, the spatial positional relationships of the target are strongly coupled with high-dimensional features extracted from the sensor data. This forms target vector feature points, which are then associated to create vector tracks. Finally, the associativity between vector points and between vector tracks and vector tracks significantly improves associativity under various complex conditions, enhancing situational awareness capabilities. Summary of the Invention

[0006] The purpose of this invention is to address the problems of existing target association technologies, which are based solely on spatial location association and are limited by sensor and filtering algorithm performance, easily leading to problems such as failure to associate, incorrect association, and ambiguous association. Furthermore, these technologies suffer from poor association performance in densely packed targets and spatiotemporally discontinuous observation data, making it impossible to associate and form tracks, resulting in poor awareness of stealth targets. This invention provides a vector track-based association method, device, and readable storage medium. This method extracts high-dimensional vector features from sensor-perceived data through deep learning and strongly couples them with target spatial location data to form vector point data. Based on this vector point data, a target track association method is proposed. This method combines the characteristics of spatial location and high-dimensional vector features to solve association problems in dense tracks, discontinuous tracks, and isolated points, effectively improving association performance in various complex situations and enhancing situational awareness.

[0007] The technical solution of the present invention is as follows:

[0008] A correlation method based on vector tracks includes:

[0009] Offline category vector extraction model training steps:

[0010] Step S1: Obtain raw data from various sensors, preprocess and transform the raw data, and construct a training dataset;

[0011] Step S2: For the purpose of classification or clustering, a deep network is used to train a category feature vector extraction model to obtain a unique category vector extraction model for each type of sensor data.

[0012] Online processing steps:

[0013] Step S3: Acquire target location data and raw sensor data in real time from the sensor end, extract the corresponding category vector model, extract category vector data from the raw data, and combine the target location data and category vector data to form vector points;

[0014] Step S4: Associate the obtained vector points with the vector points or vector tracks in the vector database.

[0015] Furthermore, the sensor includes a primary radar, an ESM, an optical sensor, and a secondary radar; the sensor's raw data includes signal data and image data.

[0016] Furthermore, the preprocessing includes data cropping and noise reduction; the feature transformation includes time-frequency domain feature transformation of the signal data.

[0017] Furthermore, the category vector extraction model is constructed using a feature extraction backbone network, including a residual network and a Transformer network.

[0018] Further, step S4 includes:

[0019] Step S41: Based on the position data in the vector points, perform a coarse association with the key position data of the vector tracks in the vector database. If it is associated with a unique track, update the vector track; if there is ambiguity in the association, use the category vector to associate the vector tracks with ambiguity; if it is not associated with any track, associate it with isolated points in the vector database and use the category vector to associate the tracks.

[0020] Step S42: If a vector point trace, such as a polysemous track, exists within the association threshold when associated based on category vectors, take the track with the closest vector distance as the associated track and update the track. If there is no track within the association threshold, associate it with isolated points or fragmented tracks in the vector database, using category vectors for association.

[0021] Step S43: If the vector point is associated with a broken batch track, update the track; if it is not associated with any broken batch track, continue to associate it with the isolated point.

[0022] Step S44: If the vector point is associated with an isolated point, a new track is generated and assigned a corresponding track number. If it is not associated with any isolated point, the vector point is stored as an isolated point in the vector database.

[0023] Furthermore, the category vector association uses a high-dimensional vector distance metric for similarity comparison and sets a distance similarity threshold.

[0024] Furthermore, updating the track refers to assigning a corresponding track number to the vector point and storing it in the vector database.

[0025] Furthermore, a discontinuing track is a track that has no new point data within a certain period of time. Based on certain criteria, it is determined to be a discontinuing track, indicating that the target has been lost.

[0026] The present invention also proposes a vector track-based association device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the vector track-based association method described above.

[0027] The present invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the vector track-based association method described above.

[0028] Compared with existing technologies, the advantages of this invention are:

[0029] This invention extracts high-dimensional vector features from sensor data using deep learning and strongly couples them with target spatial location data to form vector point data. Based on the vector point data, a target track association method is proposed. This method combines the characteristics of spatial location and high-dimensional vector features to solve the association problem in cases of dense tracks, discontinuous tracks, and isolated points, effectively improving the associativity in various complex situations and enhancing situational awareness. Attached Figure Description

[0030] Figure 1This is a general framework diagram of a vector-based correlation method;

[0031] Figure 2 Extract the model architecture diagram from the category vectors;

[0032] Figure 3 This is a schematic diagram of the vector dot generation process;

[0033] Figure 4 This is a schematic diagram of a vector-based point association method;

[0034] Figure 5 This is a schematic diagram illustrating the correlation under conditions of dense flight paths;

[0035] Figure 6 This is a schematic diagram illustrating the correlation under conditions of discontinuous flight paths and isolated points. Detailed Implementation

[0036] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0037] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0038] Example 1

[0039] Please see Figure 1 A method for association based on vector tracks, comprising:

[0040] Offline category vector extraction model training steps:

[0041] Step S1: Obtain raw data from various sensors, preprocess and transform the raw data, and construct a training dataset;

[0042] Step S2: For the purpose of classification or clustering, a deep network is used to train a category feature vector extraction model to obtain a unique category vector extraction model for each type of sensor data.

[0043] Online processing steps:

[0044] Step S3: Acquire target location data and raw sensor data in real time from the sensor end, extract the corresponding category vector model, extract category vector data from the raw data, and combine the target location data and category vector data to form vector points;

[0045] Step S4: Associate the obtained vector points with the vector points or vector tracks in the vector database.

[0046] In this embodiment, it should be noted that the sensors include commonly used battlefield awareness sensors, such as primary radar, ESM (including radar reconnaissance and communication reconnaissance), optical sensors, and secondary radar.

[0047] Specifically, in step S1, the raw data from various sensors can be acquired through actual collection or generated through simulation. Basic preprocessing of the raw data is performed, including data cropping and noise reduction. Necessary feature transformations are also performed, such as time-frequency domain feature transformation of signal data. It should be noted that the feature transformation can adopt traditional feature extraction methods, such as the Sobel operator and HOG algorithm for image data, and FFT transform and wavelet transform for signal data.

[0048] In this embodiment, it should be noted that the category vector is a high-dimensional feature vector. In order to reduce the subsequent computational overhead, it is generally recommended that the dimension not exceed 100.

[0049] In this embodiment, it should be noted that the category vector extraction model is constructed from mainstream feature extraction backbone networks, such as residual networks and Transformer networks, through pre-training, which can achieve good high-dimensional deep feature representation capabilities.

[0050] In this embodiment, it should be noted that the target location data and category vector data in step S3 are concatenated to form a high-dimensional vector.

[0051] In this embodiment, it should be noted that step S4 includes:

[0052] Step S41: Based on the position data in the vector points, perform a coarse association with the key position data of the vector tracks in the vector database. If it is associated with a unique track, update the vector track; if there is ambiguity in the association, use the category vector to associate the vector tracks with ambiguity; if it is not associated with any track, associate it with isolated points in the vector database and use the category vector to associate the tracks.

[0053] Step S42: If a vector point trace, such as a polysemous track, exists within the association threshold when associated based on category vectors, take the track with the closest vector distance as the associated track and update the track. If there is no track within the association threshold, associate it with isolated points or fragmented tracks in the vector database, using category vectors for association.

[0054] Step S43: If the vector point is associated with a broken batch track, update the track; if it is not associated with any broken batch track, continue to associate it with the isolated point.

[0055] Step S44: If the vector point is associated with an isolated point, a new track is generated and assigned a corresponding track number. If it is not associated with any isolated point, the vector point is stored as an isolated point in the vector database.

[0056] In this embodiment, it should be noted that the vector database refers to the database used for unstructured data management in the current main process.

[0057] In this embodiment, it should be noted that the location data association method in the vector point trace refers to the current mainstream location association method, such as the nearest neighbor association method.

[0058] In this embodiment, it should be noted that association ambiguity refers to the association of a point track with multiple tracks based on evaluation criteria such as distance or probability.

[0059] In this embodiment, it should be noted that the category vector association is performed by using a high-dimensional vector distance metric for similarity comparison, and a distance similarity threshold is set.

[0060] In this embodiment, it should be noted that updating the track means assigning a corresponding track number to the vector point and storing it in the vector database.

[0061] In this embodiment, it should be noted that a broken track is a track that has no new point data within a certain period of time. Based on certain criteria, it is determined to be a broken track, indicating that the target has been lost.

[0062] In this embodiment, it should be noted that isolated points refer to independent point data that have not formed a flight path.

[0063] In this embodiment, it should be noted that association ambiguity refers to the association of a point track with multiple tracks based on evaluation criteria such as distance or probability.

[0064] This embodiment also proposes a vector track-based association device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the vector track-based association method described above. Preferably, the computer program can run on a terminal device, such as a personal computer.

[0065] This embodiment also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the vector track-based association method described above. However, the apparatus of the present invention is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0066] A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0067] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0068] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0069] Example 2

[0070] This embodiment uses ESM sensor data as an example to explain in detail the processing procedure of the vector track-based association method proposed in Embodiment 1. The processing procedure for other sensors is similar. The detailed steps are as follows:

[0071] 1. ESM detects electromagnetic signals radiated by targets, including radar signals and communication signals. The first step is to preprocess and transform the signal data. This includes the following steps:

[0072] (1) Signal segment trimming

[0073] Signal segment pruning involves determining the signal segment from which the category vector to be extracted is located. Complete signals are generally quite long. To improve the training speed of the model, only the key signal segments need to be extracted. Moreover, if the signal encoding contains variable information, it will change due to changes in the information, leading to unstable learning features or "inconsistency" in the signal.

[0074] Therefore, the length of the extracted signal segment should be moderate to ensure sufficient features for feature extraction. For example, the extracted signal segment should be the frame pulse of the signal and should not carry specific encoded information, such as the signal synchronization header.

[0075] (2) Time-frequency domain transformation of signal

[0076] Applying the Discrete Fourier Transform (FFT) to the signal segment, and using the FFT function in MATLAB to process it, we can transform x(n) into X(n) using the Discrete Fourier Transform:

[0077] X(n) = abs(fft(x(n)))

[0078] Here, abs represents taking the absolute value.

[0079] (3) Data normalization

[0080] The data after Fourier transform is normalized and output. The normalization function is as follows:

[0081] X(i)=X(i) / max(X(1),X(2),...,X(n))i=1,2,...,n

[0082] 2. Training the Category Vector Extraction Model. First, a complete classification network is constructed, including a backbone network and a softmax multi-classification model. The backbone network is the category vector feature extraction model, and the softmax multi-classification model performs classification, primarily used to evaluate the quality of the feature vectors extracted by the backbone network, thereby training the backbone network. This method recommends using the Transformer model to extract category vectors from all sensor data; the category token is the aforementioned category vector. For ESM detection data, the processing procedure is as follows: Figure 2 As shown.

[0083] (1) First, the proposed signal feature data is serialized. Assume the input data x∈R M Where M refers to the feature data dimension, and the data is divided into smaller blocks according to the time dimension, with each block having a dimension of S. One-dimensional data segmentation involves dividing the entire one-dimensional data into n = M / S blocks using a one-dimensional convolution with a kernel of S, and then transforming it into serialized data using dimensionality transformation.

[0084] x∈R M →x′∈R (M / S)×S →x″∈R n×D

[0085] (2) The serialized data is processed by a Transformer encoding network to extract class vectors. During training, the class vectors are processed by a softmax multi-classification model to obtain the classification loss, thereby optimizing the encoding network. In practical applications, the trained encoding network is directly used to extract class vectors from the sensor data.

[0086] 3. Vector point generation. Target position data x, estimated from current observations, is obtained from the ESM sensor. p ∈R q The dimension is q, which is generally two-dimensional or three-dimensional data, such as distance, azimuth, elevation, or converted longitude, latitude, and altitude. A category vector x is extracted from the sensor signal data. c ∈R s These two types of data are then concatenated. This generates semantically clear vector point data. The process is as follows: Figure 3 As shown.

[0087] 4. Association of vector waypoints. The process is as follows: Figure 4 As shown. First, based on the location data, vector points are associated with vector tracks in the database. If ambiguity occurs in the association or no track is associated, category vectors are used for association. The specific process is as follows.

[0088] (1) Correlation between vector points and ambiguous tracks

[0089] Suppose that the observation vector is obtained at time k, and its class vector is A. k Based on location association, there are a total of l ambiguous vector tracks {T1,T2,...,T...} to be associated. l}, the latest n category vectors for each track are respectively;

[0090]

[0091] Then the category vector A k With track T i The "distance" can be expressed as:

[0092]

[0093] Where i = 1, 2, ..., l, α j Represents time series weights, satisfying d(A,B) represents the distance metric between two vectors.

[0094] For α j The settings need to be considered specifically, as data obtained from different modalities or different sensors may differ. For example, in the detected electromagnetic signals, the category vector characterizes the "fingerprint" features of the radiation source and is almost invariant, then α... j We can consider them all to be the same, that is... For target image data obtained from image sensors and other sources, as well as one-dimensional radar image data, the category vector may contain information about the target's instantaneous attitude changes. Therefore, in α... j The settings should satisfy α1>α2>...>α n If it can take the value

[0095] Regarding the selection of d(A,B), two main methods are considered: Euclidean distance and cosine similarity. Assuming two vectors A and B of length r, the Euclidean distance can be expressed as:

[0096]

[0097] Cosine similarity is expressed as:

[0098]

[0099] For ambiguous track association, the association result is the vector track with the smallest "distance", that is:

[0100] Successfully associated track = i,

[0101] (2) Correlation between vector points and isolated points and broken tracks

[0102] When a vector point is not associated with any real-time vector track, it is associated with an isolated point or a discontinuous batch of tracks in the vector database. Since the current vector point is not continuous with the isolated point or discontinuous batch of tracks in both time and space, an association threshold σ is set when associating based on category vectors. This threshold needs to be statistically obtained based on the actual situation.

[0103] Suppose r has an isolated point and a break in the chain of track points (taking the latest point in the break in the chain), denoted as {P1, P2, ..., P...}. r}, then the category vector A k With isolated point P i The "distance" can be expressed as:

[0104] D(A k ,P i )=d(A k ,P i )

[0105] Where d(A,B) is the distance metric method described above.

[0106] A successfully associated isolated point is the one with the shortest distance and a distance less than the association threshold. This can be represented as:

[0107]

[0108] When a broken track is successfully associated, the track is updated. When an isolated point is associated, a new track is generated and assigned a track number. When an association fails, the track is stored as an isolated point in the vector database.

[0109] like Figure 5 As shown, under dense flight path conditions, location-based association is prone to ambiguity when associating points with flight paths, leading to association errors. This proposed solution can significantly improve the association accuracy. Experiments demonstrate that, for actual ADS-B signal data acquired through reconnaissance, when there is ambiguity in the association of two and three flight paths, this solution achieves an accuracy rate exceeding 96%.

[0110] like Figure 6As shown, in cases of discontinuous track batches, location-based correlation cannot re-correlate subsequent isolated observations, leading to the formation of separate track batches or the inability to form stable tracks. This is extremely detrimental to target behavior recognition and prediction. The technology proposed in this solution can successfully correlate isolated observations with discontinuous track batches, forming tracks that are not continuous in time, greatly improving the ability to recognize such targets. Experiments show that, for actual ADS-B signal data acquired through reconnaissance, the correlation accuracy between isolated observation points and discontinuous track batches can reach over 88% under conditions of 20 discontinuous track batches.

[0111] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

[0112] This background section is provided to generally present the context of the invention. The work of the currently named inventors, the work to the extent described in this background section, and aspects of this section that did not constitute prior art at the time of application are neither expressly nor impliedly acknowledged as prior art to the invention.

Claims

1. A correlation method based on vector tracks, characterized in that, include: Offline category vector extraction model training steps: Step S1: Obtain raw data from various sensors, preprocess and transform the raw data, and construct a training dataset; Step S2: For the purpose of classification or clustering, a deep network is used to train a category feature vector extraction model to obtain a unique category vector extraction model for each type of sensor data. Online processing steps: Step S3: Acquire target location data and raw sensor data in real time from the sensor end, extract the corresponding category vector model, extract category vector data from the raw data, and combine the target location data and category vector data to form vector points; Step S4: Associate the obtained vector points with the vector points or vector tracks in the vector database; Step S4 includes: Step S41: Based on the position data in the vector points, perform a coarse association with the key position data of the vector tracks in the vector database. If it is associated with a unique track, update the vector track; if there is ambiguity in the association, use the category vector to associate the vector tracks with ambiguity; if it is not associated with any track, associate it with isolated points in the vector database and use the category vector to associate. Step S42: If a vector point trace, such as a polysemous track, exists within the association threshold when associated based on category vectors, take the track with the closest vector distance as the associated track and update the track. If there is no track within the association threshold, associate it with isolated points or fragmented tracks in the vector database, using category vectors for association. Step S43: If the vector point is associated with a broken batch track, update the track; if it is not associated with any broken batch track, continue to associate it with the isolated point. Step S44: If the vector point is associated with an isolated point, a new track is generated and assigned a corresponding track number. If it is not associated with any isolated point, the vector point is stored as an isolated point in the vector database. The association using category vectors in step S4 includes: Suppose The observation vector is obtained at each time step, and its category vector is... The latest n category vectors of the vector tracks to be associated are respectively Then the category vector With track The distance is expressed as: in , Represents time-series weights, satisfying , Represents the distance metric between two vectors; For the detected electromagnetic signals, the category vector represents the fingerprint characteristics of the radiation source and has invariance. The settings meet the requirements. For target image data and radar one-dimensional image data obtained by image sensors, which contain information on the instantaneous attitude changes of the target, then in The settings meet .

2. The association method based on vector tracks according to claim 1, characterized in that, The sensors include primary radar, ESM, optical sensors, and secondary radar; the raw sensor data includes signal data and image data.

3. The association method based on vector tracks according to claim 1, characterized in that, The preprocessing includes data trimming and noise reduction; Feature transformation includes time-frequency domain feature transformation of signal data.

4. The association method based on vector tracks according to claim 1, characterized in that, The category vector extraction model is constructed using a feature extraction backbone network, which includes a residual network and a Transformer network.

5. The association method based on vector tracks according to claim 4, characterized in that, Category vector association employs a high-dimensional vector distance metric for similarity comparison and sets a distance similarity threshold.

6. The association method based on vector tracks according to claim 1, characterized in that, Updating a track means assigning a corresponding track number to a vector point and storing it in a vector database.

7. The association method based on vector tracks according to claim 1, characterized in that, A discontinuing track is a track that has no new data points within a certain period of time. Based on certain criteria, it is determined to be a discontinuing track, indicating that the target has been lost.

8. A correlation device based on vector track, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the steps of a vector track-based association method as described in any one of claims 1-7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of a vector track-based association method as described in any one of claims 1-7.

Images