Correlation method and device based on vector track and readable storage medium

Abstract

The invention discloses an association method and device based on a vector track and a readable storage medium, and belongs to the field of target recognition. According to the association method provided by the invention, when the sensor data is processed, the spatial position relationship of the target and the deep high-dimensional features extracted based on the sensor data are strongly coupled. According to the method, target vector feature plots are formed, association is carried out based on the vector feature plots, a vector track is formed, and finally, association under various complex conditions is greatly improved based on association between the vector plots and between the vector plots and the vector track, and the situation awareness capability is improved.

Description

Technical Field

[0001] The present invention belongs to the field of target recognition, and mainly involves methods such as feature extraction based on deep learning, association based on spatial position, and high-dimensional vector distance measurement. Specifically, it relates to an association method, device and readable storage medium based on vector tracks. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.

[0003] Current track / point-track association is based on the target's spatial position. This is limited by factors such as sensor position perception accuracy and high-speed target maneuvers. This makes position-based association methods prone to failing to associate with the target or incorrectly associating with the target. Furthermore, many passive sensors are constrained by whether the target radiates signals, so observation points may be spatially discontinuous, making track or point-track association impossible. This significantly restricts the platform's situational awareness capabilities.

[0004] While obtaining the spatial position of the target, various types of battlefield sensors can also obtain the target's characteristic data on a certain observation surface. These characteristic data themselves have time-invariance or slow-changing characteristics, and through similarity comparison, they have the ability to correlate across time and space.

[0005] Therefore, the present invention proposes a vector track-based association method, device, and readable storage medium. When processing sensor data, the spatial positional relationship of the target is strongly coupled with the deep, high-dimensional features extracted from the sensor data. This generates target vector feature points, which are then associated to form vector tracks. Finally, by associating vector points with vector points and vector tracks with vector tracks, the method greatly improves the associatability in various complex situations and enhances situational awareness. Summary of the Invention

[0006] The purpose of the present invention is to provide a vector track-based association method, device and readable storage medium for the existing target association technology, which is based only on spatial position association and is limited by sensor performance and filtering algorithm performance, and is prone to problems such as association failure, erroneous association, and association ambiguity. When the targets are dense, the association performance is poor, and the observation data is discontinuous in time and space, and cannot be associated and formed into tracks, resulting in poor recognition of stealth targets. The method extracts high-dimensional vector features of sensor perception data through deep learning, and strongly couples it with the target spatial position data to form vector point track data; based on the vector point track data, a set of target track association methods is proposed, which combines the characteristics of spatial position and high-dimensional vector features to solve the association problems in cases of dense tracks, discontinuous tracks, isolated point tracks, etc., effectively improves the associatability in various complex situations, and enhances situational awareness capabilities.

[0007] The technical solutions of the present invention are as follows:

[0008] A vector track-based association method, comprising:

[0009] Offline category vector extraction model training steps:

[0010] Step S1: Obtain raw data from various sensors, preprocess and transform the raw data into features, and construct a training data set;

[0011] Step S2: For the purpose of classification or clustering, a deep network is used to train a category feature vector extraction model, and a unique category vector extraction model is obtained for each type of sensor data;

[0012] Online processing steps:

[0013] Step S3: acquiring target position data and sensor raw data from the sensor in real time, extracting the category vector data from the raw data using the corresponding category vector extraction model, and combining the target position data with the category vector data to form a vector trace;

[0014] Step S4: Associating the obtained vector point track with the vector point track or vector track in the vector database.

[0015] Furthermore, the sensors include primary radar, ESM, optical sensor and secondary radar; the sensor raw data includes signal data and image data.

[0016] Furthermore, the preprocessing includes data clipping and denoising; and the feature conversion includes time-frequency domain feature conversion of the signal data.

[0017] Furthermore, the category vector extraction model is composed of a feature extraction backbone network, including: a residual network and a Transformer network.

[0018] Furthermore, the step S4 includes:

[0019] Step S41: Based on the position data in the vector track, a rough association is performed with the key position data of the vector track in the vector database. If it is associated with a unique track, the vector track is updated; if there is an association ambiguity, the classification vector is used to associate the ambiguous vector track; if it is not associated with any track, it is associated with the isolated point track in the vector database and the classification vector is used to associate it;

[0020] Step S42: If a vector point track, such as an ambiguous track, exists within the association threshold when being associated based on the category vector, the track with the closest vector distance is taken as the associated track and updated. If there is no track within the association threshold, it is associated with an isolated point track or a discontinuous batch of tracks in the vector database and associated using the category vector;

[0021] Step S43: If the vector point track is associated with the broken batch track, the track is updated; if it is not associated with any broken batch track, it continues to be associated with the isolated point track;

[0022] Step S44: If the vector point track is associated with an isolated point track, a new track is generated and assigned a corresponding track number. If it is not associated with any isolated point, the vector point track is stored in the vector database as an isolated point track.

[0023] Furthermore, the category vectors are associated by using a high-dimensional vector distance measurement method to perform similarity comparison and setting a distance similarity threshold.

[0024] Furthermore, updating the track means assigning the vector point track to a corresponding track number and storing it in the vector database.

[0025] Furthermore, a discontinuous track is a track that has no new point track data within a certain period of time. Based on certain criteria, it is determined to be a discontinuous track, indicating that the target has been lost.

[0026] The present invention also proposes a vector track-based association device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the vector track-based association method described above are implemented.

[0027] The present invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the computer program implements the steps of the above-mentioned vector track-based association method.

[0028] Compared with the existing technology, the beneficial effects of the present invention are:

[0029] The present invention extracts high-dimensional vector features of sensor perception data through deep learning and strongly couples it with the target spatial position data to form vector point track data. Based on the vector point track data, a set of target track association methods is proposed. This method combines the characteristics of spatial position and high-dimensional vector features to solve the association problems in cases of dense tracks, discontinuous tracks, isolated point tracks, etc., effectively improving the relevance in various complex situations and enhancing situational awareness capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1This is a general framework diagram of a vector track-based association method;

[0031] Figure 2 Extract model architecture diagram for category vectors;

[0032] Figure 3 This is a schematic diagram of the vector point trace generation process;

[0033] Figure 4 Schematic diagram of the association method based on vector traces;

[0034] Figure 5 It is a schematic diagram of the correlation under dense track conditions;

[0035] Figure 6 It is a schematic diagram of the association under the conditions of discontinuous batch tracks and isolated points. DETAILED DESCRIPTION

[0036] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0037] The features and performance of the present invention are further described in detail below with reference to the embodiments.

[0038] Example 1

[0039] See also Figure 1 , a vector track-based association method, comprising:

[0040] Offline category vector extraction model training steps:

[0041] Step S1: Obtain raw data from various sensors, preprocess and transform the raw data into features, and construct a training data set;

[0042] Step S2: For the purpose of classification or clustering, a deep network is used to train a category feature vector extraction model, and a unique category vector extraction model is obtained for each type of sensor data;

[0043] Online processing steps:

[0044] Step S3: acquiring target position data and sensor raw data from the sensor in real time, extracting the category vector data from the raw data using the corresponding category vector extraction model, and combining the target position data with the category vector data to form a vector trace;

[0045] Step S4: Associating the obtained vector point track with the vector point track or vector track in the vector database.

[0046] In this embodiment, it should be noted that the sensors include commonly used sensors for battlefield perception, such as primary radar, ESM (including radar reconnaissance and communication reconnaissance, etc.), optical sensors and secondary radar, etc.

[0047] Specifically, in step S1, the raw data of various sensors can be actually collected or generated by simulation, and basic preprocessing is performed on the raw data, including data cropping, denoising, etc.; necessary feature conversion, such as time-frequency domain feature conversion of signal data; it should be noted that feature conversion can adopt traditional feature extraction methods, such as the Sobel operator and HOG algorithm of image data, and FFT transform and wavelet transform of signal data.

[0048] In this embodiment, it should be noted that the category vector is a high-dimensional feature vector. In order to reduce subsequent calculation overhead, it is generally recommended that the dimension does not exceed 100 dimensions.

[0049] In this embodiment, it should be noted that the category vector extraction model is composed of mainstream feature extraction backbone networks, such as residual networks, Transformer networks, etc., which are pre-trained and can achieve better high-dimensional deep feature expression capabilities.

[0050] In this embodiment, it should be noted that the target position data and the category vector data in step S3 are concatenated to form a high-dimensional vector.

[0051] In this embodiment, it should be noted that step S4 includes:

[0052] Step S41: Based on the position data in the vector track, a rough association is performed with the key position data of the vector track in the vector database. If it is associated with a unique track, the vector track is updated; if there is an association ambiguity, the classification vector is used to associate the ambiguous vector track; if it is not associated with any track, it is associated with the isolated point track in the vector database and the classification vector is used to associate it;

[0053] Step S42: If a vector point track, such as an ambiguous track, exists within the association threshold when being associated based on the category vector, the track with the closest vector distance is taken as the associated track and updated. If there is no track within the association threshold, it is associated with an isolated point track or a discontinuous batch of tracks in the vector database and associated using the category vector;

[0054] Step S43: If the vector point track is associated with the broken batch track, the track is updated; if it is not associated with any broken batch track, it continues to be associated with the isolated point track;

[0055] Step S44: If the vector point track is associated with an isolated point track, a new track is generated and assigned a corresponding track number. If it is not associated with any isolated point, the vector point track is stored in the vector database as an isolated point track.

[0056] In this embodiment, it should be noted that the vector database refers to the database used for unstructured data management of the current main process.

[0057] In this embodiment, it should be noted that the position data association method in the vector trace refers to the current mainstream position association method, such as the nearest neighbor association method.

[0058] In this embodiment, it should be noted that association ambiguity refers to the association of a point track with multiple tracks based on evaluation criteria such as distance or probability.

[0059] In this embodiment, it should be noted that the category vector association is to use a high-dimensional vector distance measurement method to perform similarity comparison and set a distance similarity threshold.

[0060] In this embodiment, it should be noted that updating the track means assigning a corresponding track number to the vector point track and storing the track number in the vector database.

[0061] In this embodiment, it should be noted that a discontinued track is a track without new point track data within a certain period of time. Based on certain criteria, it is determined that the track is discontinued, indicating that the target has been lost.

[0062] In this embodiment, it should be noted that isolated point traces refer to independent point trace data that do not form a track.

[0063] In this embodiment, it should be noted that association ambiguity refers to the association of a point track with multiple tracks based on evaluation criteria such as distance or probability.

[0064] This embodiment also proposes a vector track-based association device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the vector track-based association method described above are implemented; preferably, the computer program can be executed on a terminal device, such as a personal computer.

[0065] This embodiment also proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the above-mentioned vector track-based association method; however, the device of the present invention is not limited to this. In this document, the readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device or component.

[0066] The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0067] The computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. The data signal propagated may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or component. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0068] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0069] Example 2

[0070] This embodiment uses ESM sensor data as an example to explain in detail the process of the vector track-based association method proposed in Example 1. The process for other sensors is similar. The detailed steps are as follows:

[0071] 1. ESM detects electromagnetic signals radiated by the target, including radar signals, communication signals, etc., and first pre-processes the signal data and converts its features. The steps include:

[0072] (1) Signal segment clipping

[0073] Signal segment clipping is to determine the signal segment of the category vector to be extracted. The complete signal is generally long. In order to improve the model training speed, only the key signal segment needs to be intercepted. Moreover, if the signal encodes variable information, it will change due to changes in information, resulting in unstable learning features or "inconsistency" of the signal.

[0074] Therefore, the intercepted signal segment has a moderate length to ensure that there are sufficient features for feature extraction. For example, the intercepted signal segment is the frame pulse of the signal and does not carry specific coding information, such as the signal synchronization header.

[0075] (2) Signal time-frequency domain transformation

[0076] The signal segment is processed by discrete Fourier transform and the discrete Fourier transform tool FFT function in MATLAB. The discrete Fourier transform of x(n) is converted to X(n), and then:

[0077] X(n)=abs(fft(x(n)))

[0078] Among them, abs means taking the absolute value.

[0079] (3) Data normalization

[0080] The data after Fourier transformation is normalized and output. The normalization processing function is as follows:

[0081] X(i)=X(i) / max(X(1),X(2),...,X(n))i=1,2,...,n

[0082] 2. Train the category vector extraction model. First, build a complete classification network, including the backbone network and the softmax multi-classification model. The backbone network is the category vector feature extraction model, and the softmax multi-classification model implements classification, which is mainly used to evaluate the quality of the feature vectors extracted by the backbone network, so as to train the backbone network. This method recommends using the Transformer model to extract category vectors from all sensor data, and its category token is the category vector. For ESM detection data, the processing process is as follows: Figure 2 shown.

[0083] (1) First, the proposed signal feature data is serialized, assuming that the input data x∈R M , where M refers to the feature data dimension, which is divided into small blocks according to the time dimension, and the block dimension is S. One-dimensional data blocking is performed by a one-dimensional convolution with a convolution kernel of S to divide the entire one-dimensional data into n = M / S blocks, and then the dimension is converted into serialized data.

[0084] x∈R M →x′∈R (M / S)×S →x″∈R n×D

[0085] (2) The serialized data is passed through the Transformer encoding network to extract the category vector. During the training process, the category vector is passed through the softmax multi-classification model to obtain the classification loss, thereby optimizing the encoding network. In actual application, the trained encoding network is directly used to extract the category vector of the sensor data.

[0086] 3. Vector point generation. Obtain the target position data x estimated by the current observation from the ESM sensor p ∈R q , dimension is q, generally two-dimensional or three-dimensional data, such as distance, azimuth, pitch, or converted longitude, latitude and altitude, etc., extract the category vector x from the sensor signal data c ∈R s . Combine these two types of data The process is as follows: Figure 3 shown.

[0087] 4. Association of vector track points. The process is as follows Figure 4 First, based on the position data, the vector point track is associated with the vector track in the database. When the association is ambiguous or no track is associated, the category vector is used for association. The specific process is as follows.

[0088] (1) Association between vector point traces and ambiguous tracks

[0089] Assume that the observation vector is obtained at time k, and its category vector is A k , based on the position association, there are l ambiguous vector tracks to be associated {T1, T2, ..., T l}, the latest n category vectors of each track are;

[0090]

[0091] Then the category vector A k With track T i The "distance" can be expressed as:

[0092]

[0093] where i = 1, 2, ..., l, α j Represents the timing weight, satisfying d(A,B) represents the distance metric between two vectors.

[0094] Targeting α j The setting of α needs to be considered specifically. The data obtained by different modalities or different sensors may be different. For example, in the electromagnetic signal detected, the category vector represents the “fingerprint” feature of the radiation source and is almost invariant. Then α j We can consider them all the same, i.e. For target image data and radar one-dimensional image data obtained by image sensors, their category vectors may contain information about the instantaneous attitude change of the target. j The setting should satisfy α1>α2>...>α n , if it can be taken as

[0095] For the selection of d(A,B), the Euclidean distance measurement method and cosine similarity are mainly considered. Assuming two vectors A and B with length r, the Euclidean distance can be expressed as:

[0096]

[0097] Cosine similarity is expressed as:

[0098]

[0099] For ambiguous track association, the association result is the vector track with the smallest "distance", that is:

[0100] Successfully associated track = i,

[0101] (2) Association between vector points, isolated points, and fragmented tracks

[0102] When a vector point track is not associated with any real-time vector track, it is associated with isolated points or discontinuous tracks in the vector database. Since the current vector point track and the isolated points or discontinuous tracks are not continuous in time and space, the association threshold σ is set when associating based on the category vector. This threshold needs to be obtained based on actual statistics.

[0103] Assume that r has an isolated point and a broken track point (take the latest point of the broken track), which are {P1, P2, ..., P r}, then the category vector A k With isolated point P i The "distance" can be expressed as:

[0104] D(A k ,P i )=d(A k ,P i )

[0105] Where d(A,B) is the distance metric mentioned above.

[0106] The isolated point that is successfully associated is the one with a distance less than the association threshold and the shortest distance.

[0107]

[0108] When the batch track is successfully associated, the track is updated. When the isolated point track is associated, a new track is generated and assigned a track number. If the association is unsuccessful, it is treated as an isolated point track and stored in the vector database.

[0109] like Figure 5 As shown in the figure, under dense track conditions, position-based association is prone to ambiguity when associating point tracks with tracks, leading to association errors. Using this solution significantly improves association accuracy. Experiments show that using this solution for actual ADS-B signal data collected and detected, when there are two or three tracks with ambiguous associations, the accuracy can reach over 96%.

[0110] like Figure 6As shown in the figure, in the case of discontinuous tracks, position-based association cannot achieve re-association of subsequent isolated observations, resulting in the creation of new tracks or the inability to form a stable track, which is extremely detrimental to the recognition and prediction of target behavior. The technology proposed in this solution can successfully associate isolated observations with discontinuous tracks, forming non-continuous time tracks, greatly improving the recognition ability of such targets. Experiments show that for actual ADS-B signal data collected and detected, in the presence of 20 discontinuous tracks, the accuracy rate of associating isolated observation points with discontinuous tracks can reach over 88%.

[0111] The above-described embodiments merely represent specific implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of protection of the present application. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the technical concept of the present application, and all such variations and improvements fall within the scope of protection of the present application.

[0112] This background section is provided to generally present the context of the invention, and the work of the presently named inventors, the work to the extent described in this background section, and aspects of the description in this section that did not constitute prior art at the time of filing are neither explicitly nor implicitly admitted to be prior art to the present invention.

Claims

1. A vector track-based association method, characterized in that: include: Offline category vector extraction model training steps: Step S1: Obtain raw data from various sensors, preprocess and transform the raw data into features, and construct a training data set; Step S2: For the purpose of classification or clustering, a deep network is used to train a category feature vector extraction model, and a unique category vector extraction model is obtained for each type of sensor data; Online processing steps: Step S3: acquiring target position data and sensor raw data from the sensor in real time, extracting the category vector data from the raw data using the corresponding category vector extraction model, and combining the target position data with the category vector data to form a vector trace; Step S4: Associating the obtained vector point track with the vector point track or vector track in the vector database.

2. The vector track-based association method according to claim 1, characterized in that: The sensors include primary radar, ESM, optical sensor and secondary radar; the sensor raw data includes signal data and image data.

3. The vector track-based association method according to claim 1, characterized in that: The preprocessing includes data clipping and denoising; Feature transformation includes time-frequency domain feature transformation of signal data.

4. The vector track-based association method according to claim 1, characterized in that: The category vector extraction model is composed of a feature extraction backbone network, including a residual network and a Transformer network.

5. The vector track-based association method according to claim 1, characterized in that: The step S4 comprises: Step S41: Based on the position data in the vector track, a rough association is performed with the key position data of the vector track in the vector database. If it is associated with a unique track, the vector track is updated; if there is an association ambiguity, the classification vector is used to associate the ambiguous vector track; if it is not associated with any track, it is associated with the isolated point track in the vector database and the classification vector is used to associate it; Step S42: If a vector point track, such as an ambiguous track, exists within the association threshold when being associated based on the category vector, the track with the closest vector distance is taken as the associated track and updated. If there is no track within the association threshold, it is associated with an isolated point track or a discontinuous batch of tracks in the vector database and associated using the category vector; Step S43: If the vector point track is associated with the broken batch track, the track is updated; if it is not associated with any broken batch track, it continues to be associated with the isolated point track; Step S44: If the vector point track is associated with an isolated point track, a new track is generated and assigned a corresponding track number. If it is not associated with any isolated point, the vector point track is stored in the vector database as an isolated point track.

6. The vector track-based association method according to claim 5, characterized in that: Category vector association uses a high-dimensional vector distance measurement method to perform similarity comparison and set a distance similarity threshold.

7. The vector track-based association method according to claim 5, characterized in that: Updating the track means assigning the vector point track to the corresponding track number and storing it in the vector database.

8. The vector track-based association method according to claim 5, characterized in that: A discontinuous track is one in which there is no new track data within a certain period of time. Based on certain criteria, it is determined to be a discontinuous track, indicating that the target has been lost.

9. A vector track-based association device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the vector track-based association method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the vector track-based association method according to any one of claims 1 to 8 are implemented.

Images