Data-driven lattice neural network multi-source heterogeneous sensor trajectory data association method

Abstract

This invention relates to a data-driven method for associating trajectory data from multiple heterogeneous sensors using a lattice neural network, belonging to the field of electronic digital data processing technology. The method includes the following steps: constructing a fully connected feedforward neural network to extract features from trajectory data with known identification codes; inputting the features into a constructed fuzzy lattice classifier to learn parameters; using the learned model to predict trajectory data with unknown identification codes to label them with pseudo-labels; and then associating data with the same pseudo-label at the same time. This invention combines traditional neural networks with a fuzzy lattice classifier to achieve efficient point-to-point association of multi-sensor trajectory data, while simultaneously filling the gap in point-to-point association of multi-sensor trajectory data.

Description

Technical Field

[0001] This invention belongs to the field of electronic digital data processing technology, specifically relating to a data-driven method for associating trajectory data of multi-source heterogeneous sensors using lattice neural networks. Background Technology

[0002] Data association aims to determine the correspondence between targets and measurement information, and is an upstream step in track fusion. In multi-sensor fusion systems, when multiple targets exist, the system struggles to distinguish whether measurement data originates from real targets, other targets, or false alarms (clutter). Therefore, it is necessary to employ data association algorithms to handle the various uncertainties inherent in multi-sensor measurement data.

[0003] Currently, commonly used data association methods mainly fall into three categories. Fuzzy theory-based data association: The design of membership functions often relies on expert experience and lacks rigorous data-driven basis; in complex environments with dense targets and severe clutter interference, its association accuracy decreases significantly; it is difficult to effectively handle highly nonlinear or dynamically changing association scenarios. Probabilistic data association: This method highly depends on precise knowledge of the target motion model and noise statistical characteristics. However, in practical applications, the target motion model (such as maneuvering patterns) is often complex and difficult to model accurately, and sensor noise characteristics may also have uncertainties or non-Gaussianity. Model mismatch will directly lead to errors in association probability calculation, thereby causing association performance deterioration or even track divergence. Neural network-based data association: As the current mainstream direction, this method utilizes the powerful feature extraction and pattern recognition capabilities of neural networks. However, existing methods still have significant shortcomings: This method requires a large amount of training data, and current research and applications mostly focus on association at the level of track segments or the overall track, lacking mature neural network model algorithms specifically for efficient and accurate point-to-point association of original sensor measurement points. Furthermore, how to effectively embed domain knowledge (such as spatiotemporal constraints) and the model's generalization ability are also challenges.

[0004] Therefore, at this stage, it is necessary to design a data-driven lattice neural network method for associating trajectory data from multiple heterogeneous sensors to solve the above problems. Summary of the Invention

[0005] The purpose of this invention is to provide a data-driven lattice neural network method for associating trajectory data of multi-source heterogeneous sensors, in order to solve the technical problems existing in the prior art, namely, the problem of associating trajectory data of multi-source heterogeneous sensors of maneuvering targets.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: A data-driven method for associating trajectory data from multiple heterogeneous sensors using lattice neural networks includes the following steps: S1. Construct a fully connected feedforward neural network to extract features from trajectory data with known identity codes; S2. Input the features into the constructed fuzzy lattice classifier to learn the parameters; S3. Use the learned model to predict track data with unknown identification codes to label pseudo-labels, and then associate data with the same pseudo-label at the same time.

[0007] Furthermore, before proceeding to step S1, the collected multi-source trajectory data needs to be preprocessed, as follows: First, the data is normalized; then, the data is labeled and encoded so that the trajectory data can be used for training. The number of targets is determined based on the collected flight track data and identification codes. Then, perform label encoding, with the encoding being 0 to... Since the integers are not explicitly defined, the task of associating waypoints is transformed into a task of classifying waypoints. Then, the DBSCAN denoising algorithm is used to detect outliers and remove them from each class of waypoint data. Finally, each feature dimension of the waypoint data is standardized using the following formula:

[0008] here It is the first Feature data in each dimension It is the first The mean of each feature dimension. It is the first The variance of each dimension feature.

[0009] Furthermore, in step S1, the construction of the fully connected feedforward neural network is as follows: Let the trajectory feature data be For input, where This represents the position parameter, therefore setting the network layer 0 to have One input; set up The first linear output is:

[0010] here This represents the connection weight matrix from layer 0 to layer 1. This represents the bias vector from layer 0 to layer 1. Since trajectory data is non-linear feature data, a non-linear function needs to be added; a non-linear activation function is selected. The first nonlinear output is obtained as follows:

[0011] A neural network with layers is represented as:

[0012] In the last hidden layer, the output probability vector of the traditional neural network is abstracted into lattice points in Euclidean space, and then a partial order relation is assigned. Lattice theory is used instead of the traditional method of taking the maximum probability component to assist in the output of the final value.

[0013] Furthermore, the construction of the fuzzy lattice classifier in step S2 is as follows: Training process: Assume the features output by the FNN and their corresponding labels are encoded as a dataset. ,in , is the Euclidean space data output by the FNN. ; For each type of data, generate a superbox according to the following process: (1) For the first Trajectory feature data, classifying the dataset into this category. Other data in all categories ; (2) Take out first A trajectory data point This forms a trivial hyperbox composed of a single point. , record storage The container of a superbox is ; (3) For each , judge Is it in If a superbox is selected, proceed to the next data; otherwise,... and All superboxes in the interval form a new interval superbox, denoted as . The rules for forming a new superbox are as follows: set up It is a test point, set as a super box. The resulting new superbox is ; (4) Select There is no The superbox of the dataset is denoted as . ; (5) If If the set is empty, then the second data point will be added to form a new single-point trivial superbox. Repeat for the next data point; (6) If If there is only one superbox, then replace the superbox with the correct one. The corresponding superbox is then used to repeat the process for the next data point; (7) If If there is more than one superbox, select the smallest superbox and replace it with the one that is currently in use. The corresponding superbox is then used to repeat the process for the next data point; (8) After completing the above operations, you will obtain the superbox for each type. This leads to the superboxes of all categories. Each superbox has a corresponding label, where , The category corresponding to the superbox; Testing process: Based on all the hyperboxes output during the training process, the embedding measure between each input test trajectory data point and the hyperbox can be calculated; the testing process is as follows: (1) Each superbox has a real label. , ; (2) Input test trajectory feature data Calculate the embedding measure with each superbox.

[0014] Definition: Let It is a set. binary relations on A partial order relation is called a partial order relation if it satisfies: (1) Reflexivity: ; (2) Antisymmetry: ; (3) Transitivity: , At this time, it is called It is a partially ordered set. Definition: In a partially ordered set In the middle, if any two elements All have supremacy and indeterminate boundary Then it is called a partially ordered set. One cell; In fact, from the definition of a case, we can obtain that if It is a grid. Any two elements in The union and intersection of lattices can be uniquely determined. On the other hand, operations in lattices can be simplified by using intersection and union operations; Definition: Let It is grid A subset, if for any There will always be Then it is called For grid Subcells; Definition: A positive definite function is defined as a function that takes values ​​of 0 on a lattice. function It satisfies the following conditions: (1) ; (2) ; Definition: Embedding measure is defined in The upper value is function The following four conditions must be met: (1) ; (2) ; (3) ; (4) . Theorem: If It is grid, It is a monotonic function on a lattice, defined as a function. , ,but yes Upper measurement; Proposition: Let the value function be: Define the following two functions: (1) ;

[0015] (2) ;

[0016] By definition of value functions, the two functions above are embedding measures; Definition: A fuzzy lattice is a fuzzy set. ,in It is a grid, that makes , denoted as ; The specific calculation process is given below: Define the value function for each feature dimension as follows: , in , The selected parameters are, where These represent the maximum and minimum values ​​in the corresponding dimensions, respectively. Therefore, in Define a value function: ; yes The function is monotonically positive, thus providing the formula for the function of calculating the value of multidimensional feature data; To compute interval-valued data, define an isomorphic function:

[0017] Take the median value of the data in the corresponding dimension, that is ; Therefore, for interval values The data can provide a value function: . So for single-point data Set up a super box Then the embedding measure is calculated as follows: . Therefore, the above process can provide data. For each category Chinese Super League Box The embedding measure makes ; Calculate the label corresponding to the largest superbox. ; Output It belongs to ; A fuzzy lattice classifier generates intervals from each class of the dataset. These intervals can be viewed as hypercubes in Euclidean space, hence they are also called hyperboxes. After generating hyperboxes from each class, each hyperbox is labeled with the corresponding class, and then all the hyperboxes from all classes are grouped together. When new data is input for testing, by defining an embedding measure on the interval lattice, since a single point is a trivial interval lattice, the embedding degree of the new data point with all hyperboxes can be calculated, and the label class of the hyperbox with the highest embedding degree is selected as the label class of the new data point.

[0018] Furthermore, the specific steps in step S3 for predicting pseudo-labels for track data from other sensors are as follows: The features of other sensor track data to be predicted are standardized in the same way as the training data, and then input into the lattice neural network to predict and output the predicted pseudo-labels.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: A data-driven lattice neural network model is constructed to achieve point-to-point association of multi-sensor track data with few samples. The core of lattice theory lies in its upper and lower bound operation mechanism, while the core of classification tasks is actually implicitly comparing similarity relationships. Therefore, this invention embeds the lattice structure into the neural network and achieves efficient and accurate point-to-point association of multi-sensor data by comparing the order relationships in different feature streams. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the validation set prediction association confusion matrix of an embodiment of this solution.

[0021] Figure 2 This is a schematic diagram of the training and validation loss function curves in an embodiment of this solution.

[0022] Figure 3 This is a schematic diagram of the test set confusion matrix of an embodiment of this solution.

[0023] Figure 4 This is a schematic diagram of the confusion matrix of the prediction results in an embodiment of this solution. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described embodiments are merely some embodiments of the invention, and not all embodiments.

[0025] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention. It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0026] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0027] Example: A data-driven method for associating trajectory data from multiple heterogeneous sensors using lattice neural networks includes the following steps: S1. Construct a fully connected feedforward neural network to extract features from trajectory data with known identity codes; S2. Input the features into the constructed fuzzy lattice classifier to learn the parameters; S3. Use the learned model to predict track data with unknown identification codes to label pseudo-labels, and then associate data with the same pseudo-label at the same time.

[0028] Furthermore, before proceeding to step S1, the collected multi-source trajectory data needs to be preprocessed, as follows: First, the data is normalized; then, the data is labeled and encoded so that the trajectory data can be used for training. The number of targets is determined based on the collected flight track data and identification codes. Then, perform label encoding, with the encoding being 0 to... Since the integers are not explicitly defined, the task of associating waypoints is transformed into a task of classifying waypoints. Then, the DBSCAN denoising algorithm is used to detect outliers and remove them from each class of waypoint data. Finally, each feature dimension of the waypoint data is standardized using the following formula:

[0029] here It is the first Feature data in each dimension It is the first The mean of each feature dimension. It is the first The variance of each dimension feature.

[0030] Furthermore, in step S1, the construction of the fully connected feedforward neural network is as follows: Let the trajectory feature data be For input, where This represents the position parameter, therefore setting the network layer 0 to have One input; set up The first linear output is:

[0031] here This represents the connection weight matrix from layer 0 to layer 1. This represents the bias vector from layer 0 to layer 1. Since trajectory data is non-linear feature data, a non-linear function needs to be added; a non-linear activation function is selected. The first nonlinear output is obtained as follows:

[0032] A neural network with layers is represented as:

[0033] In the last hidden layer, the output probability vector of the traditional neural network is abstracted into lattice points in Euclidean space, and then a partial order relation is assigned. Lattice theory is used instead of the traditional method of taking the maximum probability component to assist in the output of the final value.

[0034] Furthermore, the construction of the fuzzy lattice classifier in step S2 is as follows: Training process: Assume the features output by the FNN and their corresponding labels are encoded as a dataset. ,in , is the Euclidean space data output by the FNN. ; For each type of data, generate a superbox according to the following process: (1) For the first Trajectory feature data, classifying the dataset into this category. Other data in all categories ; (2) Take out first A trajectory data point This forms a trivial hyperbox composed of a single point. , record storage The container of a superbox is ; (3) For each , judge Is it in If a superbox is selected, proceed to the next data; otherwise,... and All superboxes in the interval form a new interval superbox, denoted as . The rules for forming a new superbox are as follows: set up It is a test point, set as a super box. The resulting new superbox is ; (4) Select There is no The superbox of the dataset is denoted as . ; (5) If If the set is empty, then the second data point will be added to form a new single-point trivial superbox. Repeat for the next data point; (6) If If there is only one superbox, then replace the superbox with the correct one. The corresponding superbox is then used to repeat the process for the next data point; (7) If If there is more than one superbox, select the smallest superbox and replace it with the one that is currently in use. The corresponding superbox is then used to repeat the process for the next data point; (8) After completing the above operations, you will obtain the superbox for each type. This leads to the superboxes of all categories. Each superbox has a corresponding label, where , The category corresponding to the superbox; Testing process: Based on all the hyperboxes output during the training process, the embedding measure between each input test trajectory data point and the hyperbox can be calculated; the testing process is as follows: (1) Each superbox has a real label. , ; (2) Input test trajectory feature data Calculate the embedding measure with each superbox.

[0035] Definition: Let It is a set. binary relations on A partial order relation is called a partial order relation if it satisfies: (1) Reflexivity: ; (2) Antisymmetry: ; (3) Transitivity: , At this time, it is called It is a partially ordered set. Definition: In a partially ordered set In the middle, if any two elements All have supremacy and indeterminate boundary Then it is called a partially ordered set. One cell; In fact, from the definition of a case, we can obtain that if It is a grid. Any two elements in The union and intersection of lattices can be uniquely determined. On the other hand, operations in lattices can be simplified by using intersection and union operations; Definition: Let It is grid A subset, if for any There will always be Then it is called For grid Subcells; Definition: A positive definite function is defined as a function that takes values ​​of 0 on a lattice. function It satisfies the following conditions: (1) ; (2) ; Definition: Embedding measure is defined in The upper value is function The following four conditions must be met: (1) ; (2) ; (3) ; (4) . Theorem: If It is grid, It is a monotonic function on a lattice, defined as a function. , ,but yes Upper measurement; Proposition: Let the value function be: Define the following two functions: (1) ;

[0036] (2) ;

[0037] By definition of value functions, the two functions above are embedding measures; Definition: A fuzzy lattice is a fuzzy set. ,in It is a grid, that makes , denoted as ; The specific calculation process is given below: Define the value function for each feature dimension as follows: , in , The selected parameters are, where These represent the maximum and minimum values ​​in the corresponding dimensions, respectively. Therefore, in Define a value function: ; yes The function is monotonically positive, thus providing the formula for the function of calculating the value of multidimensional feature data; To compute interval-valued data, define an isomorphic function:

[0038] Take the median value of the data in the corresponding dimension, that is ; Therefore, for interval values The data can provide a value function: . So for single-point data Set up a super box Then the embedding measure is calculated as follows: . Therefore, the above process can provide data. For each category Chinese Super League Box The embedding measure makes ; Calculate the label corresponding to the largest superbox. ; Output It belongs to ; A fuzzy lattice classifier generates intervals from each class of the dataset. These intervals can be viewed as hypercubes in Euclidean space, hence they are also called hyperboxes. After generating hyperboxes from each class, each hyperbox is labeled with the corresponding class, and then all the hyperboxes from all classes are grouped together. When new data is input for testing, by defining an embedding measure on the interval lattice, since a single point is a trivial interval lattice, the embedding degree of the new data point with all hyperboxes can be calculated, and the label class of the hyperbox with the highest embedding degree is selected as the label class of the new data point.

[0039] Furthermore, the specific steps in step S3 for predicting pseudo-labels for track data from other sensors are as follows: The features of other sensor track data to be predicted are standardized in the same way as the training data, and then input into the lattice neural network to predict and output the predicted pseudo-labels.

[0040] Experimental simulation results: Dataset Introduction: There are two types of sensor data: Dataset 1 has identification codes (used for model learning), and Dataset 2 does not have identification codes (to be predicted). The selected feature data for both are longitude, latitude, and altitude. Dataset 1 is divided into a training set (70%), a validation set (15%), and a test set (15%). The training set is used to train the model, while the validation set is used to evaluate the model's performance at the end of each training cycle. By observing and analyzing the validation loss and accuracy, we can determine the model's performance on unseen data. After the model is trained, the performance on the validation set is as follows: Figure 1 As shown.

[0041] Validation set association prediction accuracy: 98.03%; The loss curves for the training and validation sets are as follows: Figure 2 As shown.

[0042] Conclusion: The model converges well and does not exhibit overfitting. It can be used to predict the virtual class of data associated with other sensors. Test set performance as follows Figure 3 As shown.

[0043] Test set prediction accuracy: 98.03%; The prediction results for dataset 2 are as follows Figure 4 As shown.

[0044] Prediction accuracy: 98.51%.

[0045] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A data-driven lattice neural network multi-source heterogeneous sensor trajectory data association method, characterized in that, Includes the following steps: S1. Construct a fully connected feedforward neural network to extract features from trajectory data with known identity codes; S2. Input the features into the constructed fuzzy lattice classifier to learn the parameters; S3. Use the learned model to predict track data with unknown identification codes to label pseudo-labels, and then associate data with the same pseudo-label at the same time.

2. The data-driven lattice neural network method for associating trajectory data of multi-source heterogeneous sensors according to claim 1, characterized in that, Before proceeding to step S1, the collected multi-source trajectory data needs to be preprocessed, as follows: First, the data is normalized; then, the data is labeled and encoded so that the trajectory data can be used for training. The number of targets is determined based on the collected flight track data and identification codes. Then, perform label encoding, with the encoding being 0 to... Since the integers are not integers, the task of associating waypoints is transformed into a task of classifying waypoints. Then, the DBSCAN denoising algorithm is used to detect outliers and remove them from the trajectory data for each class. Finally, each feature dimension of the trajectory data is standardized using the following formula: here It is the first Feature data in each dimension It is the first The mean of each dimension feature, It is the first The variance of each dimension feature.

3. The data-driven lattice neural network multi-source heterogeneous sensor trajectory data association method according to claim 2, characterized in that, In step S1, the construction of the fully connected feedforward neural network is as follows: Let the trajectory feature data be For input, where This represents the position parameter, therefore setting the network layer 0 to have One input; Let The first linear output is then: here This represents the connection weight matrix from layer 0 to layer 1. This represents the bias vector from layer 0 to layer 1. Since trajectory data is non-linear feature data, a non-linear function needs to be added; a non-linear activation function is selected. The first nonlinear output is obtained as follows: The neural network representation of a layer is given by: In the last hidden layer, the output probability vector of the traditional neural network is abstracted into lattice points in Euclidean space, and then a partial order relation is assigned. Lattice theory is used instead of the traditional method of taking the maximum probability component to assist in the output of the final value.

4. The data-driven lattice neural network multi-source heterogeneous sensor trajectory data association method according to claim 3, characterized in that, The construction of the fuzzy lattice classifier in step S2 is as follows: Training process: Assume that the features output by the FNN and the corresponding label encoding are data sets where is the data of the Euclidean space of the FNN output, ; For each type of data, generate a superbox according to the following process: (1) for the first class of trajectory feature data, dividing the data set into the class data and all other class data ; (2) first take out one trajectory data point , form a trivial hyperbox composed of a single point , store the container of the first class hyperbox as ; (3) For each , judge Is it in If a superbox is selected, proceed to the next data; otherwise,... and All superboxes in the interval form a new interval superbox, denoted as . The rules for forming a new superbox are as follows: set up It is a test point, set as a superbox. The resulting new superbox is ; (4) select in the middle of in the middle of the data set, denoted by ; (5) If If the set is empty, then the second data point will be added as a new single-point trivial superbox. Repeat for the next data point; (6) If If there is only one super box, replace the super box in the corresponding super box in the and repeat the next data. (7) If more than one hyperbox, select the smallest one of the hyperboxes, and replace the hyperbox in the corresponding hyperbox, and repeat the next data; (8) After the above operation, the hyper-box of each class is obtained , and the hyper-box of all classes is obtained . Each hyper-box has a label corresponding to it, wherein , is the class corresponding to the hyper-box. Testing process: Based on all the hyperboxes output during the training process, the embedding measure between each input test trajectory data point and the hyperbox can be calculated; the testing process is as follows: (1) Each superbox has a real tag , ; (2) inputting test trajectory feature data , calculating the embedding measure with each hyperbox Definition: Let be a set, and be a binary relation on . (1) Self-reflexivity: ; (2) Antisymmetry: ; (3) Transmissibility: , This is called a partial order set. Definition: A poset is called a lattice if every two elements have a supremum and an infimum. ​​​​​ In fact, from the definition of lattice, if is a lattice, the join and meet of any two elements in can be uniquely determined. In another word, the operations in a lattice can be simplified by using the join and meet operations; Definition: Let It is grid A subset, if for any There will always be Then it is called For grid Subcells; Definition: A positive definite function is defined as a function that takes values ​​of 0 on a lattice. function It satisfies the following conditions: (1) ; (2) ； Definition: An embedding measure is a function defined on the space of all measures taking values in ; satisfying the following 4 conditions: (1) ； (2) ； (3) ； (4) . Theorem: If is a lattice, is a monotone function on the lattice, define the function , then is a metric on Proposition: Let the value function: ; define the following two functions: (1) ； (2) ； By definition of value functions, the two functions above are embedding measures; Definition: A fuzzy lattice is a fuzzy set , where is a lattice, such that , denoted by ; The specific calculation process is given below: Define the value function for each feature dimension as follows: , in , The selected parameters are, where These represent the maximum and minimum values ​​in the corresponding dimensions, respectively. Thus, the value function is defined on the set ； is an upper monotone positive function, thus giving the formula for the value function of the multidimensional characteristic data To compute interval-valued data, define an isomorphic function: taking the middle value in the corresponding dimension data, i.e. ; Thus, for interval values Data can give value functions: . So for single-point data Set up a super box Then the embedding measure is calculated as follows: . Thus the above procedure gives data For each class The embedding measure of a superbox Let ； Computing the label corresponding to the largest hyperbox ; The output is obtained is a member of ; The fuzzy lattice classifier generates intervals from each class of the dataset. These intervals can be viewed as hypercubes in Euclidean space, hence they are also called hyperboxes. After generating some superboxes for each data category, each superbox is labeled with the category name, and then all the superboxes for all categories are grouped together. When new data is input for testing, by defining an embedding measure on the interval grid, since a single point is a trivial interval grid, the embedding degree of the new data point with all superboxes can be calculated, and the label category of the superbox with the highest embedding degree is selected as the label category of the new data point.

5. The data-driven lattice neural network multi-source heterogeneous sensor trajectory data association method according to claim 1, characterized in that, The specific steps in step S3 for predicting pseudo-labels for track data from other sensors are as follows: The features of other sensor track data to be predicted are standardized in the same way as the training data, and then input into the lattice neural network to predict and output the predicted pseudo-labels.

Images