Trajectory data semi-supervised classification method and device

Abstract

The application provides a trajectory data semi-supervised classification method and device, and a twin neural network model in metric learning is used to utilize label data to solve the problem of insufficient data in the label data set. The two-stage model training is used to solve the problem of missing categories. First, the model is pre-trained using the unlabeled data set, then the pre-trained model is used to form a twin neural network model, and a small amount of labeled data is used for fine-tuning. In the fine-tuning process, the supervised and unsupervised parts of the twin neural network model are trained at the same time, so that the twin neural network model has the classification ability of labeled categories and the clustering ability of unlabeled categories.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a semi-supervised classification method and apparatus for trajectory data. Background Technology

[0002] With the development of machine learning and deep learning technologies, trajectory classification using neural network models has gradually become the mainstream method. However, training neural network models requires a large amount of high-quality labeled data. Under conditions of insufficient labeled data and limited prior information, overfitting is prone to occur during training, resulting in poor generalization ability of the trained model and low accuracy in trajectory data classification.

[0003] Therefore, ensuring the accuracy of trajectory data classification becomes a problem. Summary of the Invention

[0004] This application provides the following technical solution:

[0005] This application provides a semi-supervised classification method for trajectory data, including:

[0006] Acquire multiple trajectory data;

[0007] Each pair of trajectory data from multiple trajectory datasets is input into a Siamese neural network model to obtain a first feature vector and a second feature vector determined by the first and second autoencoders in the Siamese neural network model, as well as the similarity between the first and second feature vectors determined by the similarity calculator in the Siamese neural network model. The first and second autoencoders share network parameters. The Siamese neural network model is trained based on labeled trajectory data, and the first and second autoencoders are trained based on unlabeled trajectory data.

[0008] Clustering is performed on each of the aforementioned similarities to obtain multiple clusters, and the trajectory data corresponding to the similarities within each cluster are grouped into one class.

[0009] Optionally, the Siamese neural network model is constructed in the following manner:

[0010] Obtain unlabeled trajectory data;

[0011] The label-free trajectory data is input into the autoencoder to obtain the trajectory output data determined by the autoencoder;

[0012] Determine whether the first loss function value converges, where the first loss function value represents the difference between the trajectory output data and the unlabeled trajectory data;

[0013] If convergence occurs, training ends.

[0014] If convergence is not achieved, update the network parameters of the autoencoder and return to the step of obtaining unlabeled trajectory data.

[0015] Based on the trained autoencoder, a Siamese neural network model is constructed. The Siamese neural network model includes a first autoencoder and a second autoencoder, both of which have the same structure as the trained autoencoder.

[0016] Optionally, the Siamese neural network model is trained in the following way:

[0017] Obtain labeled trajectory data pairs;

[0018] The first and second labeled trajectory data in the labeled trajectory data pair are input into the Siamese neural network model to obtain the third feature vector determined by the first autoencoder and the fourth feature vector determined by the autoencoder in the Siamese neural network model, as well as the similarity between the third and fourth feature vectors determined by the similarity calculator.

[0019] Calculate a second loss function value, a third loss function value, and a fourth loss function value. The fourth loss function value represents the similarity between the third feature vector and the fourth feature vector and the difference between the labeled trajectory data and the labeled labels. The second loss function value represents the difference between the first trajectory output data determined by the first autoencoder and the first labeled trajectory data. The third loss function value represents the difference between the second trajectory output data determined by the second autoencoder and the second labeled trajectory data.

[0020] Determine whether the second loss function value, the third loss function value, and the fourth loss function value have converged;

[0021] If convergence occurs, training ends.

[0022] If convergence is not achieved, update the network parameters of the Siamese neural network model and return to the step of obtaining labeled trajectory data pairs.

[0023] Optionally, obtain labeled trajectory data pairs, including:

[0024] Acquire multiple labeled trajectory data;

[0025] By combining every two labeled trajectory data points from the plurality of labeled trajectory data, multiple labeled trajectory data pairs are obtained.

[0026] Select one labeled trajectory data pair from the plurality of labeled trajectory data pairs.

[0027] Optionally, clustering is performed on each of the aforementioned similarities to obtain multiple clusters, and the trajectory data corresponding to the similarities within each cluster are grouped into one class, including:

[0028] Select k similarities from each of the aforementioned similarities as k cluster centers to initially determine k clusters, where each cluster contains the cluster centers, and k is not greater than the total number of similarities.

[0029] For each cluster similarity to be clustered, the distance between the cluster similarity to be clustered and each of the cluster centers is calculated to obtain multiple distances. The cluster similarity to be clustered is assigned to the cluster to which the cluster center corresponding to the shortest distance among the multiple distances belongs. The cluster similarity to be clustered is one of the similarities among the multiple similarities except for the k cluster centers.

[0030] The cluster centers of each cluster are recalculated to obtain the current cluster centers;

[0031] Determine whether the difference between the current cluster center and the historical cluster centers is less than a threshold;

[0032] If the value is less than the threshold, then the clustering process ends.

[0033] If the similarity is not less than the threshold, return to the step of calculating the distance between the similarity to be clustered and each cluster center for each cluster similarity.

[0034] This application also provides a semi-supervised classification device for trajectory data, comprising:

[0035] The acquisition module is used to acquire multiple trajectory data.

[0036] The determination module is used to input every two trajectory data points from multiple trajectory data points into a Siamese neural network model to obtain a first feature vector and a second feature vector determined by a first autoencoder and a second autoencoder in the Siamese neural network model, and the similarity between the first feature vector and the second feature vector determined by a similarity calculator in the Siamese neural network model. The first autoencoder and the second autoencoder share network parameters. The Siamese neural network model is trained based on labeled trajectory data, and the first autoencoder and the second autoencoder are trained based on unlabeled trajectory data.

[0037] The clustering module is used to cluster the various similarities to obtain multiple clusters, and to group the trajectory data corresponding to the similarities within the clusters into one class.

[0038] Optionally, the device further includes:

[0039] Build modules are used for:

[0040] Obtain unlabeled trajectory data;

[0041] The label-free trajectory data is input into the autoencoder to obtain the trajectory output data determined by the autoencoder;

[0042] Determine whether the first loss function value converges, where the first loss function value represents the difference between the trajectory output data and the unlabeled trajectory data;

[0043] If convergence occurs, training ends.

[0044] If convergence is not achieved, update the network parameters of the autoencoder and return to the step of obtaining unlabeled trajectory data.

[0045] Based on the trained autoencoder, a Siamese neural network model is constructed. The Siamese neural network model includes a first autoencoder and a second autoencoder, both of which have the same structure as the trained autoencoder.

[0046] Optionally, the device further includes:

[0047] The training module is used for:

[0048] Obtain labeled trajectory data pairs;

[0049] The first and second labeled trajectory data in the labeled trajectory data pair are input into the Siamese neural network model to obtain the third feature vector determined by the first autoencoder and the fourth feature vector determined by the autoencoder in the Siamese neural network model, as well as the similarity between the third and fourth feature vectors determined by the similarity calculator.

[0050] Calculate a second loss function value, a third loss function value, and a fourth loss function value. The fourth loss function value represents the similarity between the third feature vector and the fourth feature vector and the difference between the labeled trajectory data and the labeled labels. The second loss function value represents the difference between the first trajectory output data determined by the first autoencoder and the first labeled trajectory data. The third loss function value represents the difference between the second trajectory output data determined by the second autoencoder and the second labeled trajectory data.

[0051] Determine whether the second loss function value, the third loss function value, and the fourth loss function value have converged;

[0052] If convergence occurs, training ends.

[0053] If convergence is not achieved, update the network parameters of the Siamese neural network model and return to the step of obtaining labeled trajectory data pairs.

[0054] Optionally, the process by which the training module acquires labeled trajectory data pairs specifically includes:

[0055] Acquire multiple labeled trajectory data;

[0056] By combining every two labeled trajectory data points from the plurality of labeled trajectory data, multiple labeled trajectory data pairs are obtained.

[0057] Select one labeled trajectory data pair from the plurality of labeled trajectory data pairs.

[0058] Optionally, the clustering module is specifically used for:

[0059] Select k similarities from each of the aforementioned similarities as k cluster centers to initially determine k clusters, where each cluster contains the cluster centers, and k is not greater than the total number of similarities.

[0060] For each cluster similarity to be clustered, the distance between the cluster similarity to be clustered and each of the cluster centers is calculated to obtain multiple distances. The cluster similarity to be clustered is assigned to the cluster to which the cluster center corresponding to the shortest distance among the multiple distances belongs. The cluster similarity to be clustered is one of the similarities among the multiple similarities except for the k cluster centers.

[0061] The cluster centers of each cluster are recalculated to obtain the current cluster centers;

[0062] Determine whether the difference between the current cluster center and the historical cluster centers is less than a threshold;

[0063] If the value is less than the threshold, then the clustering process ends.

[0064] If the similarity is not less than the threshold, return to the step of calculating the distance between the similarity to be clustered and each cluster center for each cluster similarity.

[0065] Compared with the prior art, the beneficial effects of this application are as follows:

[0066] In this application, the first and second autoencoders are trained on unlabeled trajectory data, ensuring that both autoencoders learn to extract features from the unlabeled trajectory data. The Siamese neural network model includes the first and second autoencoders, enabling it to extract unlabeled trajectory data. Furthermore, by training the Siamese neural network model on labeled trajectory data, it learns to extract features from labeled trajectory data, improving the accuracy of feature extraction. By inputting every two trajectory data points into the Siamese neural network model, the accuracy of the first and second feature vectors is guaranteed, thus ensuring the similarity between them. Clustering these similarities ensures the accuracy of the clustering. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 This is a flowchart illustrating a semi-supervised classification method for trajectory data provided in this application;

[0069] Figure 2 This is a schematic diagram of the structure of the self-encoder provided in this application;

[0070] Figure 3 This is a schematic diagram of the structure of an encoding network and a decoding network provided in this application;

[0071] Figure 4 This is a schematic diagram of the structure of a twin neural network model provided in this application;

[0072] Figure 5 This is a schematic diagram of the training process of a twin neural network model provided in this application;

[0073] Figure 6 This is a schematic diagram of a simulation trajectory dataset provided in this application;

[0074] Figure 7 This is a schematic diagram of the loss curve for the first stage of training provided in this application;

[0075] Figure 8 This is a schematic diagram of the loss curve for a second-stage training process provided in this application;

[0076] Figure 9 This is a diagram illustrating the comparison of clustering effects provided in this application;

[0077] Figure 10 This is a schematic diagram of the structure of a semi-supervised classification device for trajectory data provided in this application. Detailed Implementation

[0078] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0079] To address the aforementioned issues, this application provides a semi-supervised classification method for trajectory data, which will be described below.

[0080] Reference Figure 1 The above is a flowchart illustrating a semi-supervised classification method for trajectory data provided in this application. Figure 1 As shown, the method may include, but is not limited to, the following steps:

[0081] Step S11: Obtain multiple trajectory data.

[0082] Step S12: Input every two trajectory data points from multiple trajectory data points into the Siamese neural network model to obtain the first feature vector and the second feature vector determined by the first autoencoder and the second autoencoder in the Siamese neural network model, and the similarity between the first feature vector and the second feature vector determined by the similarity calculator in the Siamese neural network model. The first autoencoder and the second autoencoder share network parameters. The Siamese neural network model is trained based on labeled trajectory data, and the first autoencoder and the second autoencoder are trained based on unlabeled trajectory data.

[0083] In this embodiment, the Siamese neural network model can be constructed, but is not limited to, through the following methods:

[0084] S121. Obtain unlabeled trajectory data.

[0085] Unlabeled trajectory data can be understood as trajectory data that is not labeled. Labels can represent the category of trajectory data.

[0086] S122. Input the labelless trajectory data into the autoencoder to obtain the trajectory output data determined by the autoencoder;

[0087] In this embodiment, the structure of the autoencoder may include, but is not limited to, an encoding network, a feature vector layer, and a decoding network, such as... Figure 2 As shown.

[0088] Encoding networks may include, but are not limited to, an input layer and a GRU (Gate Recurrent Unit) layer; decoding networks may include, but are not limited to, a RepeatVector layer, a GRU layer, and an output layer, such as... Figure 3 As shown, GRU, like LSTM, is a model proposed to solve the long-term memory problem. Its performance is comparable to LSTM, but it has fewer parameters. In the Encoder network, the GRU layer is used to read the trajectory points of the Input trajectory data sequentially. During processing, it can preserve the correlation between points before and after the trajectory, thus reading the temporal features of the trajectory input data. The feature vector layer obtains feature vectors based on the temporal features. The RepeatVector layer is used to regenerate the time series from the extracted feature vectors. In the Decoder network, the GRU, which is symmetrical to the GRU in the Encoder network, reconstructs the trajectory output data based on the time series generated by the RepeatVector layer.

[0089] The input layer can normalize the attributes of the trajectory input data and pad with zeros to keep the trajectory input data of consistent length, which facilitates the processing of the GRU layer.

[0090] S123. Determine whether the first loss function value has converged. The first loss function value represents the difference between the trajectory output data and the unlabeled trajectory data.

[0091] In this embodiment, the first loss function value can be calculated using the following equation:

[0092]

[0093] Where X1 represents unlabeled trajectory data, This indicates the trajectory output data.

[0094] If convergence occurs, proceed to step S124; if convergence does not occur, proceed to step S125.

[0095] S124, End of training.

[0096] S125. Update the network parameters of the autoencoder and return to step S121.

[0097] In this embodiment, steps S121-S125 are the first stage of unsupervised learning training. By executing steps S121-S125, the unlabeled trajectory data can be fully utilized for the initial training of the model. The trained model is required to accurately capture the general features in a large amount of unlabeled trajectory data, ensuring that the model has basic clustering ability for both labeled and unlabeled categories in the labeled dataset.

[0098] S126. Based on the trained autoencoder, construct a Siamese neural network model, wherein the Siamese neural network model includes a first autoencoder and a second autoencoder, and both the first autoencoder and the second autoencoder have the same structure as the trained autoencoder.

[0099] The structure of the twin neural network model is as follows Figure 4 As shown, Figure 4 As shown, the Siamese neural network model includes: a first autoencoder and a second autoencoder. The first autoencoder and the second autoencoder share network parameters. The first autoencoder can receive first trajectory input data (Input1), and the second autoencoder can receive second trajectory input data (Input2). The first autoencoder determines a first feature vector of the first trajectory input data, and the second autoencoder determines a second feature vector of the second trajectory input data. The loss function is used to calculate the loss between the first feature vector and the second feature vector.

[0100] Because the aforementioned Siamese neural network model lacks guidance, the features extracted by the Siamese neural network model built after the first stage of training may not be effective. Therefore, in the second stage, labeled trajectory data is used to adjust the Siamese neural network model, enabling it to extract more effective features. Specifically:

[0101] Based on the construction of the twin neural network model, such as Figure 5 As shown, a Siamese neural network model can be trained, but is not limited to, in the following ways:

[0102] S127. Obtain labeled trajectory data pairs.

[0103] A labeled trajectory data pair includes two labeled trajectory data. Labeled trajectory data can be understood as trajectory data with labels, where labels can represent the category of the trajectory data.

[0104] It should be noted that two labeled trajectory data pairs have the same label. For example, if both labeled trajectory data pairs have a label of 1, it indicates that the two labeled trajectory data pairs belong to different categories; if both labeled trajectory data pairs have a label of 0, it indicates that the two labeled trajectory data pairs belong to the same category.

[0105] This step may include, but is not limited to:

[0106] S1271. Obtain multiple labeled trajectory data.

[0107] S1272. Combine every two labeled trajectory data in the plurality of labeled trajectory data to obtain a plurality of labeled trajectory data pairs.

[0108] In this embodiment, by combining every two labeled trajectory data points from the plurality of labeled trajectory data points to obtain multiple labeled trajectory data pairs, the training samples are expanded, which can increase the number of training iterations and thus more effectively utilize a small amount of labeled trajectory data for training.

[0109] S1273. Select one labeled trajectory data pair from the plurality of labeled trajectory data pairs.

[0110] S128. Input the first labeled trajectory data and the second labeled trajectory data in the labeled trajectory data pair into the Siamese neural network model to obtain the third feature vector determined by the first autoencoder and the fourth feature vector determined by the autoencoder in the Siamese neural network model, and the similarity between the third feature vector and the fourth feature vector determined by the similarity calculator.

[0111] S129. Calculate the second loss function value, the third loss function value, and the fourth loss function value, wherein the fourth loss function value characterizes the similarity between the third feature vector and the fourth feature vector and the difference between the labeled trajectory data and the labeled tags; the second loss function value characterizes the difference between the first trajectory output data determined by the first autoencoder and the first labeled trajectory data; and the third loss function value characterizes the difference between the second trajectory output data determined by the second autoencoder and the second labeled trajectory data.

[0112] In this embodiment, the second loss function value (Loss2) and the second loss function value (Loss3) can be calculated using the above relational formula 1, and the corresponding trajectory training data and trajectory output data can be substituted into relational formula 1.

[0113] The similarity between the third and fourth feature vectors can include, but is not limited to, the distance between the third and fourth feature vectors.

[0114] In this embodiment, the fourth loss function value (Loss4) can be calculated using the following relational formula:

[0115]

[0116] Where X1 and X2 represent the first and second labeled trajectory data, and Y is the label annotated on the first and second labeled trajectory data. Y = 0 when X1 and X2 belong to the same class, otherwise Y = 1. Dw represents the distance between the third and fourth feature vectors.

[0117] When Y=0, the network parameters of the Siamese neural network model are updated with the goal of minimizing Dw. When Y=1, if Dw is greater than m, the network parameters of the Siamese neural network model are not updated; if Dw is less than m, the network parameters of the Siamese neural network model are updated with the goal of increasing Dw to m. This makes the distance between similar positive trajectory data pairs in the feature domain closer and the distance between dissimilar negative trajectory data pairs farther, allowing the Siamese neural network model to effectively measure the similarity between trajectory data.

[0118] S1210. Determine whether the second loss function value, the third loss function value, and the fourth loss function value have converged.

[0119] This step may include, but is not limited to: calculating the sum of the second loss function value, the third loss function value, and the fourth loss function value, and determining whether the sum of the second loss function value, the third loss function value, and the fourth loss function value converges.

[0120] By determining whether the second, third, and fourth loss function values ​​converge, joint training is achieved in the second stage, simultaneously training both the supervised (i.e., the similarity calculator) and unsupervised (first and second autoencoders) parts of the Siamese neural network model. The first and second autoencoders, while adjusted based on the second and third loss function values, are also guided by the fourth loss function value, ensuring that they can extract more effective features. (First autoencoder, second autoencoder, and similarity calculator are mentioned.)

[0121] If convergence occurs, proceed to step S1211; if convergence does not occur, proceed to step S1212.

[0122] S1211, End of training;

[0123] S1212. Update the network parameters of the twin neural network model and return to the step of obtaining labeled trajectory data pairs.

[0124] Step S13: Cluster the various similarities to obtain multiple clusters, and group the trajectory data corresponding to the similarities within the clusters into one class.

[0125] This step may include, but is not limited to:

[0126] S131. Select k similarities from each of the aforementioned similarities as k cluster centers, initially determine k clusters, wherein each cluster contains the cluster centers, and k is not greater than the total number of similarities;

[0127] S132. For each cluster similarity to be clustered, calculate the distance between the cluster similarity to be clustered and each of the cluster centers to obtain multiple distances, and assign the cluster similarity to be clustered to the cluster center corresponding to the shortest distance among the multiple distances. The cluster similarity to be clustered is one of the similarities among the multiple similarities except for the k cluster centers.

[0128] S133. Recalculate the cluster center of each cluster to obtain the current cluster center;

[0129] S134. Determine whether the difference between the current cluster center and the historical cluster center is less than a threshold.

[0130] If the value is less than the threshold, proceed to step S135; if the value is not less than the threshold, return to step S132.

[0131] S135, End clustering.

[0132] In this embodiment, the similarity between trajectory data is determined using a Siamese neural network model, and then each similarity is clustered. Compared with direct classification, this approach can better preserve the model's ability to handle possible unlabeled categories.

[0133] In this embodiment, the first and second autoencoders are trained on unlabeled trajectory data, ensuring that they learn to extract features from the unlabeled trajectory data. The Siamese neural network model includes the first and second autoencoders, enabling it to extract unlabeled trajectory data. Furthermore, by training the Siamese neural network model with labeled trajectory data, it learns to extract features from labeled trajectory data, improving the accuracy of feature extraction. By inputting every two trajectory data points into the Siamese neural network model, the accuracy of the first and second feature vectors is guaranteed, thus ensuring the similarity between them. Clustering these similarities ensures the accuracy of the clustering.

[0134] In this embodiment, an experimental method is also provided to verify the actual working effect of the semi-supervised classification method for trajectory data provided in this application. Specifically:

[0135] The experimental environment was a laptop computer with Windows 10 operating system, 16GB of RAM, and an NVIDIA GeForce GTX 1050 Ti graphics card. The algorithm model was implemented using Python 3.6 and TensorFlow 1.12.0.

[0136] Experimental data:

[0137] The basic data in this embodiment is civil aviation trajectory data recorded by the ADS-B (Automatic Dependent Surveillance-Broadcast) system. The original data was downloaded from https: / / flightadsb.variflight.com and includes information such as flight number, time, latitude and longitude, altitude, speed, and heading angle. To reduce variables in the experiment, only the timestamp and latitude / longitude features—the two most basic spatiotemporal features—were retained and normalized. After preprocessing such as filtering, interpolation, and smoothing, ideal experimental trajectory data was constructed using airways as category labels. Trajectories from 12 relatively complex air routes were selected to create a simulation trajectory dataset, as shown below. Figure 6 .

[0138] Based on the aforementioned dataset, application conditions such as insufficient label data and missing label categories are simulated by controlling the number of samples and categories in the label dataset.

[0139] Algorithm implementation:

[0140] This case study was implemented using Python 3.6 and TensorFlow 1.12.0.

[0141] First, we write and train the autoencoder. We use the TensorFlow framework to build the autoencoder network structure and use a GRU network structure to read the temporal features of the trajectory.

[0142] The loss function uses mean squared error. The suitability of the network model design can be judged by observing changes in the loss.

[0143] like Figure 7 As shown, the loss curve in the first stage of training decreases steadily and eventually converges to a low level, indicating that the model training is normal. It can extract features from the trajectory quite well.

[0144] The training effect in the second phase can also be observed through changes in the loss curve. The loss in the second phase consists of two parts: the loss of the unsupervised component autoencoder and the loss of the Siamese neural network model. The loss curve under normal training conditions is as follows: Figure 8 As shown, the autoencoder loss remains stable during training, while the loss of the Siamese neural network model decreases.

[0145] Furthermore, PCA diagrams can be drawn using the various similarities obtained from the Siamese neural network model to make a preliminary observation of the overall clustering effect, thereby adjusting the network structure.

[0146] One problem encountered during the training of Siamese neural network models is imbalanced training samples. In a pair of input data forming a training sample, the number of training samples of the same class with a label of 0 is less than the number of training samples of different classes with a label of 1. Imbalanced training data can lead to a decline in model training performance. Therefore, during the training process, it is necessary to manually increase the number of training sample pairs composed of the same class to ensure experimental results.

[0147] The effect of this treatment was verified through a simple experiment. The results of the two clustering methods with and without the treatment are shown in Table 1. It can be seen that the treatment can effectively improve the processing effect of the Siamese neural network model.

[0148] Table 1

[0149]

[0150] In this embodiment, several key parameters are set during training: number of iterations = 50, batch size = 256, initial learning rate = 0.001, and an adaptive learning rate adjustment is designed. The settings of these parameters are related to factors such as the size of the dataset, image size, and the size of trajectory points in the image; adjustments need to be made based on the actual situation in specific experiments.

[0151] Finally, the similarity between every two trajectory data in the target dataset is calculated using a Siamese neural network model. After obtaining the similarity matrix, the k-means method is used for clustering in this case.

[0152] Performance metrics:

[0153] Considering the presence of some unlabeled categories in the dataset, this application uses clustering metrics to measure the accuracy of the results. The evaluation metrics chosen are purity and KL (Kullback-Leible) divergence.

[0154] Purity is a metric applicable to both clustering and classification, and in classification it is commonly referred to as classification accuracy. However, since clustering results cannot directly correspond to actual classes, purity calculation requires first assigning a correspondence between clusters and classes, using the class with the most samples in each cluster as the corresponding class. The calculation formula is as follows:

[0155]

[0156] In the formula, N represents the total number of samples; Q = {ω1, ω2, ..., ω3} j} represents the clusters after clustering; C = {c1, c2, ..., c3}j} indicates the correct category; ω k c represents all samples in the k-th cluster after clustering; j This represents the true sample in the j-th category. The purity value ranges from [0, 1], with a larger value indicating better clustering performance.

[0157] The concept of KL divergence originates from probability theory and information theory, and is also known as relative entropy. It is used to measure the difference between two distributions. The smaller the difference between the two distributions, the smaller the KL divergence. When the two distributions are identical, the KL divergence is 0. Its calculation formula is as follows:

[0158]

[0159] In the formula, p and q represent the distribution of the clustering results and the distribution of the actual samples, respectively. The smaller the KL divergence, the more similar the distribution of the clustering results is to the actual distribution, and the better the clustering effect. This index is mainly used to observe the overall clustering effect. When there is an imbalance in the classification accuracy of each class, the KL divergence can be used to intuitively reflect this.

[0160] Performance validation under conditions of insufficient sample size:

[0161] Under semi-supervised conditions, the clustering method of this application is compared with other classic semi-supervised algorithms to examine the ability of the clustering method of this application to utilize the labeled data.

[0162] Comparing the algorithms, we selected the label propagation algorithm and the model transfer algorithm.

[0163] Label propagation is a popular semi-supervised clustering algorithm. Simply put, it starts with labeled data, calculates the distance between samples, and assigns the closest samples to the corresponding category. Compared to unsupervised clustering methods, it makes better use of labeled data and can significantly improve clustering accuracy.

[0164] The principle of model transfer algorithms is based on the principle of transfer learning, which involves transferring a network trained on a clustering model to an LSTM network and then fine-tuning it. This application uses this principle to create a transfer network for its model.

[0165] We also used unsupervised clustering and LSTM classification models trained directly on a small amount of labeled data as comparison algorithms. Table 2 shows the clustering results of the above algorithms and the clustering method proposed in this application.

[0166] Table 2

[0167]

[0168]

[0169] The results are shown in Table 2. The direct clustering model and the LSTM model trained with a small number of labels performed the worst. This is because they only used unlabeled and labeled data, respectively, failing to fully utilize known information. Furthermore, the classification model trained with few labels performed even worse than the basic clustering model, indicating that the problem of insufficient labels was severe in this experiment. The labeled data was insufficient to reflect the characteristics of the dataset, making it impossible to effectively train the classification model.

[0170] The label propagation and transfer training LSTM models, two semi-supervised classification algorithms, significantly improved the performance. These two semi-supervised algorithms can be regarded as enhancements to the clustering model and the directly trained classification model, respectively, and can better integrate the label dataset and the target data to improve classification performance.

[0171] The algorithm designed in this application achieves superior results in both cluster purity and KL divergence. This demonstrates that this application can effectively utilize labeled data and is an excellent semi-supervised classification algorithm.

[0172] Performance verification under missing category conditions:

[0173] The ability to handle labeled categories has been verified in the previous section. This section further examines the algorithm's ability to cluster unlabeled categories in the target dataset.

[0174] In this application, the ability to handle unlabeled categories in the target dataset mainly stems from a two-stage training method that first trains the clustering model and then fine-tunes it. Therefore, a conventional Siamese neural network training method that directly uses labeled data without clustering pre-training is chosen for comparison. This verifies the effectiveness of the two-stage training design. The overall clustering performance of the two methods is shown in Table 3 below, and the LSTM classification model and label propagation algorithm are selected as comparison algorithms.

[0175] Table 3

[0176]

[0177]

[0178] As can be seen, the performance of both LSTM and label propagation classification networks rapidly declines due to their inability to classify unknown categories. In contrast, the clustering method using the similarity matrix derived from the Siamese neural network exhibits significantly higher stability. Among these, the model fine-tuning method demonstrates superior performance. This is particularly evident in the KL divergence metric, which reflects the overall distribution of the clustering results; the autoencoder-based fine-tuning method produces a more reasonable distribution of clustering results.

[0179] To more intuitively compare the classification performance of the two methods for unlabeled categories, the performance of directly trained Siamese neural networks on various trajectory processing methods is statistically analyzed. Figure 9 As shown in section (a), the clustering method of this application demonstrates the effectiveness of clustering for various trajectories. Figure 9 As shown in section (b), categories 11 and 12 correspond to the categories of the deleted tags.

[0180] like Figure 9 As shown, the directly trained model exhibits a rapid decline in classification performance for unlabeled two-class data, leading to a significant overall performance degradation. This can be attributed to the lack of foundational weights in the clustering model. Consequently, the Siamese neural network model only performs well on labeled data with known classes during training, resulting in an overfit-like phenomenon. It may even classify unlabeled classes with other classes, further degrading performance. This also leads to an unreasonable overall distribution of clustering results, severely impacting the KL divergence metric. In contrast, this application demonstrates greater stability and robustness for partially unlabeled classes.

[0181] Corresponding to the above embodiment of a semi-supervised classification method for trajectory data provided in this application, this application provides an embodiment of a semi-supervised classification device for trajectory data.

[0182] Please see Figure 10 The semi-supervised classification device for trajectory data includes: an acquisition module 100, a determination module 200, and a clustering module 300.

[0183] Module 100 is used to acquire multiple trajectory data.

[0184] The determination module 200 is used to input every two trajectory data from multiple trajectory data into a Siamese neural network model to obtain a first feature vector and a second feature vector determined by a first autoencoder and a second autoencoder in the Siamese neural network model, and the similarity between the first feature vector and the second feature vector determined by a similarity calculator in the Siamese neural network model. The first autoencoder and the second autoencoder share network parameters. The Siamese neural network model is trained based on labeled trajectory data, and the first autoencoder and the second autoencoder are trained based on unlabeled trajectory data.

[0185] The clustering module 300 is used to cluster the various similarities to obtain multiple clusters, and to group the trajectory data corresponding to the similarities within the clusters into one class.

[0186] In this embodiment, the trajectory data semi-supervised classification device may further include:

[0187] Build modules are used for:

[0188] Obtain unlabeled trajectory data;

[0189] The label-free trajectory data is input into the autoencoder to obtain the trajectory output data determined by the autoencoder;

[0190] Determine whether the first loss function value converges, where the first loss function value represents the difference between the trajectory output data and the unlabeled trajectory data;

[0191] If convergence occurs, training ends.

[0192] If convergence is not achieved, update the network parameters of the autoencoder and return to the step of obtaining unlabeled trajectory data.

[0193] Based on the trained autoencoder, a Siamese neural network model is constructed. The Siamese neural network model includes a first autoencoder and a second autoencoder, both of which have the same structure as the trained autoencoder.

[0194] In this embodiment, the trajectory data semi-supervised classification device may further include:

[0195] The training module is used for:

[0196] Obtain labeled trajectory data pairs;

[0197] The first and second labeled trajectory data in the labeled trajectory data pair are input into the Siamese neural network model to obtain the third feature vector determined by the first autoencoder and the fourth feature vector determined by the autoencoder in the Siamese neural network model, as well as the similarity between the third and fourth feature vectors determined by the similarity calculator.

[0198] Calculate a second loss function value, a third loss function value, and a fourth loss function value. The fourth loss function value represents the similarity between the third feature vector and the fourth feature vector and the difference between the labeled trajectory data and the labeled labels. The second loss function value represents the difference between the first trajectory output data determined by the first autoencoder and the first labeled trajectory data. The third loss function value represents the difference between the second trajectory output data determined by the second autoencoder and the second labeled trajectory data.

[0199] Determine whether the second loss function value, the third loss function value, and the fourth loss function value have converged;

[0200] If convergence occurs, training ends.

[0201] If convergence is not achieved, update the network parameters of the Siamese neural network model and return to the step of obtaining labeled trajectory data pairs.

[0202] In this embodiment, the process by which the training module acquires labeled trajectory data pairs specifically includes:

[0203] Acquire multiple labeled trajectory data;

[0204] By combining every two labeled trajectory data points from the plurality of labeled trajectory data, multiple labeled trajectory data pairs are obtained.

[0205] Select one labeled trajectory data pair from the plurality of labeled trajectory data pairs.

[0206] In this embodiment, the clustering module can specifically be used for:

[0207] Select k similarities from each of the aforementioned similarities as k cluster centers to initially determine k clusters, where each cluster contains the cluster centers, and k is not greater than the total number of similarities.

[0208] For each cluster similarity to be clustered, the distance between the cluster similarity to be clustered and each of the cluster centers is calculated to obtain multiple distances. The cluster similarity to be clustered is assigned to the cluster to which the cluster center corresponding to the shortest distance among the multiple distances belongs. The cluster similarity to be clustered is one of the similarities among the multiple similarities except for the k cluster centers.

[0209] The cluster centers of each cluster are recalculated to obtain the current cluster centers;

[0210] Determine whether the difference between the current cluster center and the historical cluster centers is less than a threshold;

[0211] If the value is less than the threshold, then the clustering process ends.

[0212] If the similarity is not less than the threshold, return to the step of calculating the distance between the similarity to be clustered and each cluster center for each cluster similarity.

[0213] It should be noted that each embodiment focuses on describing the differences from other embodiments, and the same or similar parts between the embodiments can be referred to accordingly. For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments.

[0214] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0215] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0216] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0217] The above provides a detailed description of a semi-supervised classification method and apparatus for trajectory data provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A semi-supervised classification method for trajectory data, characterized in that, include: Acquire multiple trajectory data; The trajectory data is civil aviation trajectory data; Each pair of trajectory data from multiple trajectory datasets is input into a Siamese neural network model to obtain a first feature vector and a second feature vector determined by the first and second autoencoders in the Siamese neural network model, as well as the similarity between the first and second feature vectors determined by the similarity calculator in the Siamese neural network model. The first and second autoencoders share network parameters. The Siamese neural network model is trained based on labeled trajectory data, and the first and second autoencoders are trained based on unlabeled trajectory data. Cluster the various similarities to obtain multiple clusters, and group the trajectory data corresponding to the similarities within the clusters into one class. The Siamese neural network model is constructed in the following manner: Obtain unlabeled trajectory data; The label-free trajectory data is input into the autoencoder to obtain the trajectory output data determined by the autoencoder; Determine whether the first loss function value converges, where the first loss function value represents the difference between the trajectory output data and the unlabeled trajectory data; If convergence occurs, training ends. If convergence is not achieved, update the network parameters of the autoencoder and return to the step of obtaining unlabeled trajectory data. Based on the trained autoencoder, a Siamese neural network model is constructed. The Siamese neural network model includes a first autoencoder and a second autoencoder, both of which have the same structure as the trained autoencoder. The twin neural network model was trained in the following way: Obtain labeled trajectory data pairs; The first and second labeled trajectory data in the labeled trajectory data pair are input into the Siamese neural network model to obtain the third feature vector determined by the first autoencoder and the fourth feature vector determined by the autoencoder in the Siamese neural network model, as well as the similarity between the third and fourth feature vectors determined by the similarity calculator. Calculate a second loss function value, a third loss function value, and a fourth loss function value. The fourth loss function value represents the similarity between the third feature vector and the fourth feature vector and the difference between the labeled trajectory data and the labeled labels. The second loss function value represents the difference between the first trajectory output data determined by the first autoencoder and the first labeled trajectory data. The third loss function value represents the difference between the second trajectory output data determined by the second autoencoder and the second labeled trajectory data. Determine whether the second loss function value, the third loss function value, and the fourth loss function value have converged; If convergence occurs, training ends. If convergence is not achieved, update the network parameters of the Siamese neural network model and return to the step of obtaining labeled trajectory data pairs.

2. The method according to claim 1, characterized in that, Obtain labeled trajectory data pairs, including: Acquire multiple labeled trajectory data; By combining every two labeled trajectory data from the plurality of labeled trajectory data, multiple labeled trajectory data pairs are obtained; Select one labeled trajectory data pair from the plurality of labeled trajectory data pairs.

3. The method according to claim 1, characterized in that, Clustering is performed on each of the aforementioned similarities to obtain multiple clusters. The trajectory data corresponding to the similarities within each cluster are then grouped into one class, including: Select k similarities from each of the aforementioned similarities as k cluster centers to initially determine k clusters, where each cluster contains the cluster centers, and k is not greater than the total number of similarities. For each cluster similarity to be clustered, the distance between the cluster similarity to be clustered and each of the cluster centers is calculated to obtain multiple distances. The cluster similarity to be clustered is assigned to the cluster to which the cluster center corresponding to the shortest distance among the multiple distances belongs. The cluster similarity to be clustered is one of the similarities among the multiple similarities except for the k cluster centers. The cluster centers of each cluster are recalculated to obtain the current cluster centers; Determine whether the difference between the current cluster center and the historical cluster centers is less than a threshold; If the value is less than the threshold, then the clustering process ends. If the similarity is not less than the threshold, return to the step of calculating the distance between the similarity to be clustered and each cluster center for each cluster similarity.

4. A semi-supervised classification device for trajectory data, characterized in that, include: The acquisition module is used to acquire multiple trajectory data. The trajectory data is civil aviation trajectory data; The determination module is used to input every two trajectory data points from multiple trajectory data points into a Siamese neural network model to obtain a first feature vector and a second feature vector determined by a first autoencoder and a second autoencoder in the Siamese neural network model, and the similarity between the first feature vector and the second feature vector determined by a similarity calculator in the Siamese neural network model. The first autoencoder and the second autoencoder share network parameters. The Siamese neural network model is trained based on labeled trajectory data, and the first autoencoder and the second autoencoder are trained based on unlabeled trajectory data. The clustering module is used to cluster the various similarities to obtain multiple clusters, and to group the trajectory data corresponding to the similarities within the clusters into one class. The device further includes: Build modules are used for: Obtain unlabeled trajectory data; The label-free trajectory data is input into the autoencoder to obtain the trajectory output data determined by the autoencoder; Determine whether the first loss function value converges, where the first loss function value represents the difference between the trajectory output data and the unlabeled trajectory data; If convergence occurs, training ends. If convergence is not achieved, update the network parameters of the autoencoder and return to the step of obtaining unlabeled trajectory data. Based on the trained autoencoder, a Siamese neural network model is constructed. The Siamese neural network model includes a first autoencoder and a second autoencoder, both of which have the same structure as the trained autoencoder. The device further includes: The training module is used for: Obtain labeled trajectory data pairs; The first and second labeled trajectory data in the labeled trajectory data pair are input into the Siamese neural network model to obtain the third feature vector determined by the first autoencoder and the fourth feature vector determined by the autoencoder in the Siamese neural network model, as well as the similarity between the third and fourth feature vectors determined by the similarity calculator. Calculate a second loss function value, a third loss function value, and a fourth loss function value. The fourth loss function value represents the similarity between the third feature vector and the fourth feature vector and the difference between the labeled trajectory data and the labeled labels. The second loss function value represents the difference between the first trajectory output data determined by the first autoencoder and the first labeled trajectory data. The third loss function value represents the difference between the second trajectory output data determined by the second autoencoder and the second labeled trajectory data. Determine whether the second loss function value, the third loss function value, and the fourth loss function value have converged; If convergence occurs, training ends. If convergence is not achieved, update the network parameters of the Siamese neural network model and return to the step of obtaining labeled trajectory data pairs.

5. The apparatus according to claim 4, characterized in that, The process by which the training module acquires labeled trajectory data pairs specifically includes: Acquire multiple labeled trajectory data; By combining every two labeled trajectory data from the plurality of labeled trajectory data, multiple labeled trajectory data pairs are obtained; Select one labeled trajectory data pair from the plurality of labeled trajectory data pairs.

6. The apparatus according to claim 4, characterized in that, The clustering module is specifically used for: Select k similarities from each of the aforementioned similarities as k cluster centers to initially determine k clusters, where each cluster contains the cluster centers, and k is not greater than the total number of similarities. For each cluster similarity to be clustered, the distance between the cluster similarity to be clustered and each of the cluster centers is calculated to obtain multiple distances. The cluster similarity to be clustered is assigned to the cluster to which the cluster center corresponding to the shortest distance among the multiple distances belongs. The cluster similarity to be clustered is one of the similarities among the multiple similarities except for the k cluster centers. The cluster centers of each cluster are recalculated to obtain the current cluster centers; Determine whether the difference between the current cluster center and the historical cluster centers is less than a threshold; If the value is less than the threshold, then the clustering process ends. If the similarity is not less than the threshold, return to the step of calculating the distance between the similarity to be clustered and each cluster center for each cluster similarity.

Images