Shielding target association method based on depth subspace clustering

Abstract

The invention discloses an occlusion target association method based on depth subspace clustering, and the method comprises the steps: firstly obtaining a target which needs to be associated through an existing target detection method; then, a deep learning method is used for obtaining feature representation of a to-be-associated target, and a self-expression coefficient matrix is constructed according to extracted target features and used for learning a self-expression relation between data; and integrating a reconstruction error and a regularization term, defining a loss function, and learning a target self-expression matrix, so that data can be effectively reconstructed, and a good subspace clustering structure is achieved. And finally, utilizing a self-supervised learning technology and iterative learning to obtain a stable pseudo tag to realize automatic association of the target in the subspace, thereby effectively solving the problem of target occlusion possibly occurring in target association.

Description

Technical Field

[0001] The present invention belongs to the technical field of target association, and more specifically, relates to an occluded target association method based on deep subspace clustering. Background Art

[0002] Object association technology plays an important role in many fields such as autonomous driving, intelligent monitoring and behavior recognition. However, in complex environments with dense crowds such as shopping malls, this technology faces challenges such as object occlusion and appearance similarity.

[0003] Traditional target association techniques, such as statistical-based methods (e.g., joint probabilistic data association) and geometric-based methods (e.g., nearest neighbor assignment), usually rely on simplified appearance and motion models. However, these methods often have difficulty achieving their expected performance when faced with challenges in occluded environments. Therefore, how to reliably associate targets when occlusion occurs has become a key issue that needs to be urgently addressed in the field of target association technology. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and provide an occluded target association method based on deep subspace clustering to achieve effective association of targets in occluded scenes.

[0005] To achieve the above-mentioned object of the invention, the present invention provides an occluded target association method based on deep subspace clustering, which is characterized by comprising the following steps:

[0006] (1) Target detection: Detect the target to be associated from the video;

[0007] (2) Target feature initialization: Using deep learning technology, select a suitable feature extractor to extract initial features for each target;

[0008] (3) Initialization of the self-expressive matrix: The features of all targets are integrated into a data matrix, in which each column represents a target. By learning the self-expressive matrix between targets, the features of any target can be approximately represented by a linear combination of the features of other targets. At the same time, a loss function containing reconstruction error and regularization terms is designed to ensure that the self-expressive matrix can not only efficiently reconstruct the data but also maintain a good clustering structure. In the process of optimizing the self-expressive coefficient matrix, more discriminative target features are further refined;

[0009] (4) Pseudo-label initialization: Select a common clustering algorithm to perform preliminary subspace clustering on the self-expression matrix Z and assign an initial pseudo-label to each target;

[0010] (5) Network update optimization: Comprehensive feature extraction, self-expression coefficient matrix update and pseudo-label optimization losses. Automatically update the weights of the neural network through the gradient descent method, update the target's feature vector, self-expression coefficient matrix and pseudo-label until the target pseudo-label is stable or reaches the preset training rounds;

[0011] (6) Automatic object association: According to the final pseudo-label, objects with the same label are classified into one category, and instances of the same object are associated in chronological order.

[0012] The object of the present invention is achieved in this way.

[0013] The present invention is based on an occluded target association method based on deep subspace clustering. All targets that need to be associated are detected by a target detection method, and features of the targets are extracted using deep learning, and a self-expression coefficient matrix is ​​constructed. At the same time, a loss function that integrates the self-expression error and the regularization term is designed to ensure that the self-expression matrix can not only efficiently reconstruct the data, but also reveal the underlying clustering structure. A subspace that is more suitable for target association is discovered through deep learning, rather than directly associating targets in the full space. In view of the fact that unknown targets cannot be identified in advance in unknown scenes, traditional supervised learning methods may not meet actual needs, so self-supervised learning technology is adopted. This method uses pseudo-labels generated by clustering results to guide the feature extraction of deep learning and the optimization of the self-expression matrix, thereby learning more representative feature representations in low-dimensional space. Through continuous cyclic iterations, the present invention not only significantly improves the accuracy of target association, but also realizes feature dimensionality reduction, thereby reducing the complexity of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of a specific implementation method of the occluded target association method based on deep subspace clustering of the present invention;

[0015] Figure 2 Schematic diagram of deep subspace clustering based on self-expression in the present invention, which includes an encoder E and a decoder D, X represents the initialization feature, represents the data reconstructed by the autoencoder;

[0016] Figure 3 Schematic diagram of the process of optimizing clustering to obtain pseudo labels in the present invention, X represents data features, C is the preliminary clustering result, C′ is the optimized clustering result, is the pseudo label assigned to the cluster;

[0017] Figure 4 It is a schematic diagram of the overall network of the occluded target association method based on deep subspace clustering in the present invention. DETAILED DESCRIPTION

[0018] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0019] Figure 1 It is a flow chart of a specific implementation of the occluded target association method based on deep subspace clustering of the present invention.

[0020] In this embodiment, if Figure 1 As shown, the occluded target association method based on deep subspace clustering of the present invention includes the following steps:

[0021] S1: Object Detection

[0022] Use some existing target detection methods to detect the targets that need to be associated. For example, in actual operation, you can directly use the YOLO pre-trained model to reduce training costs. If you want to use a custom dataset, you need to train or fine-tune the detection model to adapt to specific targets and environments. After using the model to predict the location of the target in the image, it usually outputs a bounding box to locate the target. The bounding box is usually defined by four parameters: the center coordinates of the target (x, y) and the width (w) and height (h) of the target. Detect all the targets that need to be associated in the video and obtain the target dataset O.

[0023] The performance of target detection directly affects the effect of subsequent target association, so this step requires a high degree of accuracy and efficiency.

[0024] S2: Target feature initialization

[0025] After obtaining the target, it is necessary to obtain features from the target that are helpful for subsequent association tasks. These features may include appearance features, motion features, and other features. In some cases, features from different sources may be combined, such as appearance features and motion features, to improve tracking accuracy.

[0026] In the specific implementation, the deep learning method is used to extract the initial target features, which can automatically learn complex feature representations from the data without manually designing features. At the same time, data enhancement techniques such as image transformation and noise injection can be used to improve the robustness and generalization ability of features. The deep learning model Φ is used to extract the representation vectors of all samples in the target data set O to obtain the set X of initial target features.

[0027] S3: Initialize the self-expression coefficient matrix

[0028] The core idea of ​​the self-expressive matrix is ​​that data points can be reconstructed by linear combinations of other data points in the same subspace. This means that each data point is a linear combination of other data points, thus expressing the global structure of the data. The self-expressive matrix contains the coefficients of how each data point is represented by a linear combination of other data points. Optimizing this matrix is ​​a key task in deep subspace clustering algorithms. Once the self-expressive matrix is ​​optimized, it can be used to construct a similarity matrix and apply clustering algorithms to achieve clustering of data points.

[0029] In specific implementation, Figure 2 As shown in the figure, an AutoEncoder neural network with θ as a parameter is combined to discover a more representative low-dimensional representation of the data. The initial feature X is nonlinearly mapped to the latent space Z through the encoder E, and the data is converted into a low-dimensional representation. Then, a self-expression matrix H is constructed using Z. It is expected that the self-expression matrix H represents the linear relationship between data points, and each data point can be represented as a linear combination of other points in the same subspace. In the application scenario of target association, it is assumed that data points with the same target attributes should be in the same subspace. By identifying these subspaces, data points with the same attributes can be effectively associated.

[0030] S4: Initialize pseudo labels

[0031] 4.1) When processing the target data set, due to the lack of manual annotation, the present invention adopts a method to assign a temporary pseudo-label to each sample for use in the model training process.

[0032] The implementation of label estimation relies on clustering analysis of feature vectors. The core idea of ​​clustering algorithms is to group similar samples together and separate dissimilar samples. Based on this, it can be inferred that samples of different category centers formed after clustering are unlikely to belong to the same category. Using this logic, we can create a set of benchmark samples for the unlabeled target dataset and assign temporary labels based on the distance relationship between them. This method allows a preliminary classification framework to be built for the dataset without explicit labels, which can then be used to train the model.

[0033] Specifically, we first use the model Φ to extract all samples o in the target dataset O i The initial characterization vector x i ; Then they are clustered into class, get Category Center in C is the number of true categories of the target dataset tThe estimated value of ; Finally, each category center is assigned a different label. For each other unlabeled sample, find the category center closest to it and assign the same label as the category center as the pseudo label, where the sample x i , x j The distance d(x i ,x j ) can be calculated as follows:

[0034] d(x i ,x j )=∥x i -x j ∥2

[0035] Among them, ∥x i -x j ∥2 represents sample x i , x j It is worth noting that, unlike classification tasks, the pseudo labels of unsupervised clustering results do not need to be numerically equal to the true identity labels, and they can correspond as closely as possible; There is no need to use C t Completely equal, in practical applications, an approximate value can be estimated based on empirical information. According to the above principles, the pseudo label of each sample in the target data set It can be expressed as:

[0036]

[0037] 4.2) The initial results of the clustering process usually have a certain degree of inaccuracy, and each data point is assigned a preliminary pseudo-label. Ideally, if these labels are completely correct, then all data points can be correctly classified, which will help the model learn features that distinguish different categories. However, since the pseudo-labels in the clustering process may be incorrect, especially in the early stages of model training, the model's feature extraction capabilities are insufficient, and incorrect cluster assignments will have a negative impact on model performance. Usually, due to the limited ability of the model to extract data features, the clustering results will inevitably contain some misclassified data points, which will lead to a decrease in the similarity within the clusters, making the clusters not "tight" enough. Considering the prevalence of this phenomenon, assume that in the current set of clusters, there are clusters C with lower internal cluster similarity. (i) Due to its low internal similarity, C (i) can be further divided into n i Smaller sub-clusters It should be emphasized that the further segmentation and clustering results mentioned are relative concepts. iIt is not a fixed constant. In order to search for samples that are incorrectly merged in clustering, two metrics are mainly used: intra-class distance and inter-class distance to measure whether the sample is a noise sample.

[0038] Specifically, if Figure 3 As shown, after preliminary clustering, cluster C is obtained (i) , and get a smaller cluster. (i) After further segmentation, is a smaller cluster after segmentation. In this cluster, a sample The distance between classes is D I (x), this distance is used to measure the distance between the current sample x and the cluster C to which it belongs. (i) Medium The compactness between all samples other than the included sample. Specifically, the distance is defined as follows:

[0039]

[0040] in Represents cluster C (i) Medium The remaining samples other than is the number of remaining samples, ∥xx i ∥2 is x, x in the feature space i The Euclidean distance between two samples. It should be emphasized that in practical applications, clustering C (i) Re-segmentation may lead to extreme cases where some clusters become singleton samples, so simply using It is not applicable to clustering of only singleton samples, and it will also force some samples that should be clustered to move away from each other in the feature space. To address this phenomenon, we use Calculate the total number of remaining samples in the cluster.

[0041] In order to quantify the clustering C (i) The compactness of clustering is defined as follows:

[0042]

[0043] Based on the above definition, when the inter-class distance D I (x) is less than the intra-class distance D C (C (i) ), for example

[0044] D I (x) <D C (C (i) )

[0045] It belongs to cluster C (i)The sample x will be retained in the cluster, otherwise the sample x will be removed from C (i) is excluded from the training and becomes a singleton sample, which exists as an independent cluster in the network training. In summary, for a singleton sample If the cluster C (i) Cannot be further divided into a set of small clusters Then it proves that the current cluster C (i) It is originally very compact in the feature space, so it will be retained as a whole; if it can be further divided, it is determined whether the sample x should stay in cluster C based on the condition (i) middle.

[0046] After clustering optimization, a new clustering result is obtained, and the cluster set including the singleton sample is defined as: C′={C (1) ,C (2) ,…,C (m′)}, at the same time, the pseudo label corresponding to each cluster can be obtained as:

[0047]

[0048] S5: Network update optimization

[0049] The loss of the entire network is mainly composed of three parts, namely the deep autoencoder, the self-expression matrix, and the self-supervision process based on pseudo labels. The overall network framework is as follows Figure 4 shown.

[0050] 5.1) Traditional self-expression representation technology is linear and cannot handle nonlinear scenarios. The feature X extracted initially is nonlinearly mapped to the latent space through the encoding layer of the autoencoder to learn a nonlinear mapping that is well suited for subspace clustering. The Frobenius norm is usually used to measure the reconstruction error of the data, that is:

[0051]

[0052] in, represents the data reconstructed by the autoencoder;

[0053] 5.2) When dealing with the optimization problem of the self-expressive matrix, the method used is to construct an objective function, which consists of two parts: one part is the error of data self-expression, which ensures that the original data can be accurately reconstructed through the self-expressive matrix; the other part is the regularization term, such as L1 or L2 norm, which helps to promote the sparsity or smoothness of the self-expressive matrix, thereby revealing the potential subspace structure of the data.

[0054] In this example, the self-expression matrix H represents the linear relationship between data points, and each data point can be expressed as a linear combination of other points in the same subspace. For a given data matrix X, the data X is nonlinearly mapped to the latent space Z, and the self-expression matrix H is constructed through Z. For the optimization of the self-expression matrix, it is necessary to consider the error of the data self-expression and the regularization term at the same time, that is:

[0055]

[0056] Among them, F means using the Frobenius norm, p usually takes the 1 or 2 norm to promote sparsity or smoothness, λ is the regularization parameter used to balance the influence of reconstruction error and regularization term, and diag(H)=0 is a constraint to ensure that the data point will not be represented by itself to avoid autoregression problems.

[0057] 5.3) Combining the characteristics of supervised training, after optimizing the label estimation and clustering results, the optimized pseudo-label can be added to the loss function to guide the model to conduct supervised training, so as to better learn the similarities between samples:

[0058]

[0059] Where L can represent any designed loss function, such as classification loss, contrast loss, ternary loss or any combination thereof, n t is the target quantity, x i Represents the current sample, Represents the current sample pseudo label.

[0060] Combining the losses mentioned in the above process and balancing the errors between the various parts, the final objective function is as follows:

[0061]

[0062] Among them, θ represents the network parameters, H is the self-expression matrix, Z is the low-dimensional feature representation obtained by the encoder, and x i Represents the current sample, represents the current sample pseudo label, λ1, λ2 and λ3 are hyperparameters.

[0063] In the specific optimization, a pre-training and fine-tuning strategy is adopted. Specifically, the deep autoencoder is first pre-trained, which can provide a good starting point for the subsequent training of the self-expressive layer. Then, the self-expressive layer is added on this basis, and the encoder and decoder are initialized with pre-trained parameters. In the fine-tuning stage, the gradient descent method or its variants (such as stochastic gradient descent, Adam optimizer, etc.) can be used to minimize the objective function. In the optimization process, the objective function is minimized by calculating the gradient of the objective function relative to the self-expressive matrix C and iteratively updating C.

[0064] S6: Target automatic association

[0065] In the previous step, each independent target is assigned a unique label. For the same target, the generated pseudo-labels will also be consistent. Simply by comparing the results of these pseudo-labels, it is easy to determine whether they point to the same target. This method effectively realizes the automatic association of targets in occluded environments.

[0066] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A method for associating occluded targets based on deep subspace clustering, characterized in that: The following steps are involved: (1) Target detection: Detect the target to be associated from the video; (2) Target feature initialization: Using deep learning technology, select a suitable feature extractor to extract initial features for each target; (3) Initialization of the self-expressive matrix: The features of all targets are integrated into a data matrix, in which each column represents a target. By learning the self-expressive matrix between targets, the features of any target can be approximately represented by a linear combination of the features of other targets. At the same time, a loss function containing reconstruction error and regularization terms is designed to ensure that the self-expressive matrix can not only efficiently reconstruct the data but also maintain a good clustering structure. In the process of optimizing the self-expressive coefficient matrix, more discriminative target features are further refined; (4) Pseudo-label initialization: Select a common clustering algorithm to perform preliminary subspace clustering on the self-expression matrix Z and assign an initial pseudo-label to each target; (5) Network update optimization: Comprehensive feature extraction, self-expression coefficient matrix update and pseudo-label optimization losses. Automatically update the weights of the neural network through the gradient descent method, update the target's feature vector, self-expression coefficient matrix and pseudo-label until the target pseudo-label is stable or reaches the preset training rounds; (6) Automatic object association: According to the final pseudo-label, objects with the same label are classified into one category, and instances of the same object are associated in chronological order.

2. The occluded target association method based on deep subspace clustering according to claim 1, characterized in that: In step (5), the network is updated to stabilize the pseudo-labels and further achieve automatic association of the target in the subspace. 5.1) Traditional self-expression representation technology is linear and cannot handle nonlinear scenarios. The feature X extracted initially is nonlinearly mapped to the latent space through the encoding layer of the autoencoder to learn a nonlinear mapping that is well suited for subspace clustering. The Frobenius norm is usually used to measure the reconstruction error of the data, that is: in, represents the data reconstructed by the autoencoder; 5.2) When dealing with the optimization problem of the self-expressive matrix, the method used is to construct an objective function, which consists of two parts: one part is the error of data self-expression, which ensures that the original data can be accurately reconstructed through the self-expressive matrix; the other part is the regularization term, such as L1 or L2 norm, which helps to promote the sparsity or smoothness of the self-expressive matrix, thereby revealing the potential subspace structure of the data. In this example, the self-expression matrix H represents the linear relationship between data points, and each data point can be expressed as a linear combination of other points in the same subspace. For a given data matrix X, the data X is nonlinearly mapped to the latent space Z, and the self-expression matrix H is constructed through Z. For the optimization of the self-expression matrix, it is necessary to consider the error of the data self-expression and the regularization term at the same time, that is: s.t.diag(H)=0 Among them, F means using the Frobenius norm, p usually takes the 1 or 2 norm to promote sparsity or smoothness, λ is the regularization parameter, which is used to balance the influence of reconstruction error and regularization term, and diag(H)=0 is a constraint to ensure that the data point will not be represented by itself to avoid autoregression problem; 5.3) Combining the characteristics of supervised training, after optimizing the label estimation and clustering results, the optimized pseudo-label can be added to the loss function to guide the model to conduct supervised training, so as to better learn the similarities between samples: Where L can represent any designed loss function, such as classification loss, contrast loss, ternary loss or any combination thereof, n t is the target quantity, x i represents the current sample, Represents the current sample pseudo label. Combining the losses mentioned in the above process and balancing the errors between the various parts, the final objective function is as follows: Among them, θ represents the network parameters, H is the self-expression matrix, Z is the low-dimensional feature representation obtained by the encoder, and x i represents the current sample, represents the current sample pseudo label, λ1, λ2 and λ3 are hyperparameters.

Images