Incremental semi-supervised image clustering method and system based on double-layer label propagation

Abstract

The application discloses a kind of based on the incremental semi-supervised image clustering method and system of double-layer label propagation, mainly used to solve the problem of low efficiency caused by repeated calculation when static semi-supervised clustering method faces incremental image data and incremental pairwise constraint.This application uses double-layer label propagation to process the clustering problem of increasing image data and constraint condition.In the first layer label propagation, propagate and diffuse pairwise constraint information in image data samples, and combine the membership matrix of component of image data samples at the last time, incrementally calculate the membership matrix of component of image samples at the current time.In the second layer label propagation, use the clustering result at the last time to mark cluster label information in component, and let known cluster label information propagate in component structure, then gradually expand cluster label information to the whole image dataset through the membership relationship of image sample pair component, so as to realize effective semi-supervised clustering of incremental image data.

Description

Technical Field

[0001] This invention belongs to the field of image clustering analysis, and relates to an incremental semi-supervised image clustering method and system based on two-layer label propagation. Background Technology

[0002] Image clustering is an important research area in computer vision and pattern recognition. Its purpose is to group similar images into one class based on their features, ensuring high similarity within the same class and low similarity between different classes. However, manually annotating image data requires significant manpower and time, especially in specialized fields (such as medical imaging), where expert annotation is even more costly. How to efficiently and accurately cluster the ever-increasing amount of image data with limited semi-supervised information has become a key focus for researchers.

[0003] Existing image clustering methods either design incremental learning algorithms to address the problem of continuously updating the model as new image data arrives, or research semi-supervised clustering algorithms to guide the clustering of unlabeled image data using limited labeled image data, thereby improving clustering quality. However, few studies have investigated incremental semi-supervised image clustering algorithms to address the problem of performing incremental clustering simultaneously using limited labeled image data and a large amount of unlabeled image data in an image data stream environment.

[0004] Therefore, there is a need for an incremental semi-supervised image clustering method that can efficiently process incremental image data to meet the needs of modern image data processing and is suitable for real-time detection and clustering applications, such as video surveillance and medical diagnosis. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to design an incremental semi-supervised image clustering method and system based on two-layer label propagation, so as to realize incremental semi-supervised image clustering of image data and its pairwise constraints that increase over time.

[0006] Technical solution: To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0007] An incremental semi-supervised image clustering method based on two-layer label propagation is characterized by the following steps:

[0008] (1) Given a preprocessed image data sample set at an initial time and constraint set x i Let n represent the vector formed by stretching the i-th image. 0 The initial number of samples is given, and the sample set includes the constrained image dataset at the initial time. and unconstrained image datasets This indicates that the i-th and j-th images belong to the same class. This indicates that the i-th and j-th images do not belong to the same class; a two-layer semi-supervised random walk is used to obtain the initial clustering result Y. 0 Specifically, it includes:

[0009] Random walk of the sample layer: First, the similarity matrix W between image samples is calculated by combining constraint information. 0 Row normalization is used to obtain the probability transition matrix P between image samples. 0 Then, a random walk is performed on the sample layer, and the settings are... It is an absorption state of random walk. For the transition state, each constrained image data and the portion of image data affected by it are defined as a component. The absorption probability of each constrained image data to other image data in the steady-state distribution is calculated, which serves as the membership matrix F of each image data to different components. 0 ;

[0010] Random walk at the component layer: Calculate the similarity matrix S between components by combining constraint information. 0 The probability transition matrix Q between components is obtained by using row normalization. 0 Then, eigenvalue decomposition is performed to obtain the cluster indicator matrix U of each component in the low-dimensional space. 0 Then combine the membership matrix F 0 Obtain the cluster indicator matrix G of image data samples in low-dimensional space. 0 , for G 0 Perform clustering to obtain the initial clustering result Y. 0 ;

[0011] (2) Given the newly added image data sample set at time t>0 and the newly added set of constraints and Let represent the mandatory and non-mandatory constraints added at time t, respectively, to obtain the accumulated image data sample set at time t. and cumulative constraint set The sample set includes the constrained image dataset accumulated at time t. and unconstrained image datasets and Let Y represent the accumulated image data sample set and constraint set at time t-1, respectively; the clustering result Y of all image data at time t is obtained by using two-layer semi-supervised label propagation. 0：t Specifically, it includes:

[0012] Label propagation at the sample layer: Utilizing the temporal smoothness of the spatial distribution of image data samples, the similarity matrix W among all image samples at time t is calculated incrementally. 0：t and in combination with constraint information After adjusting for similarity, row normalization is then applied to obtain the probability transition matrix P among all image samples at the current time. 0：t The membership matrix F of image samples to components at time t is incrementally calculated using the label propagation algorithm. 0：t ;

[0013] Tag propagation at the component layer: combining constraint information Calculate the similarity matrix S between components at time t. 0：t The probability transition matrix Q between components is obtained by using row normalization. 0：t Using the clustering result Y from the previous time step 0：t-1 The cluster representative component is calculated to obtain cluster label information. Through label propagation, the membership matrix R of the component to each cluster at time t is calculated. 0：t Finally, the joint membership matrix F 0：t Calculate the membership matrix G of all image samples to clusters. 0：t , for G 0：t Perform clustering to obtain the clustering result Y of all image samples at time t. 0：t .

[0014] Preferably, the similarity matrix W between image samples is calculated by combining constraint information. 0 This involves first calculating the similarity between two image samples using a Gaussian kernel function, and then exponentially scaling up or down the similarity between image data samples that meet the constraints. Constraints amplify similarity if they are satisfied. Constraints reduce similarity.

[0015] Preferably, the similarity matrix W between image samples is calculated. 0 The specific formula is as follows:

[0016]

[0017] Among them W 0 (i,j) represents the similarity matrix between the i-th and j-th image samples after adjustment, σ is the parameter of the Gaussian kernel function, and q∈(0,1] represents the adjustment coefficient for similarity.

[0018] As a preferred method, row normalization is used to obtain the probability transition matrix P between image samples. 0 =(diag(W) 0 1)) -1 W 0Where 1 represents a vector whose element values ​​are all 1, and diag(·) represents the operation of taking the diagonal vector of the matrix; the membership matrix of each image data to different components The calculation method is as follows:

[0019]

[0020] in and Let I represent the absorption probabilities of constrained and unconstrained image data, respectively. and For P 0 Internal block matrix This represents the probability transition matrix from an unconstrained image data sample to itself. This represents the probability transition matrix between unconstrained and constrained image data samples.

[0021] Preferably, the similarity matrix S between the components 0 =(F 0 ) T W 0 F 0 The component similarity matrix is ​​adjusted using constraint information to exponentially amplify or reduce the similarity between constrained components. If the constrained image samples contained in a component satisfy... Constraints amplify similarity if the constrained image samples contained in the component satisfy... Constraints reduce similarity; the cluster indicator matrix U of each component in low-dimensional space 0 From Q 0 After eigenvalue decomposition, the eigenvectors corresponding to the smallest eigenvalues ​​of the first preset number are used to form the cluster indicator matrix of the image data samples in the low-dimensional space, which is G. 0 =F 0 ×U 0 .

[0022] Preferably, the similarity matrix W among all image samples at time t is... 0：t The structure is as follows:

[0023]

[0024] in W is the similarity matrix among all image samples at the previous time step, when t=1. 0：0 =W 0 , and This is the similarity matrix between all image data samples from the previous time step and the newly added image data samples from the current time step. Create a similarity matrix between newly added image data samples at the current time; adjust W according to the newly added constraints. 0：t Adjustments are made to exponentially increase or decrease the similarity between image data samples that meet the new constraints. Constraints amplify similarity if they are satisfied. Constraints reduce similarity.

[0025] Preferably, the membership matrix F of image samples to components is then constructed incrementally using a block matrix approach. 0：t The details are as follows:

[0026]

[0027] in and These represent the membership matrices of constrained and unconstrained image data to the components, respectively. and Let represent the membership matrices of all constrained and unconstrained image data to the components at the previous time step, respectively, when t=1. and Let represent the membership matrix of unconstrained samples in the newly added image data at time t to components at historical time points, respectively. This represents the membership matrix of unconstrained samples in historical image data to the newly added component at time t. This represents the membership matrix of unconstrained samples to components in the newly added image data samples, calculated as follows:

[0028]

[0029] in This represents the transition probability matrix among all unconstrained image samples at the previous time step. This represents the transition probability matrix between all unconstrained image samples in the previous time step and the newly added constrained image samples in the current time step. This represents the transition probability matrix between newly added unconstrained image samples at the current time. This represents the transition probability matrix between newly added unconstrained image samples and constrained image samples at the current moment.

[0030] As a preferred method, the clustering result γ of all image samples at time t is calculated. 0：t Specifically, it includes:

[0031] First, calculate the similarity matrix S between components. 0：t =(F 0：t ) T W 0：t F 0：tAnd adjust it in conjunction with constraint information, exponentially amplifying or reducing the similarity between constrained components, if the constrained image samples contained in the component satisfy Constraints amplify similarity if the constrained image samples contained in the component satisfy... Constraints reduce similarity;

[0032] Next, the probability transition matrix Q between components is obtained through row normalization. 0：t =(diag(S) 0：t 1)) -1 S 0：t ;

[0033] Assuming the number of clusters remains constant during the data increment process, we can utilize the clustering results from the previous time step. To calculate the cluster center of each cluster at the current time, where This represents the k-th cluster at time t-1, and thus the set of cluster centers is obtained.

[0034]

[0035] Then, the cluster representative component closest to the cluster center at the current time is calculated to represent each cluster:

[0036]

[0037] in This represents the cluster representative component of the k-th cluster;

[0038] Next, tag propagation is performed at the component level, and the selected cluster represents the component. Given the absorbing state of a random walk and the remaining non-clustered components representing the transition states, the membership matrix R of all components to each cluster in the steady-state distribution is... 0：t Represented as:

[0039]

[0040] and These represent the membership degrees of cluster-representing components and non-cluster-representing components to each cluster, respectively. and For Q 0：t Internal block matrix This represents the mutual transition probability matrix between non-clustered components. This represents the transition probability matrix from cluster-represented components to non-cluster-represented components;

[0041] Finally, calculate the membership matrix G of all image samples to clusters at the current time. 0：t =F 0：t ×R 0：tThen, cluster them to obtain the clustering result Y of all image samples at the current time. 0：t .

[0042] A computer system includes a memory, a processor, and a computer program / instructions stored in the memory and executable on the processor, wherein the computer program / instructions, when executed by the processor, implement the steps of the incremental semi-supervised image clustering method based on two-layer label propagation.

[0043] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the incremental semi-supervised image clustering method based on two-layer label propagation.

[0044] Beneficial Effects: This invention proposes an incremental semi-supervised image clustering method based on two-layer label propagation for image clustering. This method overcomes two major drawbacks of previous methods: In traditional image clustering methods, either incremental learning algorithms are designed to solve the problem of how to continuously update the model as new image data arrives, or semi-supervised clustering algorithms are studied to guide the clustering of unlabeled image data with limited labeled image data, thereby improving the clustering quality. However, few studies have been conducted on incremental semi-supervised image clustering algorithms to solve the problem of incremental clustering using both limited labeled image data and a large amount of unlabeled image data in an image data stream environment. This method combines semi-supervised learning and incremental learning, which can both use a small amount of constrained image data to guide the clustering of image data to improve model performance, and incrementally update the model to adapt to the emergence of new image data and new constraints, maintaining the timeliness and accuracy of the model. Experiments on image data show that this method has superior performance, specifically:

[0045] 1) The proposed method is for image data with incremental transformation. It adopts incremental calculation technology, which can update the data required at the current time based on the existing data at the previous time when a new image sample arrives, thereby significantly reducing the computation cost and improving the processing efficiency.

[0046] 2) The proposed method can guide the clustering of a large amount of unconstrained image data with the help of a small amount of constraint information, thereby improving the accuracy of clustering;

[0047] 3) The distribution of image data may change over time. This method can adapt to these changes, adjust the model in a timely manner, maintain the stability and consistency of clustering results, and avoid model aging.

[0048] The advantage of this invention lies in its utilization of the relatively slow and continuous temporal change in the spatial distribution of image data samples. It combines incremental learning with semi-supervised learning techniques, applying them to image datasets that continuously increase over time. This method not only demonstrates excellent real-time processing capabilities, computational and storage efficiency, but also improves clustering quality through the addition of constraint information. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the algorithm structure in an embodiment of the present invention. Detailed Implementation

[0050] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] The incremental semi-supervised image clustering method based on two-layer label propagation disclosed in this invention has the following detailed execution steps:

[0052] (1) Given a sample set of image data at an initial time and constraint set Where n 0 x is the number of samples at the initial time. i This represents the vector formed by stretching the i-th image, where each x... i This represents a d-dimensional image data sample, where d represents the number of features in the image data. and These represent the constrained and unconstrained image datasets at that moment, respectively. and This indicates the mandatory and non-mandatory constraints at that moment. This indicates that the i-th and j-th images belong to the same class. This indicates that the i-th and j-th images do not belong to the same class; a two-layer semi-supervised random walk is used to obtain the initial clustering result Y. 0 Specifically, it includes:

[0053] (1-1) Random walk of the sample layer: Calculate the similarity matrix (W) between image samples by combining constraint information. 0 The probability transition matrix (P) between image samples is obtained by row normalization. 0 Then, a random walk is performed on the sample layer to calculate the absorption probability of each constrained image data to other image data in the steady-state distribution, which serves as the membership matrix (F) of each image data to different components. 0 );

[0054] (1-2) Random walk at the component layer: Calculate the similarity matrix (S) between components by combining constraint information. 0 The probability transition matrix (Q) between components is obtained by using row normalization. 0Then, eigenvalue decomposition is performed to obtain the cluster indicator matrix (U) of each component in the low-dimensional space. 0 Then, by combining the membership matrix of the image sample point pair components, the cluster representation matrix of the image sample data in the low-dimensional space is obtained. Finally, it is clustered to obtain the initial clustering result Y. 0 ;

[0055] (2) Given a new set of image data samples (x) at time t (t>0). t and new constraint sets and This represents the newly added mandatory and non-mandatory constraints at time t, and yields the accumulated image data samples at the current time. and cumulative constraints This represents the accumulated constrained image dataset at time t. Y represents the clustering result of all image data at the current time, obtained by using two-layer semi-supervised label propagation, representing the accumulated unconstrained image dataset. 0：t Specifically, it includes:

[0056] (2-1) Label propagation at the sample layer: Utilizing the temporal smoothness of the spatial distribution of image data samples, the similarity matrix (W) among all image samples at the current time is calculated incrementally. 0：t ), and combine information After adjusting for similarity, row normalization is then applied to obtain the probability transition matrix (P) among all image samples at the current time step. 0：t The membership matrix (F_i) of the image samples to components at the current time is incrementally calculated using the label propagation algorithm. 0：t );

[0057] (2-2) Tag propagation at the component layer: combining constraint information Calculate the similarity matrix (S) between components at the current time step. 0：t The probability transition matrix (Q) between components at the current time step is obtained by using row normalization. 0：t Using the clustering results from the previous time step (Y) 0：t-1 ) Calculate the cluster representative component and then obtain the cluster label information. Through label propagation, calculate the membership matrix (Ri) of the component to each cluster at the current time. 0：t Finally, the membership matrix (F) of the current image samples to the components is combined. 0：t ), calculate the membership matrix (G) of all image samples to clusters. 0：t Applying the k-means clustering algorithm to it, we obtain the clustering results (Y) of all image samples at the current time. 0：t ).

[0058] Step (1-1) firstly calculates the similarity between two image samples using a Gaussian kernel function, and then, in conjunction with constraint information, exponentially amplifies or reduces the similarity between image data samples that meet the constraints. If the constraints are met... Constraints amplify similarity if they are satisfied. Constraints reduce similarity; in this embodiment, a similarity matrix W is calculated between image samples. 0 The specific formula is as follows:

[0059]

[0060] Among them W 0 (i,j) represents the similarity matrix between image sample i and image sample j after adjustment, σ is the parameter of the Gaussian kernel function, and q∈(0,1] represents the adjustment coefficient for similarity, which exponentially amplifies or reduces the similarity between image data samples that meet the constraints. Row normalization is used to obtain the probability transition matrix P between image samples. 0 And obtain its block matrix structure.

[0061]

[0062] Where 1 represents a vector whose elements are all 1s, and diag(·) represents the operation of taking the diagonal vector of the matrix. and This represents the probability transition matrix from constrained image samples and unconstrained image samples to themselves. and This represents the probability transition matrix between unconstrained and constrained image samples.

[0063] Next, a random walk is performed on the sample layer, and the settings are... It is an absorption state of random walk. For the transition state, calculate the probability of all image data being absorbed by each absorption state when the steady-state distribution is reached.

[0064]

[0065] in and Let F represent the absorption probabilities of constrained and unconstrained image data, respectively, and I be the identity matrix. 0 element f ij This represents the j-th constrained image sample paired with the i-th image sample x. i The degree of influence. Define each constrained image data and the portion of image data affected by it as a component:

[0066]

[0067] Then F 0 It also represents the membership matrix of each image data to different components.

[0068] Steps (1-2) are for obtaining the initial clustering result Y. 0 First, use F 0 W 0 The similarity matrix between components is calculated using the following formula:

[0069] S 0 =(F 0 ) T W 0 F 0

[0070] Based on the component definition, the component similarity matrix is ​​adjusted using constraint information. Similarly, the similarity between constrained components is amplified or reduced exponentially. If the constrained image samples contained in a component satisfy... Constraints amplify similarity if the constrained image samples contained in the component satisfy... Constraints reduce the similarity. In this embodiment, the formula is expressed as follows:

[0071]

[0072] Where, x μ x v Indicates component Ω μ and component Ω v The constrained image samples included.

[0073] The obtained S 0 Row normalization is performed to obtain the probability transition matrix Q between components. 0 The calculation formula is as follows:

[0074] Q 0 =(diag(S) 0 1)) -1 S 0

[0075] Then to Q 0 Perform eigenvalue decomposition and use the eigenvectors corresponding to the first m smallest eigenvalues ​​to form the cluster indicator matrix of the components in the low-dimensional space. m represents the cluster number. Combining the membership matrix of the components and the cluster indicator matrix of the components from the initial image data samples, the cluster indicator matrix of the image data samples is obtained, calculated using the following formula:

[0076] G 0 =F 0 ×U 0

[0077] The obtained G0 By applying the k-means clustering algorithm, the clustering result Y of the image samples at the initial time step can be obtained. 0 .

[0078] In step (2-1), it is assumed that the newly added image data set at time t is represented as follows: Where n t The number of new samples added at time t is given. Furthermore, this image dataset also carries a small set of pairwise constraints provided by prior knowledge. Specifically, it is used to specify the relationship between the class assignments of two objects, also known as mandatory and non-mandatory constraints. A mandatory constraint between two image samples means that the two image samples tend to be assigned to the same cluster, while a non-mandatory constraint between two image data samples means that they tend to be assigned to different clusters.

[0079] Assuming that the increase in image data does not change the spatial distribution of the original image data, the total number of nodes on the graph at time t can be represented as follows: Therefore, the similarity matrix W between image samples 0：t It can be constructed incrementally:

[0080]

[0081] Here, σ is a hyperparameter controlling the decay rate of the Gaussian kernel function. To ensure consistency between the similarity between image samples and the constraints between them, a weighting method is used to adjust the similarity between different image samples to better satisfy the constraints. The adjustment rules for the weights are as follows:

[0082]

[0083] Where q∈(0,1] represents the adjustment coefficient, which exponentially amplifies or reduces the similarity between image data samples that meet the constraints. Obviously, the smaller the q value, the stronger the similarity adjustment.

[0084] Then, the similarity matrix is ​​normalized so that the sum of the elements in each row is 1, thus obtaining the probability transition matrix between all image samples at time t. The calculation formula is as follows:

[0085] P 0：t =(diag(W) 0：t 1)) -1 W 0：t

[0086] Here, 1 represents a vector whose elements are all 1s, and diag(·) represents the operation of taking the diagonal vector of the matrix. The probability transition matrix can also be represented as follows:

[0087]

[0088] in, and This represents the probability transition matrix from constrained image samples and unconstrained image samples to themselves. and This represents the probability transition matrix between unconstrained and constrained image data samples.

[0089] Next, the membership matrix F of image samples to components is incrementally constructed using a block matrix approach. 0 The details are as follows:

[0090]

[0091] in and These represent the membership matrices of constrained and unconstrained image data to the components, respectively. and Let represent the membership matrices of all constrained and unconstrained image data to the components at the previous time step, respectively. When t-1... and This represents the membership matrix of unconstrained image samples in the newly added image data at time t to components from historical time points. This represents the membership matrix of unconstrained image samples in historical image data to the newly added component at time t. This represents the membership matrix of unconstrained image samples to components in the newly added image data samples, calculated as follows:

[0092]

[0093] in This represents the transition probability matrix among all unconstrained image samples at the previous time step. This represents the transition probability matrix between all unconstrained image samples in the previous time step and the newly added constrained image samples in the current time step. This represents the transition probability matrix between newly added unconstrained image samples at the current time. This represents the transition probability matrix between newly added unconstrained image samples and constrained image samples at the current moment.

[0094] In step (2-2), a component-level random walk is performed to calculate the clustering result (Y) of all image samples at the current time. 0：t Similar to the overall process of random walks in the sample layer, the first step is to construct a similarity matrix between components. The difference is that here, the image data samples obtained from the sample layer and the membership relationship matrix F between components are used. 0：tTo calculate the similarity matrix S between the components 0：t Its formula is expressed as follows:

[0095] S 0：t =(F 0：t ) T W 0：t F 0：t

[0096] To ensure consistency between constraints and similarity within components, a similar approach is used to modify the component similarity matrix S as before. 0：t Make the following modifications:

[0097]

[0098] Where, x μ x v Indicates component Ω μ and component Ω v The constrained image samples included.

[0099] Similarly, the modified inter-component similarity matrix S can be used. 0：t Calculate the probability transition matrix between components:

[0100] Q 0：t =(diag(S) 0：t 1)) -1 S 0：t

[0101] In addition, the clustering results from the previous time step also need to be utilized. Calculate the cluster center for each cluster, where Representing the k-th cluster at time t-1, we obtain the set of cluster centers. The calculation formula is as follows:

[0102]

[0103] Then, the cluster representative component closest to the cluster center at the current time is calculated to represent each cluster:

[0104]

[0105] in This represents the cluster representative component of the k-th cluster.

[0106] Next, tag propagation is performed at the component level, and the selected cluster represents the component. For the absorbing state of a random walk, and the remaining non-clustered components as transition states, then the membership matrix (R0) of all components to each cluster in the steady-state distribution is... 0：t This can be represented as:

[0107]

[0108] and These represent the membership degrees of cluster-representing components and non-cluster-representing components to each cluster, respectively. and For Q 0：t Internal block matrix This represents the mutual transition probability matrix between non-clustered components. This represents the transition probability matrix from cluster-represented components to non-cluster-represented components.

[0109]

[0110] The membership matrix F of all image samples to components at the current time. 0：t and the cluster indicator matrix R of the components 0：t This yields the membership matrix (G) of all image samples to clusters at the current time. 0：t The calculation formula is as follows:

[0111] G 0：t =F 0：t ×R 0：t

[0112] Finally, the obtained G 0：t The data is input into k-means for clustering, yielding the clustering result Y for all image samples at the current time. 0：t .

[0113] Figure 1 A schematic diagram illustrating the algorithm of the incremental semi-supervised image clustering method based on two-layer label propagation.

[0114] To verify the effectiveness of the proposed algorithm ISCDLP (Incremental Semi-supervised Clustering based on Double-layer Label Propagation), comparative experiments were conducted on three real-world image datasets to evaluate the clustering metric NMI. These datasets vary in image size, allowing for a more accurate reflection of the effectiveness and stability of the proposed method under different image data sizes. The USPS image dataset is a handwritten digit recognition dataset used to identify handwritten digit characters in the United States Postal Service. It contains 7,291 grayscale image samples, each a 16x16 pixel image corresponding to the digits 0 to 9. The COIL20 image dataset is an image dataset for image recognition, containing images of 20 different objects. Each object's image was captured from different angles, totaling 72 images. Each image was converted into a 1024-dimensional feature vector. The Pendigits image dataset is a handwritten digit recognition dataset containing samples of 16 handwritten digits (0-9). Each sample consists of 16 features representing the strokes of the handwritten digit. This image dataset is widely used for character recognition and pattern recognition tasks. The experimental dataset is described in Table 2. The incremental semi-supervised clustering results of the three algorithms at 10 time points on three real image datasets are shown in Table 3.

[0115] Table 2 Information on the real image dataset used in the experiment

[0116]

[0117] Table 3 shows the NMI metrics of ISCDLP and other algorithms at various time slices on a real image dataset.

[0118]

[0119] As shown in Table 3, the incremental semi-supervised clustering performance of the present invention on three image datasets is better than the other two comparison algorithms, and the semi-supervised clustering effect gets better and better as the time slice increases.

[0120] An embodiment of the present invention discloses a computer system, including a memory, a processor, and a computer program / instructions stored in the memory and executable on the processor. When the computer program / instructions are executed by the processor, they implement the steps of the incremental semi-supervised image clustering method based on two-layer label propagation.

[0121] An embodiment of the present invention discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the incremental semi-supervised image clustering method based on two-layer label propagation.

Claims

1. An incremental semi-supervised image clustering method based on double-layer label propagation, characterized in that, comprising the steps of: (1) a pre-processed image data sample set at an initial time point and a constraint condition set x i represents a vector drawn from the ith image, n 0 is the number of samples at the initial time point, and the sample set includes a constrained image data set at the initial time point and an unconstrained image data set represents that the ith and jth images belong to the same class, represents that the ith and jth images do not belong to the same class; a clustering result Y at the initial time point is obtained by using a double-layer semi-supervised random walk 0 , and specifically comprises:​ Random walk of sample layer: firstly, the similarity matrix W between image samples is calculated combined with the constraint information 0 , the probability transition matrix P between image samples is obtained by row normalization 0 , then the random walk of sample layer is performed, setting as the absorbing state of random walk, as the transition state, defining each constrained image data and the part of image data affected by it as a component, calculating the absorbing probability of each constrained image data to other image data in the steady state distribution as the membership matrix F of each image data to different components 0 ; Random walk on component layer: compute similarity matrix S between components with constraint information 0 , get probability transition matrix Q between components with row normalization 0 , then get cluster indicator matrix U of each component in low dimensional space by eigenvalue decomposition 0 , get cluster indicator matrix G of image data sample in low dimensional space by combining membership matrix F 0 0 , cluster G 0 , get clustering result Y at initial time 0 ;​ (2) the newly added image data sample set at time t > 0 and the newly added constraint condition set and respectively represent the newly added necessary connection constraint and the disconnection constraint at time t, to obtain the accumulated image data sample set at time t and the accumulated constraint condition set The sample set includes the constrained image data set accumulated at time t and the unconstrained image data set and respectively represent the image data sample set and the constraint condition set accumulated at time t-1; the clustering result Y of all image data at time t is obtained by using double-layer semi-supervised label propagation 0:t , specifically comprising: Label propagation of sample layer: using the characteristic that the spatial distribution of image data samples has smoothness in time, incrementally calculating the similarity matrix W between all image samples at time t 0:t , and combining constraint information to adjust the similarity, and then using row normalization to obtain the probability transition matrix P between all image samples at the current time 0:t , and incrementally calculating the membership matrix F of the image samples to the components at time t by the label propagation algorithm 0:t ; Tag propagation at the component layer: combining constraint information Calculate the similarity matrix S between components at time t. 0:t The probability transition matrix Q between components is obtained by using row normalization. 0:t Using the clustering result Y from the previous time step 0:t-1 The cluster representative component is calculated to obtain cluster label information. Through label propagation, the membership matrix R of the component to each cluster at time t is calculated. 0:t Finally, the joint membership matrix F 0:t Calculate the membership matrix G of all image samples to clusters. 0:t , for G 0:t Perform clustering to obtain the clustering result Y of all image samples at time t. 0:t .

2. The method of claim 1, wherein, The binding constraint information calculates a similarity matrix W between image samples 0 , including first calculating the similarity of two image samples by using a Gaussian kernel function, then amplifying or reducing the similarity between image data samples satisfying the constraint condition in the form of an exponential, if the constraint is satisfied , the similarity is amplified, if the constraint is satisfied , the similarity is reduced.

3. The method of claim 2, wherein, A similarity matrix W between the image samples is computed 0 The specific formula is as follows: where W 0 (i,j) denotes the similarity matrix between the i-th and j-th image samples after adjustment, σ is the parameter of the Gaussian kernel function, and q e (0, 1] denotes the adjustment coefficient of the similarity.

4. The method of claim 1, wherein, The probability transition matrix P between image samples is obtained by row normalization 0 = (disg(W 0 1)) -1 W 0 , where 1 represents a vector with all element values being 1, diag(·) represents a diagonal vector operation of a matrix; and the membership matrix of each image data to different components is calculated as follows: wherein and respectively represent the absorbed probability of the constrained and unconstrained image data, I being the identity matrix, and P 0 is a block matrix inside, represents the probability transition matrix of the unconstrained image data samples to themselves, represents the probability transition matrix between the unconstrained image data samples and the constrained image data samples.

5. The method of claim 1, wherein, The similarity matrix S between the components 0 =(F 0 ) T W 0 F 0 The component similarity matrix is ​​adjusted using constraint information to exponentially amplify or reduce the similarity between constrained components. If the constrained image samples contained in a component satisfy... Constraints amplify similarity if the constrained image samples contained in the component satisfy... Constraints reduce similarity; the cluster indicator matrix U of each component in low-dimensional space 0 From Q 0 After eigenvalue decomposition, the eigenvectors corresponding to the smallest eigenvalues ​​of the first preset number are used to form the cluster indicator matrix of the image data samples in the low-dimensional space, which is G. 0 =F 0 ×U 0 .

6. The method of claim 1, wherein, similarity matrix W between all image samples at time t 0:t The structure is as follows: wherein is the similarity matrix between all image samples at the previous time, and W 0:0 0 , and is the similarity matrix between all image data samples at the previous time and the newly added image data samples at the current time, is the similarity matrix between the newly added image data samples at the current time; W 0:t is adjusted according to the newly added constraint condition, and the similarity between the image data samples satisfying the newly added constraint condition is enlarged or reduced in an exponential form, if the constraint is satisfied , the similarity is enlarged, and if the constraint is satisfied , the similarity is reduced.​ 7. The method of claim 1, wherein, Then the membership matrix F of the image sample pair component is incrementally constructed in the way of block matrix 0:t , as follows: wherein and respectively represent the membership matrix of the constrained image data and the unconstrained image data to the component, and respectively represent the membership matrix of all the constrained image data and the unconstrained image data to the component at the previous time, when t = 1 and respectively represent the membership matrix of the unconstrained sample in the new image data at the current t time to the component at the historical time, respectively represent the membership matrix of the unconstrained sample in the image data at the historical time to the new component at the current t time, respectively represent the membership matrix of the unconstrained sample in the new image data sample to the component, and the calculation method is as follows: wherein denotes the transition probability matrix between all unconstrained image samples at the previous time instant, denotes the transition probability matrix between all unconstrained image samples at the previous time instant and the newly added constrained image samples at the current time instant, denotes the transition probability matrix between the newly added unconstrained image samples at the current time instant, denotes the transition probability matrix between the newly added unconstrained image samples at the current time instant and the constrained image samples.

8. The method of claim 1, wherein, calculating the clustering result Y of all image samples at time t 0:t , and specifically comprising: First, the similarity matrix S between components is calculated 0:t = (F O:t ) T W 0:t F 0:t , and the constraint information is combined to adjust the similarity between components affected by the constraint in the form of amplification or reduction of the index, if the component contains the constrained image samples satisfying constraint, the similarity is amplified, if the component contains the constrained image samples satisfying constraint, the similarity is reduced; The probability transition matrix Q between the components is then obtained by row normalization O:t = (diag(S 0:t 1)) -1 S 0:t ; Assuming that the number of clusters does not change during the incremental process of data, the cluster centers of each cluster at the current time are calculated using the clustering results at the previous time wherein denotes the kth cluster at time t-1, and the cluster center set is further obtained Then calculate the cluster representative component closest to each cluster center at the current time, for representing each cluster: wherein represents a cluster representative component of the kth cluster; Then label propagation of component layer is performed, and the selected cluster representative component is set For the random walk, the absorbing state is the cluster representative component, and the remaining non-cluster representative components are the transition state. When the steady-state distribution is reached, the membership matrix R of all components to each cluster is 0:t is represented as: and denote the membership of each cluster for the cluster representative component and the non-cluster representative component, respectively, and Q = Q 0:t the block matrix inside, denotes the inter transition probability matrix between the non-cluster representative components, denotes the transition probability matrix from the cluster representative component to the non-cluster representative components; Finally, the membership matrix G of all image samples to clusters at the current time is calculated 0:t = F 0:t × R 0:t , and clustering is performed to obtain the clustering result Y of all image samples at the current time 0:t .

9. A computer system comprising a memory, a processor, and a computer program / instructions stored on the memory and executable on the processor, wherein, The computer program / instructions, when executed by the processor, implement the steps of the incremental semi-supervised image clustering method based on double-layer label propagation according to any one of claims 1-8.

10. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions, when executed by the processor, implement the steps of the incremental semi-supervised image clustering method based on double-layer label propagation according to any one of claims 1-8.

Images