Self-decoupling driven single-step multi-view image clustering method

Abstract

The invention discloses a self-decoupling driven single-step multi-view image clustering method, and the method comprises the steps: determining a corresponding sub-view sample matrix through an image data set based on each view, and enabling the sub-view sample matrix to represent all sample features under each sub-view; constructing a graph backtracking function based on consistent labels; determining a self-decoupling target function of the core sub-descriptor based on the core sub-descriptor; the weights of an anchor point matrix sample label matrix, an anchor point label matrix and all views in the single-step multi-view image clustering function are lost through alternate optimization until an objective function is converged, and a single-step multi-view image clustering algorithm is obtained; and processing the view image data based on a single-step multi-view image clustering algorithm to obtain a clustering result of the view image data, according to the invention, based on a backtracking strategy, the graph structure can be dynamically adjusted to obtain an optimal solution, and single-step multi-view image clustering is realized to improve the clustering precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image clustering, and more particularly to a self-decoupling driven single-step multi-view image clustering method, device, medium and equipment. BACKGROUND

[0002] The continuous innovation of digital signal processing can effectively improve the visual feature information content, thereby benefiting the analysis accuracy of downstream tasks such as image clustering. Among them, image clustering is a classic algorithm in the field of unsupervised learning, and when it is applied to multimedia data mining, how to extract cross-view consensus features becomes a technical key. Therefore, the traditional multi-view image clustering algorithm follows the process of "original composition-representation fusion-label learning", thereby realizing consistent semantic extraction. However, under the condition of lack of supervised information, the optimization of each module is independent and difficult to dynamically adjust to improve the task gain of label learning, which limits the accuracy of the multi-view image clustering algorithm.

[0003] In the prior art one, matrix decomposition and mean clustering are integrated into a unified framework to overcome the limitation that label learning relies on post-processing. In the prior art two, the similarity matrix of each view is split into a consistent graph and a difference graph, the consistent graph is fused by maximizing the difference, and the representation learning and label learning are included in a unified optimization framework based on spectral rotation. The prior art three combines subspace learning to realize tensor update under low rank constraint to mine high-order correlation and dynamically update the graph structure. However, in the existing multi-view image clustering algorithm, the collaborative optimization of graph learning and label learning is easily ignored, which makes it difficult to evaluate the task gain of label learning by graph learning. SUMMARY

[0004] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the first aspect of the present application proposes a self-decoupling driven single-step multi-view image clustering method, comprising:

[0005] The point cloud and image feature fusion method based on bird's eye view hybrid coding: input an image dataset containing multiple views, and determine the corresponding sub-view sample matrix based on the image dataset of each view, wherein the sub-view sample matrix represents the features of all samples under each sub-view;

[0006] Decompose the sub-view sample matrix to obtain a sample label matrix and an anchor point label matrix, and construct a graph backtracking function based on consistent labels by minimizing the structure loss between the sample label matrix, the anchor point label matrix and the affinity matrix of each view;

[0007] Solve the graph backtracking function to obtain dynamic affinity matrices of each view, self-decouple the affinity matrices of each view, generate corresponding core sub-descriptors, and determine a target function of core sub-descriptor self-decoupling based on the core sub-descriptors;

[0008] Based on the absolute minimum criterion of the objective function of the core sub-descriptor self-decoupling, a single-step multi-view image clustering function is constructed. The single-step multi-view image clustering function is used as the objective function. Under each sub-view, the anchor point matrix, sample label matrix, anchor label matrix and loss weight of each view in the single-step multi-view image clustering function are alternately optimized until the objective function converges, thus obtaining the single-step multi-view image clustering algorithm.

[0009] The single-step multi-view image clustering algorithm is used to process view image data to obtain the clustering results of the view image data.

[0010] Optionally, the expression for the graph backtracking function based on consistent labels is:

[0011]

[0012] Among them, B v Let F ∈ R be the affinity matrix for each view. n×c Let G be the sample label matrix, and G ∈ R. t×c Anchor label matrices F, G∈Ind indicate that both the sample label matrix F and the anchor label matrix G satisfy discrete constraints.

[0013] Optionally, the expression for the objective function of the core sub-descriptor self-decoupling is:

[0014]

[0015] Among them, ||·|| 2 V represents the L2 norm, and V represents the subview number.

[0016] Optionally, the expression for the single-step multi-view image clustering algorithm is:

[0017]

[0018] Among them, the latent variable a v Let represent the loss weight of the v-th view, and

[0019] Optionally, the step of alternately optimizing the anchor matrix, sample label matrix, anchor label matrix, and loss weights for each view in the single-step multi-view image clustering function includes:

[0020] Keep the other variables fixed and update the anchor matrix;

[0021] Keep the other variables fixed and update the anchor label matrix;

[0022] With the other variables fixed, update the sample label matrix;

[0023] Keep the other variables fixed and update the view weight vector;

[0024] Repeat the above steps until the single-step multi-view image clustering function decreases by less than the preset convergence threshold.

[0025] Optionally, it also includes:

[0026] The anchor point matrix under each subview is initialized by randomly generating column vectors. The anchor point matrix satisfies the orthogonality constraint, and the number of dimensions of the anchor point matrix matches the feature dimensions of the corresponding subview.

[0027] The sample label matrix and anchor label matrix are initialized by randomly generating a matrix of column vectors that satisfy discrete constraints.

[0028] Optionally, fixing the remaining variables and updating the anchor matrix includes:

[0029] Based on the optimization subproblem expression of the anchor matrix:

[0030]

[0031] Under different subviews, determine the update function expression for the anchor point matrix of the corresponding view:

[0032]

[0033] Among them, Z v The optimal solution is expressed as And U v and V v They represent the values ​​of GF respectively. T X v The left and right singular values ​​obtained by performing singular value decomposition;

[0034] Traverse each view and calculate the optimal solution of the anchor point matrix to obtain the updated anchor point matrix for each subview.

[0035] Optionally, fixing the remaining variables and updating the anchor label matrix includes:

[0036] The optimization problem expression for the anchor label matrix is:

[0037]

[0038] make The expression for the optimal solution of the anchor tag matrix is:

[0039]

[0040] Traverse each view and calculate the optimal solution for the anchor label matrix to obtain the updated anchor label matrix.

[0041] Optionally, fixing the remaining variables and updating the sample label matrix includes:

[0042] The optimization problem of the sample label matrix is ​​expressed as follows:

[0043]

[0044] make The expression for the optimal solution of the sample label matrix is:

[0045]

[0046] Traverse each view and calculate the optimal solution for the sample label matrix to obtain the updated sample label matrix.

[0047] Optionally, fixing the remaining variables and updating the view weight vector includes:

[0048] The expression for view weight is: Iterate through each view and calculate the view weight to obtain the updated view weight.

[0049] This application provides a self-decoupled, single-step multi-view image clustering method. Compared with existing technologies, its advantages are as follows: Unlike traditional multi-view image clustering algorithms that rely on manual parameter settings to achieve graph learning within the original sample space, this application uses a backtracking function and discrete sample labels to achieve sparse graph learning without hyperparameters, and can dynamically adjust the graph structure to obtain the optimal solution. Compared to traditional multi-view image clustering algorithms that rely on K-means clustering to generate anchor points and construct static graphs, this application can automatically generate affinity matrices for each view through anchor points, thereby gradually reducing feature loss between the reconstructed representation and the original samples, ultimately achieving single-step multi-view image clustering to improve clustering accuracy. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of this application, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0051] Figure 1 A flowchart of a self-decoupled driven single-step multi-view image clustering method provided in an embodiment of this application;

[0052] Figure 2 A flowchart illustrating the alternating optimization process of the self-decoupled driven single-step multi-view image clustering method provided in this application embodiment;

[0053] Figure 3 This is an algorithm block diagram of the self-decoupled driven single-step multi-view image clustering method provided in the embodiments of this application;

[0054] Figure 4 The graphs show the descent of the objective function value and the improvement of the clustering index on the MSRC multiview image dataset for the self-decoupled driven single-step multiview image clustering method provided in the embodiments of this application. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0056] This specification provides the operational steps for the methods described in the embodiments or flowcharts, but may include more or fewer operational steps based on conventional or non-inventive labor. In actual system or server product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0057] To address the limitation of traditional multi-view clustering algorithms in collaboratively extracting consistent semantics, this application proposes a self-decoupled, single-step multi-view image clustering method. Unlike methods that perform graph learning in the original feature space, this application utilizes a backtracking strategy to achieve clear graph structure mining in the discrete label space. Based on this, the original sample features are decomposed into core sub-descriptors and a coupling correlation matrix to achieve inverse representation reconstruction. The coupling correlation matrix describes the similarity relationship between the original sample features and the core sub-descriptors. Thus, by minimizing the feature loss between the original samples and the reconstructed representation, this application achieves dynamic graph updates and variable collaborative optimization, thereby realizing single-step multi-view image clustering. Since model optimization is susceptible to being misled by extreme feature losses, this application designs a loss function based on the absolute minimum criterion and implements flexible view fusion under a weight self-learning strategy, ultimately improving the accuracy of multi-view clustering.

[0058] refer to Figure 1 and Figure 3 One embodiment of this application provides a self-decoupled, single-step multi-view image clustering method. The execution process of the method can be as follows:

[0059] S10, Input an image dataset containing multiple views, and determine the corresponding subview sample matrix based on the image dataset of each view, where the subview sample matrix represents the features of all samples under each subview;

[0060] Specifically, multi-view image data is imported into an algorithm for cluster analysis, and the number of sample labels is pre-set to c. Furthermore, sample data with n samples and V views is represented as follows: Wherein, the subview sample matrix d represents the features of all samples under the v-th subview and the information of a single image corresponding to the column vector. v This represents the number of feature dimensions of the sample under the v-th subview.

[0061] For example, when obtaining a multi-view image dataset, it is necessary to first clarify the basic data information, including the number of samples, the number of sub-view feature dimensions, and the number of image label categories. For multi-view image data... In terms of existence The number of samples is n, the number of subview feature dimensions is d, and the number of image label categories is c. The multiview image dataset MSRC is selected as the clustering example. It has 210 samples, contains 6 subview data, and the number of feature dimensions for each view are 24, 576, 512, 256, and 254, respectively, and the number of image label categories is 10.

[0062] S20, decompose the subview sample matrix to obtain the sample label matrix and anchor label matrix. By minimizing the structural loss between the sample label matrix, the anchor label matrix and the common anchor graph, construct a graph backtracking function based on consistent labels.

[0063] Considering that the number of connected components in a graph equals the number of data clusters is a necessary condition for constructing an ideal graph, unlike traditional multi-view clustering algorithms that learn graphs based on sample feature similarity in the original space, this application implements sparse graph backtracking based on discrete labels. Furthermore, to ensure that the labeled reconstructed graph accurately reflects intra-cluster similarity, this application implements dynamic graph backtracking based on discrete labels. The sample label matrix is ​​defined as F∈R. n×c The anchor label matrix is ​​G∈R m×c And the common anchor point graph is B∈R n×m Then, graph backtracking based on consistent labels can be written as:

[0064] B = FG T ,stF,G∈Ind

[0065] Where F, G∈Ind indicates that both the sample label matrix and the anchor label matrix satisfy the discrete constraint, that is, there is exactly one non-zero element in the row vector that is assigned a value of 1. Therefore, this algorithm can achieve sparse graph structure backtracking based on discrete labels, and its effectiveness depends on the quality of the common anchor graph. Specifically, the effectiveness depends on the quality of the row vectors of the sample label matrix and the anchor label matrix. i and g j When the indices of non-zero elements are consistent, the elements in the dynamic anchor graph matrix That is, the i-th sample and the j-th anchor point are correlated. Where, f i =[f i1 ,f i2 ,...,f ik ,...,f ic ]∈R 1×c g j =[g j1 ,g j2 ,...,g jk ,...,g jc ]∈R 1×c Furthermore, the discrete constraint ensures that the vector has exactly one non-zero element, and that the indices of the non-zero elements are consistent, implying that f ik ≠0 and g jk ≠0 and all other elements are 0, so k is the index of the non-zero element.

[0066] S30, solve the graph backtracking function to obtain the dynamic common anchor graph, decouple the common anchor graph, generate the corresponding core sub-descriptor, and determine the objective function of core sub-descriptor self-decoupling based on the core sub-descriptor;

[0067] Traditional algorithms typically first use K-means clustering to select the anchor point Z. v A common anchor graph B is constructed using the K-nearest neighbor method. However, the quality of the statically constructed affinity matrix is ​​difficult to adjust dynamically, and the setting of the number of nearest neighbors in the K-nearest neighbor method requires human intervention and lacks prior guidance in unsupervised scenarios. Therefore, this application implements the anchor descriptor Z. v The dynamic generation and iterative updating of B are achieved. Combining the aforementioned graph backtracking strategy based on discrete labels, the objective function for the self-decoupling of the core sub-descriptor can be written as:

[0068]

[0069] Among them, the subview anchor point descriptor Z is adaptively generated based on the dynamic anchor point graph B obtained from backtracking. v Furthermore, it achieves the reconstruction of subview image features from anchor point descriptors, i.e., BZ. v →X v Therefore, the enhanced effectiveness of dynamic anchor graphs and subview anchor descriptors can be achieved by reducing reconstruction losses.

[0070] S40. Based on the absolute minimum criterion of the objective function of anchor point decoupling, a single-step multi-view image clustering function is constructed. The single-step multi-view image clustering function is used as the objective function. Under each sub-view, the anchor point descriptor matrix, sample label matrix, anchor point label matrix and loss weight of each view in the single-step multi-view image clustering function are alternately optimized until the objective function converges, thus obtaining the single-step multi-view image clustering algorithm.

[0071] In one embodiment of this application, the self-decoupled driven single-step multi-view image clustering method further includes...

[0072] The anchor point descriptor matrix under each subview is initialized by randomly generating orthogonal row vectors, so that the anchor point descriptor matrix satisfies the orthogonality constraint and the number of dimensions of the anchor point descriptor matrix matches the feature dimension of the corresponding subview image.

[0073] The sample label matrix and anchor label matrix are initialized by randomly generating row vectors that satisfy discrete constraints.

[0074] Considering the differences in the contribution of information from different views, this application can achieve adaptive adjustment of view weights. Finally, the self-decoupled, single-step multi-view image clustering method function can be written as:

[0075]

[0076] Among them, the latent variable a v Let represent the loss weight of the v-th view, and specify:

[0077]

[0078] Among them, the label matrix and feature matrix that satisfy discrete constraints can directly obtain sample label information through representation decoupling without the need for other post-processing methods.

[0079] In one embodiment of this application, the process of alternately optimizing the anchor descriptor matrix, sample label matrix, anchor label matrix, and loss weights of each view in the single-step multi-view image clustering function may include the following:

[0080] With the other variables fixed, update the anchor descriptor matrix;

[0081] Keep the other variables fixed and update the anchor label matrix;

[0082] With the other variables fixed, update the sample label matrix;

[0083] Keep the other variables fixed and update the view weight vector;

[0084] Repeat the above steps until the single-step multi-view image clustering function decreases by less than the preset convergence threshold.

[0085] refer to Figure 2 It should be noted that the variables to be updated involved in this application include the anchor point descriptor matrix under each subview. Sample label matrix F, anchor label matrix G, and loss weights for each view To optimize the objective function, this application employs an alternating optimization strategy, where optimizing one variable while keeping the others constant. Therefore, model optimization can be transformed into solving for... F, G, and the subproblems of each latent variable. The convergence criterion for the objective function is that the objective function value is below the convergence threshold, and the convergence threshold is uniformly set to 1×10. -2 If the decrease in the objective function is below the convergence threshold, optimization stops and the sample label matrix is ​​output; if the decrease in the objective function is above the convergence threshold, iterative optimization continues.

[0086] Specifically, for updating the anchor point descriptor matrix under each subview

[0087] This application employs an alternating optimization strategy to optimize the objective function. Therefore, the optimization subproblem of the anchor point descriptor matrix under each subview can be written in the following form:

[0088]

[0089] Among them, the anchor point descriptor matrix Z under different subviews v The updates are independent of each other, and therefore the update function of the anchor point descriptor matrix under the corresponding view can be finally written in the following form:

[0090]

[0091] Among them, Z v The optimal solution can be represented as Z v =U v V v T And U v and V v They represent the values ​​of GF respectively. T X v The left and right singular values ​​are obtained by performing singular value decomposition. By traversing each view and calculating the optimal solution for the anchor descriptor matrix, the anchor descriptor matrix for each subview can be completed. Update.

[0092] For updating the anchor label matrix G:

[0093] The optimization problem of the anchor label matrix G can be transformed into the following form:

[0094]

[0095] Among them, let The optimal solution for the anchor label matrix can then be written as:

[0096]

[0097] For updating the sample label matrix F:

[0098] Similarly, the optimization problem of the sample label matrix F can be transformed into the following form:

[0099]

[0100] Among them, let The optimal solution for the sample label matrix can then be written as:

[0101]

[0102] For updating the loss weights of each view

[0103] Loss weights for each view The update of the loss weight β for label propagation follows the formula below:

[0104]

[0105] Calculate the reduction value of the objective function and compare it with the convergence threshold. If the reduction value of the objective function is lower than the convergence threshold, stop the optimization and output the sample fuzzy label matrix; if the reduction value of the objective function is higher than the convergence threshold, repeat the iterative optimization. The reduction value of the objective function can be expressed as:

[0106] loss i =|obj i -obj i-1 |

[0107] Where, loss i Let obj represent the decrease in the objective function corresponding to the i-th iteration of optimization. i Let represent the objective function value after the i-th iteration of optimization, |·| denotes the absolute value operation, and the convergence threshold for the algorithm in this application is set to 1×10. -2 Based on the final output sample fuzzy label matrix, the predicted label Y can be obtained by assigning the samples to the corresponding data clusters according to the maximum membership probability. pre .

[0108] refer to Figure 4 By comparing the true label Y and the predicted label Y pre This application achieves a clustering accuracy (ACC) of 84.76%, a clustering normalized mutual information (NMI) of 73.25%, and an adjusted Land coefficient (ARI) of 68.27% on the MSRC dataset, significantly improving the clustering accuracy of multi-view image data.

[0109] S50, based on the single-step multi-view image clustering algorithm, processes the view image data to obtain the clustering results of the view image data.

[0110] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0111] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0112] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A self-decoupled, single-step multi-view image clustering method, characterized in that, include: The input consists of an image dataset containing multiple views, and the corresponding subview sample matrix is ​​determined based on the image dataset of each view. The subview sample matrix represents the features of all samples under each subview. The sample matrix of subviews is decomposed to obtain the sample label matrix and the anchor label matrix. By minimizing the structural loss between the sample label matrix, the anchor label matrix and the affinity matrix of each view, a graph backtracking function based on consistent labels is constructed. Solve the graph backtracking function to obtain the dynamic affinity matrix of each view. Perform self-decoupling on each view affinity matrix to generate the corresponding core sub-descriptor, and determine the objective function of core sub-descriptor self-decoupling based on the core sub-descriptor. Based on the absolute minimum criterion of the objective function of the core sub-descriptor self-decoupling, a single-step multi-view image clustering function is constructed. The single-step multi-view image clustering function is used as the objective function. Under each sub-view, the anchor point matrix, sample label matrix, anchor label matrix and loss weight of each view in the single-step multi-view image clustering function are alternately optimized until the objective function converges, thus obtaining the single-step multi-view image clustering algorithm. The single-step multi-view image clustering algorithm is used to process view image data to obtain the clustering results of the view image data.

2. The self-decoupled driven single-step multi-view image clustering method as described in claim 1, characterized in that, The expression for the graph backtracking function based on consistent labels is: Among them, B v Let F ∈ R be the affinity matrix for each view. n×c Let G be the sample label matrix, and G ∈ R. t×c Anchor label matrix F, G∈Ind indicates that both sample label matrix F and anchor label matrix G satisfy discrete constraints.

3. The self-decoupled driven single-step multi-view image clustering method as described in claim 1, characterized in that, The expression for the objective function of the core sub-descriptor self-decoupling is: Among them, ||·|| 2 V represents the L2 norm, and V represents the subview number.

4. The self-decoupled driven single-step multi-view image clustering method as described in claim 1, characterized in that, The expression for the single-step multi-view image clustering algorithm is: Among them, the latent variable a v Let represent the loss weight of the v-th view, and 5. The self-decoupled driven single-step multi-view image clustering method as described in claim 1, characterized in that, The method of alternating optimization of the anchor matrix, sample label matrix, anchor label matrix, and loss weights for each view in the single-step multi-view image clustering function includes: Keep the other variables fixed and update the anchor matrix; Keep the other variables fixed and update the anchor label matrix; With the other variables fixed, update the sample label matrix; Keep the other variables fixed and update the view weight vector; Repeat the above steps until the single-step multi-view image clustering function decreases by less than the preset convergence threshold.

6. The self-decoupled driven single-step multi-view image clustering method as described in claim 4, characterized in that, Also includes: The anchor point matrix under each subview is initialized by randomly generating column vectors. The anchor point matrix satisfies the orthogonality constraint, and the number of dimensions of the anchor point matrix matches the feature dimensions of the corresponding subview. The sample label matrix and anchor label matrix are initialized by randomly generating a matrix of column vectors that satisfy discrete constraints.

7. The self-decoupled driven single-step multi-view image clustering method as described in claim 5, characterized in that, The step of fixing the remaining variables and updating the anchor matrix includes: Based on the optimization subproblem expression of the anchor matrix: Under different subviews, determine the update function expression for the anchor point matrix of the corresponding view: Among them, Z v The optimal solution is expressed as And U v and V v They represent the values ​​of GF respectively. T X v The left and right singular values ​​obtained by performing singular value decomposition; Traverse each view and calculate the optimal solution of the anchor point matrix to obtain the updated anchor point matrix for each subview.

8. The self-decoupled driven single-step multi-view image clustering method as described in claim 5, characterized in that, The step of fixing the remaining variables and updating the anchor label matrix includes: The optimization problem expression for the anchor label matrix is: make The expression for the optimal solution of the anchor tag matrix is: Traverse each view and calculate the optimal solution for the anchor label matrix to obtain the updated anchor label matrix.

9. The self-decoupled driven single-step multi-view image clustering method as described in claim 5, characterized in that, The step of fixing the remaining variables and updating the sample label matrix includes: The optimization problem of the sample label matrix is ​​expressed as follows: make The expression for the optimal solution of the sample label matrix is: Traverse each view and calculate the optimal solution for the sample label matrix to obtain the updated sample label matrix.

10. The self-decoupled driven single-step multi-view image clustering method as described in claim 5, characterized in that, The step of fixing the remaining variables and updating the view weight vector includes: The expression for view weight is: Iterate through each view and calculate the view weight to obtain the updated view weight.

Images