Incomplete multi-view large-scale animal image clustering method based on attention fusion depth anchor map guidance
By using an attention-based deep anchor map-guided approach, the problem of coarse feature completion and fusion in multi-view clustering is solved, achieving high accuracy and robust clustering results under large-scale incomplete data.
Patent Information
- Application Number
- CN202511100801.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-07
- Publication Date
- 2025-11-21
AI Technical Summary
Existing multi-view clustering methods suffer from coarse feature completion and fusion when dealing with large-scale incomplete data. Explicit completion is prone to introducing noise, clustering optimization lacks depth guidance, and the balance between cross-view semantic consistency and view specificity is poor, leading to a decrease in clustering accuracy.
We adopt an attention-fusion-guided deep anchor map approach, which extracts the embedding representation from the perspective through a graph convolution-attention co-encoder, performs feature completion using an implicit completion decoder, constructs a deep anchor map for clustering optimization, and finally obtains global features and optimizes the clustering results through spectral clustering.
It achieves higher clustering accuracy and robustness under different missing rates, effectively mines the potential cluster structure of data, reduces explicit interpolation noise, and improves clustering performance.
Smart Images
Figure CN120997546A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of computer vision and biometric recognition in artificial intelligence, and specifically relates to a method for clustering non-holonomic multi-view large-scale animal images based on attention fusion deep anchor graph guidance. Background Technology
[0002] In the fields of computer vision and pattern recognition, clustering of large-scale, incomplete multi-view animal images is an extremely challenging task. With the development of remote sensing and related image acquisition technologies, a large amount of multi-view animal image data has emerged. However, this data often presents an incomplete state due to limitations in acquisition equipment and environmental interference. Existing multi-view clustering methods have the following shortcomings when processing large-scale incomplete data: First, feature completion and fusion are coarse; explicit completion easily introduces noise and makes it difficult to generate accurate and unified representations. Second, clustering optimization lacks deep guidance, failing to construct anchor point associations based on data distribution, resulting in weak adaptability to large-scale data. Third, there is a poor balance between cross-view semantic consistency and view-specificity, leading to a significant drop in clustering accuracy under high missing rates. Therefore, there is an urgent need for a clustering method that can effectively process incomplete multi-view data, accurately fuse features, deeply optimize clustering, and adapt to large-scale animal image scenarios. Summary of the Invention
[0003] To address the aforementioned problems, this invention provides a method for clustering large-scale, incomplete multi-view animal images based on attention fusion and depth anchor map guidance. The method includes the following steps:
[0004] We use a mask matrix to extract the presence samples from each viewpoint of the original incomplete multi-view large-scale animal image dataset and construct a graph adjacency matrix for them.
[0005] The existing samples and their graph adjacency matrix from each viewpoint are input into the graph convolutional-attention coencoder to obtain the embedding representation from each viewpoint. Attention fusion is then performed to obtain a complete unified representation.
[0006] The unified representation is input into the view-specific implicit completion decoder to obtain the complete multi-view prediction samples after completion. The corresponding loss function is the reconstruction loss function with mask. The stochastic gradient descent algorithm is used to minimize this loss function and the training is carried out until convergence.
[0007] The converged unified representation is subjected to k-means clustering, the cluster centers are selected as anchor points, and a depth anchor map is constructed.
[0008] Global features are obtained by performing spectral clustering on the depth anchor map. The soft clustering assignment between the global features and their cluster centers is calculated using the t-distribution, and a clustering assignment similarity matrix is constructed.
[0009] By using the clustering assignment similarity matrix as the target distribution, a clustering distribution alignment loss function is constructed, resulting in an overall loss function for attention fusion guided by deep anchor maps, which further optimizes the unified representation.
[0010] Anchor graphs are constructed using the optimized unified representation, and spectral clustering is performed to obtain clustering results. The clustering accuracy is then calculated.
[0011] Furthermore, the existing samples in the aforementioned incomplete multi-view large-scale animal image dataset are defined as:
[0012]
[0013] in, X is the matrix of existing samples under viewpoint v. v E is the original incomplete sample matrix under viewpoint v. v It is the mask matrix under the viewpoint v, defined as follows:
[0014]
[0015] in, The matrix representing the existence of samples from viewpoint v. Does the data in the i-th column exist in the original incomplete sample matrix X? v The j-th column of the data, i,j∈{1,2,...,n}, represents the index of the instance sample. A graph adjacency matrix is constructed for the existing samples from each perspective, defined as follows:
[0016]
[0017] in, D is the graph adjacency matrix under viewpoint v. v It is the degree matrix under the viewpoint v, and its elements are Let v represent the similarity matrix from viewpoint v, and n represents the identity matrix under viewpoint v. v The similarity matrix represents the number of existing samples from viewpoint v. It is calculated from existing samples using the Gaussian kernel function:
[0018]
[0019] in, This represents the similarity between sample i and sample j from viewpoint v. The bandwidth parameter of sample i under viewpoint v is represented by i,j,k∈{1,2,…,n}, which represents the instance sample index.
[0020] Furthermore, the graph convolutional-attention coencoder is defined as follows:
[0021]
[0022] in, This indicates the existence of an embedding representation of a sample from viewpoint v. It is a graph convolutional network (GCN) encoder under viewpoint v. These are the network parameters under viewpoint v; attention is used to fuse the embedding matrices under each viewpoint to obtain a complete unified representation:
[0023]
[0024] in, Let v represent the projection feature matrix of the v-th view, which contains samples of views with missing features. It is a normalization factor. The number of observable views for the i-th sample was recorded. The attention weight tensor is obtained as follows:
[0025]
[0026] Where [...] denotes the horizontal concatenation of matrices, ψ MLP () represents a nonlinear mapping implemented by a multilayer perceptron (MLP), σ(·) is the sigmoid activation function, and τ is the temperature parameter.
[0027] Furthermore, the view-specific implicit completion decoder is defined as follows:
[0028]
[0029] in, This indicates a view-dependent implicit completion decoder. These are the corresponding decoder parameters. This represents the complete multi-view prediction sample after completion;
[0030] Furthermore, the masked reconstruction loss function is defined as follows:
[0031]
[0032] in, This represents the reconstruction loss function with masking, where V represents the number of views.
[0033] Furthermore, the depth anchor map is defined as follows:
[0034]
[0035] stZ≥0,(Z) T1 = 1.
[0036] in, This represents a set of anchor points, where m is the number of anchor points. This is a depth anchor map, where γ is the sparsity coefficient, ||·|| F It is a Frobinus norm.
[0037] Furthermore, the soft clustering assignment is defined as follows:
[0038]
[0039] in, Let represent the probability that the i-th sample belongs to the j-th cluster at viewpoint v, and let U represent the global feature obtained by spectral clustering of the depth anchor map. This indicates that there are global features of the samples from viewpoint v. μ represents the global feature of the i-th existing sample from viewpoint v. j This represents the cluster center of the j-th cluster;
[0040] Furthermore, the clustering assignment similarity matrix is defined as follows:
[0041] P v =C v ·(C v ) T ,
[0042] Among them, P v C represents the clustering assignment similarity matrix under viewpoint v. v This represents the soft clustering assignment matrix from viewpoint v.
[0043] Furthermore, the clustering distribution alignment loss function is defined as follows:
[0044]
[0045] Where KL(·||·) represents the KL divergence, Let be the embedding distribution matrix of the v-th view, defined as follows:
[0046]
[0047] in, It is an embedding representation that carries different view information after attention fusion, where i,j,k,l∈{1,2,…,n} represents the embedding sample index;
[0048] Furthermore, the overall loss function for the attention fusion based on depth anchor maps is composed of a masked reconstruction loss function. Alignment loss function with clustering distribution Composition, defined as follows:
[0049]
[0050] Where λ > 0 is the equilibrium parameter. It is the overall loss function for attention fusion guided by depth anchor maps.
[0051] Furthermore, based on the obtained clustering results, the clustering accuracy on the incomplete multi-view large-scale animal image dataset is calculated.
[0052] This invention provides a clustering method for large-scale, non-holonomic, multi-view animal images based on attention fusion and deep anchor map guidance, which has the following advantages:
[0053] (1) The method adopts a multi-view clustering framework, which makes full use of the perspective consistency and perspective difference information of multi-view animal image data, and can effectively mine the potential cluster structure of the data.
[0054] (2) The method adopts graph convolutional network-attention implicit completion technology, uses graph convolutional-attention co-encoder to achieve efficient dynamic fusion between viewpoints, so that the model can obtain more comprehensive knowledge and more discriminative information. The implicit completion decoder is used to reduce explicit interpolation noise, thereby achieving better clustering performance.
[0055] (3) The method adopts a deep anchor graph-guided self-supervised technique, uses the clustering assignment matrix as the target distribution, and further optimizes the learned embedding representation, achieving the best performance and robustness under different missing rates. Attached Figure Description
[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0057] Figure 1 This is a flowchart of the non-holonomic multi-view large-scale animal image clustering method based on attention fusion deep anchor map guidance provided by the present invention;
[0058] Figure 2 This is a schematic diagram of a non-complete, multi-view, large-scale animal image dataset; Detailed Implementation
[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.
[0060] Exemplary methods
[0061] like Figure 1 This invention provides a method for clustering large-scale, non-holonomic, multi-view animal images based on attention fusion and deep anchor map guidance. The method steps are as follows:
[0062] Step S110: First, input the original data matrix of the incomplete multi-view animal image data. Extract the existence samples from each viewpoint using a mask matrix and construct a graph adjacency matrix for them. The definition of the existence samples is as follows:
[0063]
[0064] in, X is the matrix of existing samples under viewpoint v. v E is the original incomplete sample matrix under viewpoint v. v It is the mask matrix under the viewpoint v, defined as follows:
[0065]
[0066] in, The matrix representing the existence of samples from viewpoint v. Does the data in the i-th column exist in the original incomplete sample matrix X? v The j-th column of the data, i,j∈{1,2,...,n}, represents the index of the instance sample. A graph adjacency matrix is constructed for the existing samples from each perspective, defined as follows:
[0067]
[0068] in, D is the graph adjacency matrix under viewpoint v. v It is the degree matrix under the viewpoint v, and its elements are Let v represent the similarity matrix from viewpoint v, and n represents the identity matrix under viewpoint v. v The similarity matrix represents the number of existing samples from viewpoint v. It is calculated from existing samples using the Gaussian kernel function:
[0069]
[0070] in, This represents the similarity between sample i and sample j from viewpoint v. The bandwidth parameter of sample i under viewpoint v is represented by i,j,k∈{1,2,…,n}, which represents the instance sample index.
[0071] Step S120: Input the existing samples and their graph adjacency matrix from each viewpoint into the graph convolutional-attention coencoder to obtain a complete unified representation. The graph convolutional-attention coencoder is defined as follows:
[0072]
[0073] in, This indicates the existence of an embedding representation of a sample from viewpoint v. It is a graph convolutional network (GCN) encoder under viewpoint v. These are the network parameters under viewpoint v. Attention is used to fuse the embedding matrices from each viewpoint to obtain a complete unified representation.
[0074]
[0075] in, Let v represent the projection feature matrix of the v-th view, which contains samples of views with missing features. It is a normalization factor. The number of observable views for the i-th sample was recorded. The attention weight tensor is obtained as follows:
[0076]
[0077] Where [...] denotes the horizontal concatenation of matrices, ψ MLP () represents a nonlinear mapping implemented by a multilayer perceptron (MLP), σ(·) is the sigmoid activation function, and τ is the temperature parameter.
[0078] Step S130: Input the unified representation into the view-specific implicit completion decoder to obtain the completed multi-view prediction samples. Minimize the reconstruction loss function using the stochastic gradient descent algorithm and train until convergence. The expression formula of the view-specific implicit completion decoder is as follows:
[0079]
[0080] in, This indicates a view-dependent implicit completion decoder. These are the corresponding decoder parameters. This represents the complete multi-view prediction sample after completion.
[0081] The reconstruction loss function is a masked reconstruction loss function, defined as follows:
[0082]
[0083] in, This represents the reconstruction loss function with masking, where V represents the number of views.
[0084] Step S140: Perform k-means clustering on the converged unified representation, select cluster centers as anchor points, and construct a depth anchor map. The depth anchor map is defined as follows:
[0085]
[0086] in, This represents a set of anchor points, where m is the number of anchor points. This is a depth anchor map, where γ is the sparsity coefficient, ||·|| F It is a Frobinus norm.
[0087] Step S150: Perform spectral clustering on the depth anchor map to obtain global features, calculate the soft clustering assignment between the global features and their cluster centers using the t-distribution, and construct a clustering assignment similarity matrix. The soft clustering assignment is defined as follows:
[0088]
[0089] in, Let represent the probability that the i-th sample belongs to the j-th cluster at viewpoint v, and let U represent the global feature obtained by spectral clustering of the depth anchor map. This indicates that there are global features of the samples from viewpoint v. μ represents the global feature of the i-th existing sample from viewpoint v. j This represents the cluster center of the j-th cluster.
[0090] The clustering assignment similarity matrix is defined as follows:
[0091] P v =C v ·(C v ) T (12)
[0092] Among them, P v C represents the clustering assignment similarity matrix under viewpoint v. v This represents the soft clustering assignment matrix from viewpoint v.
[0093] Step S160: Using the clustering assignment similarity matrix as the target distribution, construct the clustering distribution alignment loss function to obtain the overall loss function for attention fusion guided by the depth anchor map, and further optimize the unified representation. The clustering distribution alignment loss function is defined as follows:
[0094]
[0095] Where KL(·||·) represents the KL divergence, Let be the embedding distribution matrix of the v-th view, defined as follows:
[0096]
[0097] in, It is an embedding representation that carries different view information after attention fusion, where i,j,k,l∈{1,2,…,n} represents the embedding sample index.
[0098] The overall loss function for attention fusion guided by depth anchor maps is composed of a masked reconstruction loss function. Alignment loss function with clustering distribution Composition, defined as follows:
[0099]
[0100] Where λ > 0 is the equilibrium parameter. It is the overall loss function for attention fusion guided by depth anchor maps.
[0101] Step S170: Construct anchor graphs using the optimized unified representation and perform spectral clustering to obtain clustering results, and calculate clustering accuracy.
[0102] In this implementation, firstly, existing samples from each perspective are extracted from the original data, and a graph adjacency matrix is constructed for them. Both are then input into a graph convolutional-attention co-encoder to obtain a complete unified representation. Subsequently, the unified representation is input into an implicit completion decoder to complete training. A deep anchor map is constructed for the converged unified representation. After obtaining global features through spectral clustering, a clustering assignment similarity matrix is constructed, and then the unified representation is further optimized using the overall loss. Finally, the optimized unified representation is used to construct anchor maps and perform spectral clustering to obtain the results and calculate the clustering accuracy on the dataset.
[0103] To further illustrate, if a non-complete, multi-view, large-scale animal image dataset is clustered according to this implementation method, a clustering result with higher accuracy than most methods will be obtained.
[0104] Detailed Implementation Results
[0105] This implementation uses a publicly available large-scale multi-view animal image dataset and simulates an incomplete large-scale multi-view animal image dataset according to different proportions of missing views. The dataset details are described below:
[0106] The animal image dataset is an image dataset. This dataset has the following two perspectives:
[0107] Viewpoint 1 is the image viewpoint, which contains 37,322 images belonging to 50 animal categories. Each image has a category label and can be used for tasks such as image classification, recognition, detection, and segmentation.
[0108] Perspective 2 is the attribute perspective, which includes 85 numerical attributes for each animal category. These attributes describe the animal's appearance, behavior, ecology, and other characteristics, and can be used for attribute-based classification and zero-shot learning tasks.
[0109] To verify the superiority of this implementation method, it is compared with several existing clustering methods for incomplete multi-view animal image data, including GIMVC, GHICMC, FMVACC, PMIMC, and FIMVC_VIA. Five incomplete view ratios are simulated on a publicly available large-scale multi-view animal image dataset, and the clustering accuracy (ACC) of the above comparison methods is tested. The specific experimental results are shown in Table 1.
[0110] Table 1. Clustering accuracy (ACC) of the animal dataset.
[0111]
[0112]
[0113] The data comparison in the table above clearly shows that Ours achieves the best performance, significantly improving the clustering accuracy of incomplete multi-view animal image data. The quantitative results fully demonstrate the superiority of Ours, as it better captures both viewpoint consistency and viewpoint difference information in incomplete multi-view animal image data. While performing attention fusion, Ours integrates more discriminative and comprehensive information into the learned consistent representation, and further enhances cross-view semantic consistency through deep anchor maps, achieving optimal performance and stability under different missing rates. Extensive experiments demonstrate that this method outperforms existing methods.
[0114] This implementation proposes a clustering method for large-scale incomplete multi-view animal images based on attention fusion and depth anchor map guidance. It is used to perform clustering analysis on common incomplete multi-view animal image data. A graph convolutional-attention co-encoder is used to obtain a complete unified representation carrying information from different viewpoints. Then, depth anchor map guidance is used to further optimize the unified representation. Finally, clusters are obtained from the optimized unified representation. Experimental results on five different proportions of incomplete viewpoints in a publicly available large-scale animal image dataset show that this implementation has higher clustering accuracy and better performance compared to other methods.
[0115] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.
Claims
1. A clustering method for large-scale, nonholonomic, multi-view animal images based on attention fusion and depth anchor map guidance, characterized in that... The method includes the following steps: We use a mask matrix to extract the existing samples from each viewpoint of the original incomplete multi-view large-scale animal image dataset and construct a graph adjacency matrix for them. The existing samples and their graph adjacency matrix from each viewpoint are input into the graph convolutional-attention coencoder to obtain the embedding representation from each viewpoint. Attention fusion is then performed to obtain a complete unified representation. The unified representation is input into the view-specific implicit completion decoder to obtain the complete multi-view prediction samples after completion. The corresponding loss function is the reconstruction loss function with masking. The stochastic gradient descent algorithm is used to minimize this loss function and the training is carried out until convergence. The converged unified representation is subjected to k-means clustering, the cluster centers are selected as anchor points, and a depth anchor map is constructed. Global features are obtained by performing spectral clustering on the depth anchor map. The soft clustering assignment between the global features and their cluster centers is calculated using the t-distribution, and a clustering assignment similarity matrix is constructed. By using the clustering assignment similarity matrix as the target distribution, a clustering distribution alignment loss function is constructed, resulting in an overall loss function for attention fusion guided by deep anchor maps, which further optimizes the unified representation. Anchor graphs are constructed using the optimized unified representation, and spectral clustering is performed to obtain clustering results. The clustering accuracy is then calculated.
2. The method for clustering large-scale, non-holonomic, multi-view animal images based on attention fusion and deep anchor map guidance as described in claim 1, characterized in that, Existence samples from each viewpoint are extracted from the original incomplete multi-view large-scale animal image dataset using a mask matrix, and a graph adjacency matrix is constructed for them; the existence samples are defined as follows: in, X is the matrix of existing samples under viewpoint v. v E is the original incomplete sample matrix under viewpoint v. v It is the mask matrix under the viewpoint v, defined as follows: in, The matrix representing the existence of samples from viewpoint v. Does the data in the i-th column exist in the original incomplete sample matrix X? v The data in the j-th column, i,j∈{1,2,...,n}, represents the instance sample index; The graph adjacency matrix is defined as follows: in, D is the graph adjacency matrix under viewpoint v. v It is the degree matrix under the viewpoint v, and its elements are Let v represent the similarity matrix from viewpoint v, and n represents the identity matrix under viewpoint v. v The similarity matrix represents the number of existing samples from viewpoint v. It is calculated from existing samples using the Gaussian kernel function: in, This represents the similarity between sample i and sample j from viewpoint v. The bandwidth parameter of sample i under viewpoint v is represented by i,j,k∈{1,2,…,n}, which represents the instance sample index.
3. The method for clustering large-scale, nonholonomic, multi-view animal images based on attention fusion and deep anchor map guidance according to claim 1, characterized in that, The existing samples and their graph adjacency matrix from each viewpoint are input into a graph convolutional-attention coencoder to obtain the embedding representation for each viewpoint. The process is as follows: in, This indicates the existence of an embedding representation of a sample from viewpoint v. It is a graph convolutional network (GCN) encoder under viewpoint v. These are the network parameters under viewpoint v; attention is used to fuse the embedding matrices under each viewpoint to obtain a complete unified representation: in, Let v represent the projection feature matrix of the v-th view, which contains samples of views with missing features. It is a normalization factor. The number of observable views for the i-th sample was recorded. The attention weight tensor is obtained as follows: Where [...] denotes the horizontal concatenation of matrices, ψ MLP () represents a nonlinear mapping implemented by a multilayer perceptron (MLP), σ(·) is the sigmoid activation function, and τ is the temperature parameter.
4. The method for clustering large-scale, nonholonomic, multi-view animal images based on attention fusion and deep anchor map guidance according to claim 1, characterized in that, The unified representation is input into the view-specific implicit completion decoder to obtain the completed multi-view prediction samples. The corresponding loss function is the reconstruction loss function with masking. The view-specific implicit completion decoder is defined as follows: in, This indicates a view-dependent implicit completion decoder. These are the corresponding decoder parameters. This represents the complete multi-view prediction sample after completion; The masked reconstruction loss function is defined as follows: in, This represents the reconstruction loss function with masking, where V represents the number of views.
5. The method for clustering large-scale, non-holonomic, multi-view animal images based on attention fusion and deep anchor map guidance according to claim 1, characterized in that, Cluster centers with a unified representation are selected as anchor points, and a depth anchor map is constructed; the depth anchor map is defined as follows: in, This represents a set of anchor points, where m is the number of anchor points. This is a depth anchor map, where γ is the sparsity coefficient, ||·|| F It is a Frobinus norm.
6. The method for clustering large-scale, non-holonomic, multi-view animal images based on attention fusion and deep anchor map guidance according to claim 1, characterized in that, The t-distribution is used to calculate the soft clustering assignments between global features and their cluster centers, and a clustering assignment similarity matrix is constructed. The soft clustering assignment is defined as follows: in, Let represent the probability that the i-th sample belongs to the j-th cluster at viewpoint v, and let U represent the global feature obtained by spectral clustering of the depth anchor map. This indicates that there are global features of the samples from viewpoint v. μ represents the global feature of the i-th existing sample from viewpoint v. j This represents the cluster center of the j-th cluster; The constructed clustering assignment similarity matrix is defined as follows: P v =C v ·(C v ) T , Among them, P v C represents the clustering assignment similarity matrix under viewpoint v. v This represents the soft clustering assignment matrix from viewpoint v.
7. The method for clustering large-scale, nonholonomic, multi-view animal images based on attention fusion and deep anchor map guidance according to claim 1, characterized in that, By using the clustering assignment similarity matrix as the target distribution, a clustering distribution alignment loss function is constructed, resulting in the overall loss function for attention fusion guided by deep anchor maps. The clustering distribution alignment loss function is defined as follows: Where KL(·||·) represents the KL divergence, Let be the embedding distribution matrix of the v-th view, defined as follows: in, It is an embedding representation that carries different view information after attention fusion, where i,j,k,l∈{1,2,…,n} represents the embedding sample index; The overall loss function for attention fusion guided by depth anchor maps is composed of a masked reconstruction loss function. Alignment loss function with clustering distribution Composition, defined as follows: Where λ > 0 is the equilibrium parameter. It is the overall loss function for attention fusion guided by depth anchor maps.