A large-scale incomplete multi-view animal image clustering method based on mini-batch ensemble scaled dot product attention weight anchor map
Through the small batch integration of the scaling dot product attention weight anchor diagram method, the clustering problem of non-complete multi-view animal image data is solved, the clustering accuracy and efficiency are improved, and the perspective consistency and difference of the data set are used, and the clustering is enhanced. The robustness of the model.
Patent Information
- Application Number
- CN202410016589.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-04
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-01-04
AI Technical Summary
The existing multi-view clustering method cannot effectively process non-complete multi-view animal image data, resulting in clustering accuracy and inefficiency.
Using the small batch integrated scalable dot product attention weight anchor graph method, extract and normalize features through the viewing angle missing indication matrix, randomly sample anchor features, calculate the scalable dot product attention weight anchor graph, and perform weighted fusion, and finally, through rapid Spectral clustering results are obtained.
The clustering accuracy and efficiency of non-complete multi-view animal image data is improved, and the perspective consistency and difference of the data set are fully utilized, the computational complexity is reduced, and the robustness and clustering accuracy of the model are enhanced.
Smart Images

Figure CN117830680B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of animal image data processing in animal morphology and multimedia information processing, and specifically relates to a large-scale incomplete multi-view animal image clustering method based on a small batch integrated scaled dot product attention weight anchor graph. Background Art
[0002] In the field of animal morphology and multimedia information processing, animal image data is an important data. Multi-view analysis of animal image data can reveal implicit category information such as the characteristics of different parts of animals, differences and similarities between animals, etc. Among various multi-view data learning technologies, multi-view clustering is a commonly used technology. It can use information from multiple perspectives to perform unsupervised clustering of data and can be applied to a variety of actual animal classifications. However, in the process of acquiring multi-view animal image data, problems such as missing data or data imbalance may be encountered, resulting in different availability of each sample under different perspectives, thus causing the problem of incomplete multi-view animal data. Since existing multi-view clustering methods can only handle the clustering problem of complete multi-view animal data, effective novel technical methods are needed to solve the clustering problem of incomplete multi-view animal image data. Summary of the invention
[0003] In order to solve the above problems, the present invention provides a large-scale incomplete multi-view animal image clustering method with small batch integrated scaled dot product attention weight anchor map, characterized in that the method comprises the steps of:
[0004] The presence sample features of each view are extracted according to the view missing indication matrix, and the feature vector of each view is normalized so that they all have unit modulus.
[0005] For each viewing angle, randomly sample b groups of anchor features from existing sample features, each group of anchor features contains n a anchor points. The existing sample features are replicated b times to obtain multiple groups of existence features. Multiple groups of anchor features and multiple groups of existence features are stacked into anchor feature batches and existence feature batches respectively.
[0006] For each view, based on common tensor operations (such as tensor multiplication, tensor transposition, tensor element-by-element division, tensor softmax, etc.), the scaled dot product attention weight anchor map batch between the anchor feature batch and the existing feature batch is calculated. The average value of the anchor map batch is calculated to obtain the view-specific integrated anchor map.
[0007] According to the view missing indication matrix, the view-specific integrated anchor maps of all views are weightedly fused to obtain the view common anchor map.
[0008] Repeat the first three steps t times, calculate the average of the t perspective common anchor graphs, and obtain the perspective common integrated anchor graph. Perform fast spectral clustering on the obtained perspective common integrated anchor graph to obtain the final clustering result, and calculate the clustering accuracy of the final clustering result.
[0009] Further, the missing indicator matrix M∈{0,1} n×m The presence sample features of each view extracted from a large-scale incomplete multi-view animal image dataset are defined as follows:
[0010]
[0011] in, Represents the set of sample features existing from all perspectives, is the existence feature matrix of view v. If the jth view of sample i exists, then M ij =1, otherwise M ij =0;n v is the number of samples, d v is the feature dimension, and m is the number of viewing angles. The feature vectors of each viewing angle are normalized so that they all have unit modulus length, and the expression formula is:
[0012] x i (v) ←x i (v) / ‖x i (v) ‖,i=1,2,...,n; v=1,2,...,m
[0013] Among them, i represents any sample, v represents any view, n is the number of samples, m is the number of view; ‖·‖ represents the modulus of the vector, x i (v) Represents the feature vector of sample i.
[0014] Furthermore, the plurality of groups of anchor features and the plurality of groups of existence features are stacked into an anchor feature batch and an existence feature batch, respectively. The expression formulas of the stacked anchor feature batch and existence feature batch are:
[0015]
[0016]
[0017]
[0018]
[0019] Among them, the matrix represents the randomly sampled set of anchor features, n arepresents the number of anchor points, b represents the number of batches, and tensor represents the anchor feature batch; random-sample(X,c) represents the uniform sampling of c samples from the feature matrix X; the matrix Indicates copied from The b group has characteristics, n v Indicates the number of samples, tensor Indicates the presence of feature batches.
[0020] Furthermore, the expression formula for calculating the scaled dot product attention weight anchor graph batch is:
[0021] Z (v) =softmax(X (v) ⊙(A (v) ) T / τ)
[0022] where τ>0 is the scaling factor, and the tensor Z (v) represents the scaled dot product attention weighted anchor map batch of view v, X (v) Indicates the presence of feature batches, A (v) Represents a batch of anchor features. Take any three tensors A, B, and C as an example. A⊙B=C is a tensor multiplication operation, which means that two three-dimensional tensors A and B are expanded in the first dimension respectively, and the slice matrices of all corresponding positions are multiplied by the matrix, and then stacked to obtain a new tensor C, which is defined as follows:
[0023] C i:: =A i:: ·B i::
[0024] Among them, C i:: , A i:: , B i:: Respectively represent the i-th slice matrix of tensor C, tensor A, and tensor B, and · represents general matrix multiplication. Softmax(X) is a tensor softmax operation, which means that the softmax operation is performed on all row vectors of all slice matrices after the three-dimensional tensor X is expanded according to the first dimension. It is defined as follows:
[0025]
[0026] S (v) =X (v) ⊙(A (v) ) T / τ
[0027] in, For the existence of feature batch X (v) and anchor feature batch A (v)Since all eigenvectors have unit modulus, S (v) The elements of A can be considered as cosine similarity. T is a tensor transpose operation, which means the operation of swapping the second and third dimensions of tensor A, that is, (A T ) ijk =A ikj . exp(·) and · / · are tensor element-by-element exponential function operations and tensor element-by-element division operations, respectively. For any three tensors A, B, and C, the tensor element-by-element exponential function operation is defined as exp(A) ijk =exp(A ijk ), the tensor element-wise division operation is defined as A ijk =B ijk / C ijk .
[0028] For the anchor chart batch Z (v) After expanding by the first dimension, the average value of all slice matrices is calculated, and the expression formula of the perspective-specific integrated anchor chart is obtained:
[0029]
[0030] in, Represents perspective-specific integrated anchor chart.
[0031] Furthermore, according to the view missing indication matrix, the view-specific integrated anchor graphs of all views are weightedly fused to obtain the view common anchor graph expression formula:
[0032]
[0033] Among them, M :,v represents the view vth column of the missing indicator matrix, Z represents the view common anchor graph, and Z (v) Represents the perspective-specific integrated anchor graph for perspective v.
[0034] Furthermore, the above three steps are repeated t times, and the average value of the obtained t perspective common anchor graphs is calculated to obtain the perspective common integrated anchor graph, which is expressed as follows:
[0035]
[0036] Among them, Z * Represents the common integrated anchor graph of the perspective, Z k represents the kth perspective common anchor map, and t represents the number of repetitions. The obtained perspective common integrated anchor map Z * Perform fast spectral clustering to obtain the final clustering result and calculate its clustering accuracy.
[0037] The present invention provides a large-scale incomplete multi-view animal image clustering method based on small batch integrated scaled dot product attention weight anchor map, which has the following advantages:
[0038] (1) The method adopts a multi-view clustering framework, which fully utilizes the view consistency and diversity of the incomplete multi-view animal image dataset and effectively mines the potential cluster structure.
[0039] (2) The method adopts a small batch processing sample feature technology. By processing anchor features and existence features in small batches and using hardware acceleration, the algorithm's running speed is improved, the running time is reduced, and the robustness of the model is improved.
[0040] (3) The method adopts the scaled dot product attention mechanism to learn the anchor graph edge weights, which improves the expressiveness of the anchor graph and better describes the intrinsic structure of the data. It does not need to solve the optimization problem, reduces the computational complexity, and improves the efficiency of the algorithm and the clustering accuracy and stability. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0042] Figure 1 It is a flow chart of a large-scale incomplete multi-view animal image clustering method based on a small batch integrated scaled dot product attention weight anchor map provided by the present invention;
[0043] Figure 2 It is a schematic diagram of some animal images; DETAILED DESCRIPTION
[0044] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.
[0045] Exemplary Methods
[0046] like Figure 1 The present invention provides a large-scale incomplete multi-view animal image clustering method based on a small batch integrated scaled dot product attention weight anchor map, and the method steps are as follows:
[0047] Step S110: First, according to the view missing indication matrix M∈{0,1} n×m , extract the sample features of each view in, Represents the set of sample features existing from all perspectives, is the existence feature matrix of view v. If the jth view of sample i exists, then M ij =1, otherwise M ij =0;n v is the number of samples, d v is the feature dimension, and m is the number of viewing angles. The feature vectors of each viewing angle are normalized so that they all have unit modulus length, and the expression formula is as follows:
[0048] x i (v) ←x i (v) / ‖x i (v) ‖,i=1,2,...,n; v=1,2,...,m
[0049] Among them, i represents any sample, v represents any view, n is the number of samples, m is the number of view; ‖·‖ represents the modulus of the vector, x i (v) Represents the feature vector of sample i.
[0050] Step S120: Randomly sample n existing sample features from each perspective a Anchor points are repeated b times to obtain multiple sets of anchor point features. The existing samples are copied b times to obtain multiple sets of existing features. The two are stacked to obtain the corresponding batches. The expression formula is as follows:
[0051]
[0052]
[0053] Among them, the matrix represents the randomly sampled set of anchor features, n a represents the number of anchor points, b represents the number of batches, and tensor represents the anchor feature batch; random-sample(X,c) represents the uniform sampling of c samples from the feature matrix X. The existing sample features are replicated b times to obtain multiple groups of existing features, and they are stacked into existing feature batches. The expression formula is as follows:
[0054]
[0055]
[0056] Among them, the matrix Indicates copied from The b group has characteristics, n v Indicates the number of samples, tensor Indicates the presence of feature batches.
[0057] Step S130: Calculate the scaled dot product attention weight anchor map batch between the anchor feature batch and the existing feature batch based on common tensor operations, and calculate the average value of the anchor map batch to obtain the perspective-specific integrated anchor map. The formula for calculating the scaled dot product attention weight anchor map batch is as follows:
[0058] Z (v) =softmax(X (v) ⊙(A (v) ) T / τ)
[0059] where τ>0 is the scaling factor, and the tensor Z (v) represents the scaled dot product attention weighted anchor map batch of view v, X (v) Indicates the presence of feature batches, A (v) Represents a batch of anchor features. Softmax(X) represents the softmax operation on all row vectors of all slice matrices after the three-dimensional tensor X is expanded by the first dimension. Its expression formula is:
[0060]
[0061] S (v) =X (v) ⊙(A (v) ) T / τ
[0062] in, For the existence of feature batch X (v) and anchor feature batch A (v) Since all eigenvectors have unit modulus, S (v) The elements of A can be considered as cosine similarity. T is a tensor transpose operation, which means the operation of swapping the second and third dimensions of tensor A, that is, (A T ) ijk =A ikj exp(·) and · / · are tensor element-by-element exponential function operations and tensor element-by-element division operations, respectively. (v) After expanding by the first dimension, the average value of all slice matrices is calculated to obtain the perspective-specific integrated anchor chart, which is expressed as follows:
[0063]
[0064] in, Represents perspective-specific integrated anchor chart.
[0065] Step S140: According to the view missing indication matrix, the view-specific integrated anchor graphs of all views are weightedly fused to obtain a view common anchor graph, which is expressed as follows:
[0066]
[0067] Among them, M :,v represents the view vth column of the missing indicator matrix, Z represents the view common anchor graph, and Z (v) Represents the perspective-specific integrated anchor graph for perspective v.
[0068] Step S150: Repeat steps S120-S140 multiple times, calculate the average value of the obtained t perspective common anchor graphs, and obtain the perspective common integrated anchor graph, which is expressed as follows:
[0069]
[0070] Among them, Z * Represents the common integrated anchor graph of the perspective, Z k represents the kth perspective common anchor map, and t represents the number of repetitions. The obtained perspective common integrated anchor map Z * Perform fast spectral clustering to obtain the final clustering result, and calculate the clustering accuracy of the final clustering result.
[0071] Through this implementation, first, the existing sample features of each perspective are extracted and normalized according to the perspective missing indication matrix, and then b groups of anchor features are randomly sampled therefrom, and the existing sample features are copied b times to obtain anchor feature batches and existing feature batches. Next, for each perspective, a scaled dot product attention weight anchor map batch between the anchor feature batch and the existing feature batch is calculated using tensor operations, and the scaled dot product attention weight anchor map batch is averaged to obtain a perspective-specific integrated anchor map. The perspective-specific integrated anchor maps of all perspectives are weighted and fused according to the perspective missing indication matrix to obtain a perspective common anchor map. Finally, this process is repeated t times, and the t perspective common anchor maps are averaged to obtain a perspective common integrated anchor map. The perspective common integrated anchor map is quickly spectrally clustered to obtain the final clustering result, and its clustering accuracy is calculated.
[0072] To further illustrate, assuming that an incomplete multi-view animal image dataset is clustered according to this embodiment, a clustering result with a higher clustering accuracy than most methods will be obtained.
[0073] Detailed description of the invention Results
[0074] This implementation uses a publicly available animal image dataset and simulates an incomplete animal image dataset according to different sample missing ratios. The details of the dataset are described as follows:
[0075] The animal image dataset is an image dataset. This dataset has the following two perspectives:
[0076] Perspective 1 is the image perspective, which contains 37,322 images belonging to 50 animal categories. Each image has a category label and can be used for image classification, recognition, detection, segmentation and other tasks.
[0077] Perspective 2 is the attribute perspective, which contains 85 numerical attributes for each animal category. These attributes describe the appearance, behavior, ecology and other characteristics of the animal, and can be used for tasks such as attribute-based classification and zero-shot learning.
[0078] In order to verify the superiority of this implementation (OURS), this implementation is compared with several existing methods for clustering animal image data, including AWMVC, BMVC, FMVCC, IMVC-CBG and other methods. The clustering accuracy (ACC) of these methods for five incomplete sample ratios on the above public datasets will be compared. The specific data comparison is shown in Table 1.
[0079] Table 1 Clustering accuracy (%) and running time (s) of animal dataset
[0080]
[0081] By comparing the data in the above table, it can be clearly seen that OURS achieves the best performance and takes the least running time, significantly improving the clustering accuracy of the incomplete multi-view animal image dataset. The quantitative results fully demonstrate the superiority of OURS, because OURS can better capture the perspective consistency information and perspective difference information in the incomplete multi-view animal image dataset. OURS constructs an anchor graph by integrating scaled dot product attention weights in small batches, thereby improving the experimental efficiency and enhancing the robustness to incomplete animal image data. A large number of experiments have shown that this method is superior to existing methods. Regarding the setting of the parameters of this embodiment, in all experiments, the number of anchor points n a It is set to 5c, where c is the number of clusters, and repeated b times to obtain multiple groups of anchor features, b = 20, and the scaling factor τ = 0.01.
[0082] This embodiment proposes a large-scale incomplete multi-view animal image clustering method based on a small batch integrated scaled dot product attention weighted anchor map, which is used to perform cluster analysis on incomplete animal image data. By randomly selecting multiple groups of anchor points from the existing samples of each view, anchor feature batches and existing feature batches are formed. The similarity between the anchor points and the existing features of each view is calculated using scaled dot product attention, and after obtaining the view-specific integrated anchor map, it is weightedly fused according to the missing indicator matrix to obtain the view common anchor map. Repeat multiple times, fuse multiple view common anchor maps to obtain the view common integrated anchor map, and then perform fast spectral clustering to obtain the final clustering result and the corresponding accuracy. The experimental results on five incomplete sample ratios of public animal data sets show that this embodiment has a higher clustering accuracy and better superiority than other methods.
[0083] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, and do not constitute a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or the equivalent forms of such scope and boundaries.
Claims
1. A large-scale incomplete multi-view animal image clustering method based on mini-batch ensemble scaled dot product attention weight anchor map, It is characterized in that The method comprises the steps of: S110. Extracting the existing sample features of each view according to the view missing indication matrix, that is, the sample data features actually available under the exclusive view, and normalizing the feature vectors of each view so that they all have unit modulus; S120. For each viewing angle, randomly sample b groups of anchor features from the existing sample features, each group of anchor features contains n a anchor points, replicate the existing sample features b times to obtain multiple groups of existence features, and stack the multiple groups of anchor point features and multiple groups of existence features into anchor point feature batches and existence feature batches respectively; S130. For each perspective, based on common tensor operations, including tensor multiplication, tensor transposition, tensor element-by-element division, and tensor softmax, a scaled dot product attention weight anchor map batch between the anchor feature batch and the existing feature batch is calculated, and an average value is calculated for the anchor map batch to obtain a perspective-specific integrated anchor map, wherein the attention weight anchor map batch is obtained by calculating the dot product between the anchor feature and the existing feature, and then applying scaling and softmax functions to the obtained dot product result, and the perspective-specific integrated anchor map is obtained by weighted averaging the obtained attention weight anchor map batch; S140. According to the perspective missing indication matrix, weighted fusion of the perspective-specific integrated anchor graphs of all perspectives is performed to obtain a perspective common anchor graph; S150. Repeat the first three steps t times, calculate the average of the t view common anchor graphs obtained, obtain the view common integrated anchor graph, perform fast spectral clustering on the obtained view common integrated anchor graph to obtain the final clustering result, and calculate the clustering accuracy of the final clustering result; In S110, according to the view missing indication matrix M∈{0,1} n×m , extract the presence sample features of each view: in, Represents the set of sample features existing from all perspectives, is the existence feature matrix of view v. If the jth view of sample i exists, then M ij =1, otherwise M ij =0;n v is the number of samples, d v is the feature dimension, m is the number of viewing angles, and the feature vectors of each viewing angle are normalized so that they all have unit modulus length, which is defined as follows: x i (v) ←x i (v) / ||x i (v) ||,i=1,2,...,n;v=1,2,...,m Among them, i represents any sample, v represents any view, n is the number of samples, m is the number of view, ||·|| represents the modulus of the vector, and x i (v) Represents the feature vector of sample i; In S130, the formula for calculating the scaled dot product attention weight anchor graph batch is defined as follows: where τ>0 is the scaling factor, and the tensor represents the batch of scaled dot product attention weighted anchor maps for view v, Indicates the presence of feature batches, Represents a batch of anchor features, with any three tensors To illustrate with an example, is a tensor multiplication operation, representing two three-dimensional tensors and Expand them in the first dimension respectively, then perform matrix product on all slice matrices at corresponding positions and stack them to get a new tensor The definition is as follows: in, Respectively represent tensors Tensor Tensor The i-th slice matrix of , represents the general matrix multiplication, It is a tensor softmax operation, which represents the matrix three-dimensional tensor After expanding by the first dimension, all row vectors of all slice matrices are subjected to softmax operation, which is defined as follows: in, For the existence of feature batch and anchor feature batch The product of the transpose of is divided by the scaling factor τ. Since all eigenvectors have unit modulus, The elements of can be regarded as cosine similarity, It is a tensor transpose operation, which means swapping tensors. The second and third dimensions of exp(·) and . / . are tensor element-by-element exponential function operations and tensor element-by-element division operations, respectively. For any three tensors The element-by-element exponential function of a tensor is defined as The element-wise division operation of a tensor is defined as For the anchor chart batch After expanding by the first dimension, the average value of all slice matrices is calculated to obtain the perspective-specific integrated anchor graph, which is defined as follows: in, Represents perspective-specific integrated anchor chart.
2. The large-scale incomplete multi-view animal image clustering method based on mini-batch integrated scaled dot product attention weight anchor graph according to claim 1, It is characterized in that Randomly sample n from existing sample features a anchor points, and repeat b times to obtain multiple groups of anchor point features, each group contains n a anchor points, replicate the existing sample features b times to obtain multiple groups of existing features, stack the multiple groups of anchor features and the multiple groups of existing features into anchor feature batches and existence feature batches respectively, where the formula for randomly sampling b groups of anchor features and stacking them into anchor feature batches is as follows: Among them, the matrix represents the randomly sampled set of anchor features, n a represents the number of anchor points, b represents the number of batches, and tensor represents the anchor feature batch; random-sample(X,c) represents extracting c samples from the feature matrix X by uniform sampling, replicating the existing sample features b times to obtain multiple groups of existing features, and stacking them into the existing feature batch. The formula is as follows: Among them, the matrix Indicates copied from The b group has characteristics, n v Indicates the number of samples, tensor Indicates the presence of feature batches.
3. The large-scale incomplete multi-view animal image clustering method based on small batch integrated scaled dot product attention weight anchor map according to claim 1, It is characterized in that According to the view missing indicator matrix, the view-specific integrated anchor graphs of all views are weightedly fused to obtain the view common anchor graph, which is defined as follows: Among them, M :,v represents the view vth column of the missing indicator matrix, Z represents the view common anchor graph, and Z (v) Represents the perspective-specific integrated anchor graph for perspective v.
4. The large-scale incomplete multi-view animal image clustering method based on mini-batch integrated scaled dot product attention weight anchor map according to claim 1, It is characterized in that The average value of the obtained t perspective common anchor maps is calculated to obtain the perspective common integrated anchor map, which is defined as follows: Among them, Z * Represents the common integrated anchor graph of the perspective, Z k represents the k-th perspective common anchor map, t represents the number of repetitions, and the obtained perspective common integrated anchor map Z * Perform fast spectral clustering to obtain the final clustering result, and calculate the clustering accuracy of the final clustering result.
