An Incomplete Multi-view Animal Image Clustering Method Based on Graph Convolutional Manifold Regularized Deep Spectral Clustering Network

Abstract

An incomplete multi-view animal image clustering method based on a graph convolutional manifold regularization deep spectral clustering network belongs to the field of animal image clustering processing in image information processing. First, the present invention extracts existing sample features from an incomplete multi-view large batch of animal image datasets and normalizes the feature vectors. Then, an incomplete multi-view deep spectral clustering network (DSCN-IMC) is constructed. This network first uses a multi-view encoder to extract the common features of incomplete multi-view images, obtains an unregularized spectral embedding through a linear layer, and constructs an orthonormalization layer to obtain a regularized constrained view common spectral embedding. Finally, the mini-batch stochastic gradient descent algorithm is used to optimize the manifold-regularized spectral clustering overall loss function until convergence. And k-means clustering is performed on the spectral embedding learned by the network to obtain the final clustering result. Compared with other methods, the present invention has a faster computing speed, a reusable network model, and is relatively less affected by the data missing rate, and has better clustering performance.

Description

Technical Field

[0001] The present invention belongs to the field of animal image clustering processing in image information processing, and particularly relates to an incomplete multi-view animal image clustering method based on a graph convolutional manifold regularization deep spectral clustering network. Background Art

[0002] In the field of image information processing, with the development of Internet technology, the number of multimedia information, especially images, has shown an explosive growth. Therefore, how to improve the accuracy and speed of massive image clustering has become particularly important. Diversified data acquisition methods and diversified feature extraction algorithms facilitate the acquisition of multi-view image data. The data obtained by using a feature extraction algorithm to extract features from animal images is called multi-view animal image data, which can provide more comprehensive and accurate animal appearance and behavior information and is of great significance for species recognition, classification, and biodiversity research. Multi-view data provides complementary and discriminative information and helps to improve the robustness of clustering. However, due to factors such as shooting angle, light, and occluders, some perspective data is missing, resulting in the incomplete problem of multi-view animal images. Existing spectral clustering methods have good effects in solving the clustering problem of incomplete multi-view data. However, as the number of perspectives increases, the problem of high time complexity of the spectral clustering algorithm becomes more prominent. Moreover, traditional spectral clustering algorithms cannot achieve model reuse and need to be solved repeatedly on some data sets. The problems of high complexity and non-reusability of traditional spectral clustering algorithms have not been properly solved. Therefore, more efficient, more robust, and reusable technical methods are needed to solve the spectral clustering problem of incomplete multi-view animal images. Summary of the Invention

[0003] To solve the above problems, the present invention provides an incomplete multi-view animal image clustering method based on a graph convolutional manifold regularization deep spectral clustering network, and the method includes the steps of:

[0004] Extract existing sample features from an incomplete multi-view large-scale animal image dataset, and normalize each component of the feature vector into a standard format with a mean of 0 and a standard deviation of 1.

[0005] Construct a multi-view deep spectral clustering network composed of a multi-view graph convolutional encoding layer, a feature fusion layer, and a spectral embedding output layer, and randomly initialize the network weights.

[0006] For each perspective, construct a k-nearest neighbor similarity matrix and a high-dimensional manifold distribution matrix from the existing features respectively.

[0007] Input the existence sample features and the corresponding k-nearest neighbor similarity matrices of all perspectives into the multi-perspective deep spectral clustering network, and calculate the spectral clustering loss function for the perspective common spectral embedding output by the network. At the same time, impose a manifold regularization constraint on the representation learned by the network to obtain the overall loss function of the manifold-regularized multi-perspective spectral clustering.

[0008] Use the mini-batch stochastic gradient descent algorithm to optimize the overall loss function of the manifold-regularized spectral clustering until convergence, and then perform k-means clustering on the spectral embedding learned by the network to obtain the final clustering result.

[0009] According to the obtained clustering result, calculate the clustering accuracy on the incomplete multi-perspective large-scale animal image dataset.

[0010] Furthermore, extract the existence sample features from an incomplete multi-perspective large-scale animal image dataset, and normalize each component of the feature vector into a standard format with a mean of 0 and a standard deviation of 1. The existence sample features are denoted as where, represents the set of existence sample features of all perspectives, is the existence feature matrix of perspective v; M ∈ {0, 1} n×m represents the perspective missing indication matrix. If the j-th perspective of sample i exists, then M ij = 1, otherwise M ij = 0; n v is the number of existence samples, d v is the feature dimension, and m is the number of perspectives.

[0011] And normalize each component of the feature vector into a standard format with a mean of 0 and a standard deviation of 1 as follows:

[0012]

[0013] where, i represents any existence sample, v represents any perspective, n is the number of samples, and m is the number of perspectives; represents the feature vector of sample i, and respectively represent the mean and standard deviation of

[0014] Furthermore, construct a multi-perspective deep spectral clustering network composed of a multi-perspective graph convolutional encoding layer, a feature fusion layer, and a spectral embedding output layer.

[0015] Each perspective module of the multi-perspective graph convolutional encoding layer consists of two graph convolutional modules:

[0016]

[0017] Among them, relu is a non-linear activation function layer, batchNorm is a batch normalization layer, and conv is a graph convolution layer; is the existing sample feature matrix, is the output of the first graph convolution module, H v is the output of the second graph convolution module, A v is the k-nearest neighbor similarity matrix. The final output of the multi-view graph convolution encoding layer is the low-dimensional representation H exclusive to m views v , v = 1, 2, …, m.

[0018] The feature fusion layer performs weighted fusion on all the low-dimensional representations H exclusive to each view v , v = 1, 2, …, m to obtain the representation H learned by the network, which is defined as follows:

[0019]

[0020] Among them, M :,v represents the v-th column of the missing indicator matrix.

[0021] The definition of the spectral embedding output layer is as follows:

[0022]

[0023] Among them, linear is a fully connected layer with weights and biases, and orthnorm is an orthonormalization layer; is the view-common spectral embedding output by the network, is the un-orthonormalized spectral embedding, is the representation learned by the network; d is the feature dimension of the learned representation, and c is the number of clusters.

[0024] The role of the orthonormalization layer is to orthogonalize the input matrix so that (I c is the c-order identity matrix). The definition of the orthonormalization layer is as follows:

[0025]

[0026] Among them, W orthnorm is the weight matrix of the orthonormalization layer, and L is a lower triangular matrix, which is obtained by performing Cholesky decomposition on the un-orthonormalized spectral embedding , and is defined as follows:

[0027]

[0028] During the network optimization process, W orthnorm is solved by alternating optimization with other network weights.

[0029] Furthermore, for each perspective with existing samples, a k-nearest neighbor similarity matrix and a high-dimensional manifold distribution matrix are constructed respectively.

[0030] The k-nearest neighbor similarity matrix A of perspective v v is defined as follows:

[0031]

[0032] where is the element in the i-th row and j-th column of the k-nearest neighbor similarity matrix , and i and j represent two nodes on the k-nearest neighbor graph respectively; is the k-neighborhood of node i, which consists of the k nodes closest to node i; if there is an edge between node i and node j, then

[0033] The definition of the high-dimensional manifold distribution matrix of perspective v is as follows:

[0034]

[0035] where are the feature vectors of existing samples i, j, l, and k respectively; is the Gaussian kernel width of sample i, which is determined by the perplexity parameter ppl. represents the probability that sample i is a neighbor of sample j in the high-dimensional space; is the element in the i-th row and j-th column of the manifold distribution matrix P v , representing the probability that one of sample i and sample j is a neighbor of the other in the high-dimensional space; ‖·‖ F represents the Frobenius norm of the matrix. The matrix describes the manifold distribution of samples in the high-dimensional space, so it is called the high-dimensional manifold distribution matrix.

[0036] Furthermore, the feature of existing samples of all perspectives and the corresponding k-nearest neighbor similarity matrices are input into the deep spectral clustering network, and the multi-perspective spectral clustering loss function is calculated for the perspective common spectral embedding output by the network and the k-nearest neighbor similarity matrices of multiple perspectives. At the same time, a manifold regularization constraint is imposed on the representation learned by the network to obtain the overall loss function of manifold-regularized multi-perspective spectral clustering, which is defined as follows:

[0037]

[0038] where is the multi-perspective spectral clustering loss function, M :,v represents the v-th column of the missing indicator matrix, A v is the k-nearest neighbor similarity matrix of perspective v, Y is the perspective common spectral embedding output by the network, is a diagonal matrix, and its definition is $tr(·)$ represents the trace of a matrix.

[0039] The manifold regularization constraint term of the network is defined as follows:

[0040]

[0041] Where $Q$ v is the low-dimensional manifold distribution matrix of perspective $v$, and its definition is as follows:

[0042]

[0043] Where are the representation vectors corresponding to samples $i$, $j$, $l$, $k$ of the representation $H$ learned by the network for perspective $v$ respectively; is the element in the $i$-th row and $j$-th column of matrix $Q$ v , representing the probability that sample $i$ and sample $j$ are neighbors of each other in the low-dimensional space. $Q$ v describes the manifold distribution of samples in the low-dimensional space, so it is called the low-dimensional manifold distribution matrix.

[0044] The overall loss function described above consists of the multi-perspective spectral clustering loss function and the manifold regularization constraint term , and its definition is as follows:

[0045]

[0046] Where $\lambda>0$ is the trade-off parameter, is the overall loss function described above.

[0047] Furthermore, the mini-batch stochastic gradient descent algorithm is used to optimize the overall loss function of the multi-perspective spectral clustering with manifold regularization until convergence. Among them, the update rule of the network weights is defined as follows:

[0048]

[0049] Where $\Theta$ is the network weights, $\alpha$ is the learning rate, is the value of the overall loss function estimated on the mini-batch data, is the partial derivative of orthnorm with respect to the network weights $\Theta$. The update rule of the weight matrix $W$ of the orthonormalization layer

[0050]

[0051] Where $W$ orthnorm is the weight matrix of the orthonormalization layer, and $L$ is obtained by performing orthonormalization on the non-orthonormalized spectral embedding Perform Cholesky decomposition on the lower triangular matrix.

[0052] Alternately apply the above network weight update rule and the orthogonal normalization layer weight update rule until The value of converges.

[0053] Furthermore, based on the obtained clustering results, the clustering accuracy on the incomplete multi-view large-scale animal image dataset is calculated.

[0054] This paper provides an incomplete multi-view animal image clustering method based on a graph convolutional manifold regularized deep spectral clustering network, which has the following advantages:

[0055] (1) The method adopts a multi-view clustering framework, which can mine the potential connections between multiple visual features contained in large-scale animal images, make full use of the consistency and difference between multi-view animal image information, and effectively improve the accuracy of spectral clustering.

[0056] (2) The method uses graph convolutional neural network technology to construct an incomplete multi-view deep spectral clustering network. The presence sample features and the corresponding k-nearest neighbor similarity matrix of each view are input into the network, and the network output is the common spectral embedding of each view. After training, this network model can be reused to generate approximate spectral embeddings for samples with the same distribution.

[0057] (3) Unlike traditional spectral clustering, this method uses a mini-batch stochastic gradient descent algorithm to optimize the network parameters. After several rounds of traversal of the dataset, the network converges and obtains an approximate spectral embedding map. Therefore, this method has linear time complexity and constant space complexity, and has good operational efficiency for large-scale datasets. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 This is a flow chart of an incomplete multi-view animal image clustering method based on a graph convolution manifold regularized deep spectral clustering network provided by the present invention;

[0060] Figure 2 This is a schematic diagram of the incomplete multi-view large-scale animal image dataset Animal-50. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In addition, in the following descriptions, the descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present invention.

[0062] Exemplary Method

[0063] As Figure 1 , the present invention provides an incomplete multi-view animal image clustering method based on a graph convolutional manifold regularization deep spectral clustering network. The steps of the method are as follows:

[0064] Step S110: First, extract the existing sample features from the incomplete multi-view large-scale animal image dataset, and normalize each component of the feature vector into a standard format with a mean of 0 and a standard deviation of 1. The existing sample features are denoted as where M ∈ {0, 1} n×m represents the view missing indicator matrix. If the j-th view of sample i exists, then M ij = 1; otherwise, M ij = 0.

[0065] Step S120: Construct a multi-view deep spectral clustering network composed of a multi-view graph convolutional encoding layer, a feature fusion layer, and a spectral embedding output layer. Each view module of the multi-view graph convolutional encoding layer consists of two graph convolutional modules, which are defined as follows:

[0066]

[0067] where relu is a non-linear activation function layer, batchNorm is a batch normalization layer, and conv is a graph convolutional layer.

[0068] The feature fusion layer performs weighted fusion on all view-specific low-dimensional representations H v , v = 1, 2,..., m to obtain the representation H learned by the network, which is defined as follows:

[0069]

[0070] where M :,v represents the v-th column of the missing indicator matrix.

[0071] The spectral embedding output layer is defined as follows:

[0072]

[0073] where linear is a fully connected layer with weights and biases, and orthnorm is an orthonormalization layer; is the perspective common spectral embedding of the network output, is the spectral embedding that is not orthonormalized. is the representation learned by the network; d is the feature dimension of the learned representation, and c is the number of clusters.

[0074] The definition of the orthonormalization layer is as follows:

[0075]

[0076] where W orthnorm is the weight matrix of the orthonormalization layer, and L is the lower triangular matrix of

[0077] Step S130: Construct a k-nearest neighbor similarity matrix and a high-dimensional manifold distribution matrix for each perspective with existing samples. The k-nearest neighbor similarity matrix A v of perspective v is defined as follows:

[0078]

[0079] where, is the element in the i-th row and j-th column of the k-nearest neighbor similarity matrix , and i and j respectively represent two nodes on the k-nearest neighbor graph; is the k-neighborhood of node i.

[0080] The definition of the high-dimensional manifold distribution matrix of perspective v is as follows: [[ID=SO]]

[0081]

[0082] where, are the feature vectors of existing samples i, j, l, and k respectively; is the Gaussian kernel width of sample i. is the element in the i-th row and j-th column of the manifold distribution matrix .

[0083] Step S140: Input the feature of existing samples of all perspectives and the corresponding k-nearest neighbor similarity matrices into the deep spectral clustering network, and calculate the multi-perspective spectral clustering loss function for the perspective common spectral embedding output by the network and the k-nearest neighbor similarity matrices of multiple perspectives. At the same time, apply the manifold regularization constraint to the representation learned by the network to obtain the overall loss function of manifold-regularized multi-perspective spectral clustering, which is defined as follows:

[0084] c

[0085] where, is the multi-perspective spectral clustering loss function, and M :,v represents the v-th column of the missing indicator matrix, and A vis the k-nearest neighbor similarity matrix of perspective v, and Y is the perspective common spectral embedding output by the network. is a diagonal matrix, which is defined as tr(·) represents the trace of a matrix.

[0086] The manifold regularization constraint term of the network is defined as follows:

[0087] ]>

[0088] where Q v is the low-dimensional manifold distribution matrix of perspective v.

[0089] The overall loss function described above consists of the multi-perspective spectral clustering loss function and the manifold regularization constraint term and is defined as follows:

[0090]

[0091] where λ > 0 is the trade-off parameter, is the overall loss function described above.

[0092] Step S150: Use the mini-batch stochastic gradient descent algorithm to optimize the manifold-regularized multi-perspective spectral clustering overall loss function to convergence. The update rule for the network weights is defined as follows:

[0093]

[0094] where Θ is the network weights and α is the learning rate, is the value of the overall loss function estimated on the mini-batch data, is the partial derivative of orthnorm with respect to the network weights Θ. The update rule for the weight matrix W

[0095]

[0096] where W orthnorm is the weight matrix of the orthonormalization layer and L is the lower triangular matrix of . Alternately apply the above update rules until the value of

[0097] converges.

[0098] Through this embodiment, first, sample features are extracted from an incomplete multi-view large-scale animal image dataset, and the feature vectors are normalized. Then, a multi-view deep spectral clustering network is constructed, and a multi-view spectral clustering loss function is calculated for the view-common spectral embedding output by the network and the k-nearest neighbor similarity matrices of multiple views. At the same time, the overall loss function of multi-view spectral clustering with manifold regularization constraint is trained until convergence. Finally, k-means clustering is performed on the spectral embedding learned by the network to obtain the final clustering result. After obtaining the clustering result, the clustering accuracy on the dataset is calculated.

[0099] Furthermore, it is assumed that an incomplete multi-view animal image dataset is clustered according to this embodiment, and a clustering result with an accuracy higher than most methods will be obtained.

[0100] Specific embodiment results

[0101] This embodiment uses a publicly available multi-view animal image dataset and simulates an incomplete multi-view large-scale animal dataset according to different sample missing ratios. The dataset is described as follows:

[0102] The Animal-50 animal image multi-view dataset consists of 10,158 animal images from 50 categories, and two types of features are extracted using the DECAF network and the VGG-19 network.

[0103] This dataset has the following two views:

[0104] View 1 is the DECAF feature, which is the image feature extracted by a deep convolutional neural network, capturing high-level semantic information such as texture, shape, edge, and color in the image.

[0105] View 2 is the VGG-19 feature, which extracts different levels of features from the image through multiple convolutional and pooling layers, as well as fully connected layers, including texture, edge, and geometric shape.

[0106] To verify the superiority of this embodiment (OURS), this embodiment is compared with several existing incomplete multi-view animal image clustering methods, including methods such as BMVC, EOMSCCA, FIMVC-VIA, and FMVACC. The clustering accuracies (ACC) of these methods for five incomplete sample ratios on the above-mentioned publicly available dataset will be compared, and the specific data comparison is shown in Table 1.

[0107] Table 1 Animal-50 dataset clustering accuracy (%)

[0108]

[0109] From the data comparison in the above table, it can be clearly seen that OURS achieves the best performance, significantly improving the clustering accuracy of incomplete multi-view animal images. The quantitative results fully demonstrate the superiority of OURS because OURS can better capture the view compatibility information and view complementarity information in incomplete multi-view animal data. OURS constructs an incomplete multi-view deep spectral clustering network, obtaining a spectral embedding with manifold regularization constraints, improving the computational speed of spectral clustering and providing a reusable network model. Thus, the clustering effect of incomplete multi-view animal images is improved. A large number of experiments show that this method is superior to existing methods. Regarding the parameter settings of this embodiment, in all experiments, the parameter k of the k-nearest neighbor graph matrix is set to 5.

[0110] This embodiment proposes a method for clustering incomplete multi-view animal images based on a graph convolutional manifold regularization deep spectral clustering network, which is used for clustering analysis of large-scale incomplete multi-view animal images. First, the existing sample features of the incomplete multi-view animal image dataset are extracted, and the feature vectors are normalized. Then, an incomplete multi-view deep spectral clustering network (DSCN-IMC) is constructed, and the network output obtains a common spectral embedding with regularization constraints. Finally, the network is trained using a deep spectral clustering network constructed by sample mini-batches, and k-means clustering is performed on the spectral embedding learned by the network to obtain the final clustering result. The experimental results on five incomplete sample ratios of the animal image dataset Animal-50 show that this embodiment has a faster calculation speed and a reusable network model after training compared with other methods, can generate approximate spectral embeddings for samples with the same distribution, is relatively less affected by the data missing rate, and has better clustering performance.

[0111] It should be understood that the above specific embodiments of the present invention are only used for exemplary illustration or explanation of the principles of the present invention, and do not constitute a limitation to the present invention. Therefore, any modifications, equivalent replacements, improvements, etc. made without departing from the spirit and scope of the present invention shall be included within the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modification examples falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A non-complete multi-view animal image clustering method based on a graph convolutional manifold regularization deep spectral clustering network, characterized in that The method includes the steps of: Extracting the existing sample features from an incomplete multi-view large-scale animal image dataset, and normalizing each component of the feature vector into a standard format with a mean of 0 and a standard deviation of 1; Constructing a multi-view deep spectral clustering network composed of a multi-view graph convolutional encoding layer, a feature fusion layer, and a spectral embedding output layer, and randomly initializing the network weights; wherein, each view module of the multi-view graph convolutional encoding layer is composed of two graph convolutional modules; the feature fusion layer performs weighted fusion on all view-specific low-dimensional representations to obtain a unified representation; the spectral embedding output layer is composed of a fully connected layer and an orthonormalization layer; For each view, constructing a k-nearest neighbor similarity matrix and a high-dimensional manifold distribution matrix from the existing features respectively; Inputting the existing sample features of all views and the corresponding k-nearest neighbor similarity matrices into the multi-view deep spectral clustering network, and calculating the spectral clustering loss function for the view-common spectral embedding output by the network; at the same time, imposing a manifold regularization constraint on the representation learned by the network to obtain the overall loss function of the manifold-regularized multi-view spectral clustering; Using the mini-batch stochastic gradient descent algorithm to optimize the overall loss function of the manifold-regularized spectral clustering until convergence, and then performing k-means clustering on the spectral embedding learned by the network to obtain the final clustering result; Calculating the clustering accuracy on the incomplete multi-view large-scale animal image dataset according to the obtained clustering result.

2. The incomplete multi-view animal image clustering method based on the graph convolutional manifold regularization deep spectral clustering network according to claim 1, characterized in that, Extract the existing sample features from an incomplete multi-view large-scale animal image dataset, and normalize each component of the feature vector into a standard format with a mean of 0 and a standard deviation of 1; the existing sample features are denoted as where represents the set of existing sample features for all views, is the existing feature matrix for view v; M ∈ {0, 1} n×m represents the view missing indication matrix. If the j-th view of sample i exists, then M ij = 1, otherwise M ij = 0; n v is the number of existing samples, d v is the feature dimension, and m is the number of views; And normalizing each component of the feature vector into a standard format with a mean of 0 and a standard deviation of 1, as follows: i = 1, 2, …, n, v = 1, 2, …, m where \(i\) represents any existing sample, \(v\) represents any perspective, \(n\) is the number of samples, and \(m\) is the number of perspectives; represents the feature vector of sample \(i\), and respectively represent the mean and standard deviation of 3. The incomplete multi-view animal image clustering method based on the graph convolutional manifold regularization deep spectral clustering network according to claim 1, characterized in that, Constructing a multi-view deep spectral clustering network composed of a multi-view graph convolutional encoding layer, a feature fusion layer, and a spectral embedding output layer; Each view module of the multi-view graph convolutional encoding layer consists of two graph convolutional modules Composed of: Among them, relu is a non-linear activation function layer, batchNorm is a batch normalization layer, and conv is a graph convolutional layer; is the existing sample feature matrix, is the output of the first graph convolutional module, H v is the output of the second graph convolutional module, A v is the k-nearest neighbor similarity matrix; the final output of the multi-view graph convolutional encoding layer is the m perspective-specific low-dimensional representations H v , v = 1, 2, …, m; The feature fusion layer performs weighted fusion on all view-specific low-dimensional representations H v , where v = 1, 2, …, m, to obtain the representation H learned by the network, which is defined as follows: where M :,v represents the v-th column of the missing indication matrix; The definition of the spectral embedding output layer is as follows: Among them, linear is a fully connected layer with weights and biases, and orthnorm is an orthonormalization layer; is the perspective common spectral embedding output by the network, is the spectral embedding without orthonormalization, is the representation learned by the network; d is the feature dimension of the learned representation, and c is the number of clusters; The role of the orthonormalization layer is to orthonormalize the input matrix, so that where I c is the c-order identity matrix, and the definition of the orthonormalization layer is as follows: where W orthnorm is the weight matrix of the orthonormalization layer, and L is a lower triangular matrix obtained by performing Cholesky decomposition on the non-orthonormalized spectral embedding and is defined as follows: During the network optimization process, W orthnorm is solved by alternating optimization with other network weights.

4. The incomplete multi-view animal image clustering method based on the graph convolutional manifold regularization deep spectral clustering network according to claim 1, characterized in that Constructing a k-nearest neighbor similarity matrix and a high-dimensional manifold distribution matrix for each view of the existing samples respectively; The k-nearest neighbor similarity matrix A of perspective v v is defined as follows: Among them, is the element at the i-th row and j-th column of the k-nearest neighbor similarity matrix , where i and j respectively represent two nodes on the k-nearest neighbor graph; is the k-neighborhood of node i, which consists of the k nodes closest to node i; if there is an edge between node i and node j, then otherwise The definition of the high-dimensional manifold distribution matrix of view v is as follows: Among them, are the feature vectors of samples i, j, l, and k respectively; is the Gaussian kernel width of sample i, which is determined by the perplexity parameter ppl; represents the probability that sample i is a neighbor of sample j in the high-dimensional space; is the manifold distribution matrix P v The element in the i-th row and j-th column, representing the probability that one of sample i and sample j is a neighbor of the other in the high-dimensional space; ‖·‖ F represents the Frobenius norm of the matrix; the matrix describes the manifold distribution of samples in the high-dimensional space, so it is called the high-dimensional manifold distribution matrix.

5. The incomplete multi-view animal image clustering method based on the graph convolutional manifold regularization deep spectral clustering network according to claim 1, characterized in that, Inputting the existing sample features of all views and the corresponding k-nearest neighbor similarity matrices into the deep spectral clustering network, and calculating the multi-view spectral clustering loss function for the view-common spectral embedding output by the network and the k-nearest neighbor similarity matrices of multiple views; at the same time, imposing a manifold regularization constraint on the representation learned by the network to obtain the overall loss function of the manifold-regularized multi-view spectral clustering, defined as follows: Among them, is the multi-view spectral clustering loss function, M :,v represents the v-th column of the missing indicator matrix, A v is the k-nearest neighbor similarity matrix of view v, Y is the view common spectral embedding output by the network, is a diagonal matrix, which is defined as tr(·) represents the trace of a matrix; The manifold regularization constraint term of the network is defined as follows: where Q v is the low-dimensional manifold distribution matrix of the viewing angle v, and is defined as follows: Among them, are the representation vectors of samples i, j, l, and k corresponding to the perspective v of the network-learned representation H, respectively; is the matrix Q v The element in the i-th row and j-th column, representing the probability that one of samples i and j is a neighbor of the other in the low-dimensional space; Q v Describes the manifold distribution of samples in the low-dimensional space, so it is called the low-dimensional manifold distribution matrix; The overall loss function described above consists of a multi-view spectral clustering loss function and a manifold regularization constraint term and is defined as follows: where λ > 0 is a trade-off parameter, is the overall loss function described above.

6. The incomplete multi-view animal image clustering method based on the graph convolutional manifold regularization deep spectral clustering network according to claim 1, characterized in that, Use the mini-batch stochastic gradient descent algorithm to optimize the overall loss function of multi-view spectral clustering with manifold regularization to convergence; where the update rule of the network weights is defined as follows: where Θ is the network weight, α is the learning rate, is the overall loss function value estimated on the mini-batch data, is the partial derivative of orthnorm with respect to the network weight Θ; the update rule of the orthonormalization layer's weight matrix W is defined as follows: where W orthnorm is the weight matrix of the orthonormalization layer, and L is the lower triangular matrix obtained by performing Cholesky decomposition on the non-orthonormalized spectral embedding ; Alternately apply the above network weight update rule and the orthonormalization layer weight update rule until the value converges.

7. The incomplete multi-view animal image clustering method based on the graph convolutional manifold regularization deep spectral clustering network according to claim 1, characterized in that Calculating the clustering accuracy on the incomplete multi-view large-scale animal image dataset according to the obtained clustering result.

Images