Clustering method and device for graph regular non-negative matrix factorization and medium

Abstract

The invention belongs to the technical field of image classification, and discloses a clustering method and device for graph regular non-negative matrix factorization and a medium, and the method comprises the steps: obtaining matrix information data and a to-be-clustered number; defining four initialized sparse matrixes, respectively and correspondingly constructing a matrix E, a weight matrix U, a coefficient matrix V and an assistance matrix Z, and setting an initial hyper-parameter value mu; determining a target equation and the maximum number of iterations, and iteratively solving a construction matrix E, a weight matrix U, a coefficient matrix V, an assistance matrix Z, a Lagrange operator Y1 and a Lagrange operator Y2 according to an updating formula; and outputting a final coefficient matrix V under the condition that the number of iterations is maximum or iteration converges, and obtaining a clustering result. According to the method, the target function construction method is improved, and better constraint conditions are added to capture deep information features, so that the robustness and accuracy of image classification are improved, and a better clustering effect is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of image clustering analysis, and more specifically, to a clustering method, device and medium for graph regular non-negative matrix decomposition. Background Art

[0002] The traditional NMF method uses the two-norm to construct the initial matrix, which has achieved good performance in most tasks. However, it still has some limitations. One of the main problems is that the two-norm is used to construct the objective function, and it is well known that the two-norm is unstable with noise and outliers. In order to obtain good clustering results, higher requirements are placed on the input data. However, in the real world, noise and outliers are almost inevitable. Even if some methods are taken to deal with these abnormal data, the clustering results will be affected. This study innovatively proposes a non-negative matrix factorization clustering method based on graph regularization. This method incorporates a more rigorous constraint paradigm through a carefully designed matrix optimization strategy, significantly enhancing the robustness of the algorithm. This improvement not only reduces the stringent requirements on the quality of the initial input data, but also effectively resists the interference of outliers on the feature extraction process even when the data set is mixed with outliers, ensuring the stability and accuracy of the results. Therefore, the algorithm shows a wider range of applicability and potential in various practical application scenarios, especially in the field of image recognition, such as face recognition tasks. Even in the face of a data set containing some abnormal images, it can still maintain excellent performance and achieve high-precision recognition effects. Summary of the invention

[0003] In order to solve the above problems existing in the prior art, the present invention provides a clustering method, device and medium for graph regular non-negative matrix decomposition, which improves the robustness and accuracy of image classification and obtains better clustering effect by improving the objective function construction method and adding better constraint conditions to capture deep information features.

[0004] In order to achieve the above object, the present invention adopts the following technical solution:

[0005] According to a first aspect of the present invention, a clustering method for graph regular non-negative matrix decomposition is provided, the method comprising:

[0006] Obtaining matrix information data and the number of clusters to be clustered, wherein the matrix information data represents the user's rating of an item or the interaction between the user and the item in a recommendation system, or represents the interaction information between users in social network analysis, or represents the co-occurrence relationship between word vectors and word vectors in natural language processing, or the position information and gray value information of each pixel after grayscale processing of image data;

[0007] Define four initialization sparse matrices, corresponding to the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z, and set the initial hyperparameter value μ;

[0008] Determine the target equation and the maximum number of iterations, and iteratively solve the construction matrix E, weight matrix U, coefficient matrix V, assistance matrix Z, and Lagrangian operators Y1 and Y2 according to the update formula;

[0009] When the number of iterations is maximum or the iterations converge, the final coefficient matrix V is output to obtain the clustering result.

[0010] Furthermore, the matrix information data is represented as X∈R m×n , where each eigenvalue matrix X consists of n column vectors x i ∈R m Composition, R m Represents an m-dimensional space, m represents the dimension of the sample, and n represents the number of samples.

[0011] Furthermore, the number to be clustered is a positive integer not exceeding 10.

[0012] Furthermore, the initial hyperparameter value μ is set to 0.01.

[0013] Furthermore, the objective equation is expressed as:

[0014]

[0015] In the formula, (·) Hx Expressed as ∑ i log(1+∥(·) i ∥2), i represents the number of rows of the matrix;

[0016] Tr() represents the trace of the matrix;

[0017] I represents the identity matrix;

[0018] α, λ1, λ2 represent hyper parameters;

[0019] T represents matrix transpose;

[0020] Furthermore, the maximum number of iterations is determined according to the size of the data set, the computational complexity, and the limitation of computing resources.

[0021] Furthermore, the update formulas of the construction matrix E, the weight matrix U, the coefficient matrix V, and the assistance matrix Z are obtained through the following steps:

[0022] Expand the objective equation to obtain the Lagrangian function L:

[0023]

[0024] The Karush-Kuhn-Tucker condition is used to obtain the update formulas of the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z.

[0025] Furthermore, the update formula for constructing the matrix E is:

[0026]

[0027] In the formula, Q i represents the i-th row of matrix Q;

[0028] The update formula of the weight matrix U is:

[0029]

[0030] The update formula of the coefficient matrix V is:

[0031]

[0032] In the formula, P and W both represent construction matrices, W = (V T V+λ2I) -1 V T ;

[0033] The update formula of the assistance matrix Z is:

[0034]

[0035] In the formula, R represents a matrix constructed to simplify calculations, and the solution of Z is Z = max(R, 0);

[0036] The update formula of Lagrangian operators Y1 and Y2 and hyperparameter value μ is:

[0037] Y1=Y1+μ(ZV),

[0038] Y2=Y2+μ(X-UV T -E)

[0039] μ=ρμ.

[0040] Where ρ represents the Lagrangian parameter in the optimization process, which indicates the degree of updating of μ.

[0041] According to a second aspect of the present invention, a clustering device for graph regular non-negative matrix decomposition is provided, the device comprising:

[0042] A data acquisition module is configured to acquire matrix information data and the number of items to be clustered, wherein the matrix information data represents the user's rating of items or the interaction between users and items in the recommendation system, or represents the interaction information between users in social network analysis, or represents the co-occurrence relationship between word vectors and word vectors in natural language processing, or the position information and gray value information of each pixel after the image data is grayed out;

[0043] The matrix construction module is configured to define four initialization sparse matrices, corresponding to the construction matrix E, the weight matrix U, the coefficient matrix V, and the assistance matrix Z, and set the initial hyperparameter value μ;

[0044] An iterative solution module is configured to determine the target equation and the maximum number of iterations, and iteratively solve the construction matrix E, the weight matrix U, the coefficient matrix V, the assistance matrix Z, and the Lagrangian operators Y1 and Y2 according to the update formula;

[0045] The clustering output module is configured to output the final coefficient matrix V to obtain the clustering result when the number of iterations is maximum or the iterations converge.

[0046] According to a third aspect of the present invention, there is provided a readable storage medium storing one or more programs, wherein the one or more programs can be executed by one or more processors to implement the method as described above.

[0047] The present invention has at least the following beneficial effects:

[0048] The present invention improves the objective function construction method and adds better constraints to capture deep information features, thereby improving the robustness and accuracy of image classification and obtaining better clustering effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flow chart of a graph regular non-negative matrix decomposition clustering method according to an embodiment of the present invention is shown.

[0050] Figure 2 A structural diagram of a clustering device for graph regular non-negative matrix decomposition according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and specific embodiments, but are not intended to limit the present invention. For the various steps described herein, if there is no necessity for a causal relationship between each other, the order in which they are described as examples herein should not be regarded as a limitation, and those skilled in the art should know that they can be adjusted in order, as long as the logic between them is not destroyed, resulting in the inability to implement the entire process.

[0052] Explanation of the name:

[0053] Karush-Kuhn-Tucker conditions: In optimization problems, a set of necessary conditions used to determine whether a candidate solution is a local optimal solution. It is a constraint condition used for nonlinear programming problems and is applicable to problems with equality and inequality constraints. The KKT condition combines the objective function, equality constraints, and inequality constraints to determine the possible optimal solution by examining the gradient and Lagrange multipliers.

[0054] The embodiment of the present invention proposes a clustering method of graph regular non-negative matrix decomposition, please refer to Figure 1 , which is a flow chart of the method.

[0055] The clustering method of regular non-negative matrix factorization of the graph includes the following steps:

[0056] Step 1: Obtain the matrix information data X∈R for cluster analysis m×n And the number of clusters required, where each eigenvalue matrix X consists of n column vectors x i ∈R m Composition, R m It is represented as an m-dimensional space, where m represents the latitude of the sample and n represents the number of samples.

[0057] It should be noted that the matrix information data described in this article contains matrices about the relationship between observed objects. In different application scenarios, matrix information data represents different data. For example, in a recommendation system, matrix information data represents the user's rating of an item or the interaction between a user and an item; in social network analysis, matrix information data represents the interaction between users; in natural language processing, matrix information data represents the co-occurrence relationship between words. In image recognition, matrix information data is the position information and gray value information of each pixel after the image data is grayed out.

[0058] The number of clusters required is the number of clusters. The number of clusters is an integer and is usually determined based on the specific problem and data characteristics. In order to ensure the practicality of the results, it is recommended to set the R value within 10 when the amount of data is not large. This can make the number of clusters moderate, avoid generating too many clusters, and improve the interpretability and practicality of the clustering results.

[0059] Step 2: Define four initialization sparse matrices, corresponding to the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z, and set the initial hyperparameter value μ.

[0060] Exemplarily, the initial hyperparameter value μ is set to 0.01.

[0061] Step 3: Determine the target equation and the maximum number of iterations, and iteratively solve the construction matrix E, weight matrix U, coefficient matrix V, assistance matrix Z, and Lagrangian operators Y1 and Y2 according to the update formula.

[0062] In an exemplary embodiment, in step 3, the objective equation is:

[0063]

[0064] in(·) Hx Expressed as ∑ i log(1+||(·) i ||2);

[0065] Tr() represents the trace of the matrix;

[0066] I represents the identity matrix;

[0067] α, λ1, and λ2 are represented as hyperparameters;

[0068] T represents matrix transpose;

[0069] In an exemplary embodiment, in step 3, when setting the maximum number of iterations, factors such as the size of the data set, the complexity of the algorithm, and the limitation of computing resources need to be considered.

[0070] In an exemplary embodiment, in step 3, the update formulas of the construction matrix E, the weight matrix U, the coefficient matrix V, and the assistance matrix Z are obtained by the following steps:

[0071] The target equation can be expanded according to the Augmented Lagrange Multiplier (ALM) to obtain the Lagrange function L:

[0072]

[0073] Where:

[0074] The Karush-Kuhn-Tucker condition is used to obtain the update formulas of the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z.

[0075] 8. The method according to claim 1, characterized in that the update formula of the construction matrix E is:

[0076]

[0077] In the formula, Q i represents the i-th row of matrix Q;

[0078] The update formula of the weight matrix U is:

[0079]

[0080] The update formula of the coefficient matrix V is:

[0081]

[0082] In the formula, P and W both represent construction matrices, W = (V T V+λ2I) -1 V T ;

[0083] The update formula of the assistance matrix Z is:

[0084]

[0085] In the formula, R represents a matrix constructed to simplify calculations, and the solution of Z is Z = max(R, 0);

[0086] The update formula of Lagrangian operators Y1 and Y2 and hyperparameter value μ is:

[0087] Y1=Y1+μ(ZV),

[0088] Y2=Y2+μ(X-UV T -E)

[0089] μ=ρμ.

[0090] Where ρ represents the Lagrangian parameter in the optimization process, which indicates the degree of updating of μ.

[0091] Step 4: After the number of iterations reaches a maximum or the iterations converge, the final coefficient matrix V is output after the solution is completed, that is, the clustering result is obtained.

[0092] The embodiment of the present invention also provides a clustering device for graph regular non-negative matrix decomposition, such as Figure 2As shown, the device comprises:

[0093] The data acquisition module 201 is configured to acquire matrix information data and the number of items to be clustered, wherein the matrix information data represents the user's rating of items or the interaction between users and items in the recommendation system, or represents the interaction information between users in social network analysis, or represents the co-occurrence relationship between word vectors and word vectors in natural language processing, or the position information and gray value information of each pixel after the image data is grayed out;

[0094] The matrix construction module 202 is configured to define four initialization sparse matrices, corresponding to the construction matrix E, the weight matrix U, the coefficient matrix V, and the assistance matrix Z, and set the initial hyperparameter value μ;

[0095] Iterative solution module 203 is configured to determine the target equation and the maximum number of iterations, and iteratively solve the construction matrix E, the weight matrix U, the coefficient matrix V, the assistance matrix Z, and the Lagrangian operators Y1 and Y2 according to the update formula;

[0096] The clustering output module 204 is configured to output the final coefficient matrix V to obtain the clustering result when the number of iterations is maximum or the iterations converge.

[0097] In some embodiments, the matrix information data is represented as X∈R m×n , where each eigenvalue matrix X consists of n column vectors x i ∈R m Composition, R m Represents an m-dimensional space, m represents the dimension of the sample, and n represents the number of samples.

[0098] In some embodiments, the number of clusters to be clustered is a positive integer not exceeding 10.

[0099] In some embodiments, the initial hyperparameter value μ is set to 0.01.

[0100] In some embodiments, the objective equation is expressed as:

[0101]

[0102] In the formula, (·) Hx Expressed as ∑ i log(1+∥(·) i ∥2), i represents the number of rows of the matrix;

[0103] Tr() represents the trace of the matrix;

[0104] I represents the identity matrix;

[0105] α, λ1, λ2 represent hyper parameters;

[0106] T represents matrix transpose;

[0107] In some embodiments, the maximum number of iterations is determined based on the size of the data set, computational complexity, and limitations of computing resources.

[0108] In some embodiments, the update formulas of the construction matrix E, the weight matrix U, the coefficient matrix V, and the assistance matrix Z are obtained by the following steps:

[0109] Expand the objective equation to obtain the Lagrangian function L:

[0110]

[0111] The Karush-Kuhn-Tucker condition is used to obtain the update formulas of the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z.

[0112] In some embodiments, the update formula for constructing the matrix E is:

[0113]

[0114] In the formula, Q i represents the i-th row of matrix Q;

[0115] The update formula of the weight matrix U is:

[0116]

[0117] The update formula of the coefficient matrix V is:

[0118]

[0119] In the formula, 2U T U) T , P and W both represent construction matrices, W=(V T V+λ2I) -1 V T ;

[0120] The update formula of the assistance matrix Z is:

[0121]

[0122] In the formula, R represents a matrix constructed to simplify calculations, and the solution of Z is Z = max(R, 0);

[0123] The update formula of Lagrangian operators Y1 and Y2 and hyperparameter value μ is:

[0124]

[0125] Where ρ represents the Lagrangian parameter in the optimization process, which indicates the degree of updating of μ.

[0126] It should be noted that the various device structures described in this embodiment belong to the same technical concept as the previously described method, and achieve the same technical effect through the same principle, which will not be repeated here.

[0127] An embodiment of the present invention further provides a readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the method described in any of the above embodiments.

[0128] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present invention with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims will be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the implementation of this application, and the examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0129] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. For example, those of ordinary skill in the art can use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the present invention. This should not be interpreted as a feature of an invention that is not claimed for protection being necessary for any claim. On the contrary, the subject matter of the present invention may be less than all the features of the embodiments of a specific invention. Thus, the following claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently used as a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of the present invention should be determined with reference to the full scope of the equivalent forms of the attached claims and these claims.

Claims

1. A graph regular non-negative matrix decomposition clustering method, characterized in that: The method comprises: Obtaining matrix information data and the number of clusters to be clustered, wherein the matrix information data represents the user's rating of an item or the interaction between the user and the item in a recommendation system, or represents the interaction information between users in social network analysis, or represents the co-occurrence relationship between word vectors and word vectors in natural language processing, or the position information and gray value information of each pixel after grayscale processing of image data; Define four initialization sparse matrices, corresponding to the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z, and set the initial hyperparameter value μ; Determine the target equation and the maximum number of iterations, and iteratively solve the construction matrix E, weight matrix U, coefficient matrix V, assistance matrix Z, and Lagrangian operators Y1 and Y2 according to the update formula; When the number of iterations is maximum or the iterations converge, the final coefficient matrix V is output to obtain the clustering result.

2. The method according to claim 1, characterized in that The matrix information data is represented by X∈R m×n , where each eigenvalue matrix X consists of n column vectors x i ∈R m Composition, R m Represents an m-dimensional space, m represents the dimension of the sample, and n represents the number of samples.

3. The method according to claim 1, characterized in that The number to be clustered is a positive integer not exceeding 10.

4. The method according to claim 1, characterized in that: The initial hyperparameter value μ is set to 0.

01.

5. The method according to claim 2, characterized in that: The objective equation is expressed as: s.t.Z≥0,Z=V,V T V=I,E=X-UV T . In the formula, (·) Hx Expressed as ∑ i log(1+∥(·) i ∥2), i represents the number of rows of the matrix; Tr() represents the trace of the matrix; I represents the identity matrix; α, λ1, λ2 represent hyper parameters; T stands for matrix transpose.

6. The method according to claim 1, characterized in that The maximum number of iterations is determined based on the size of the data set, computational complexity, and limitations of computing resources.

7. The method according to claim 5, characterized in that The update formulas of the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z are obtained through the following steps: Expand the objective equation to obtain the Lagrangian function L: s.t.Z≥0,V T V=I The Karush-Kuhn-Tucker condition is used to obtain the update formulas of the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z.

8. The method according to claim 1, characterized in that The update formula for constructing the matrix E is: In the formula, Q=X-UV T +Y2 / μ Q i represents the i-th row of matrix Q; The update formula of the weight matrix U is: The update formula of the coefficient matrix V is: In the formula, P and W both represent construction matrices, W = (V T V+λ2I) -1 V T ; The update formula of the assistance matrix Z is: In the formula, R represents a matrix constructed to simplify calculations, and the solution of Z is Z = max(R, 0); The update formula of Lagrangian operators Y1 and Y2 and hyperparameter value μ is: Y1=Y1+μ(ZV), Y2=Y2+µ(X-UV T -BY) μ=ρμ. Where ρ represents the Lagrangian parameter in the optimization process, which indicates the degree of updating of μ.

9. A clustering device for graph regular non-negative matrix decomposition, characterized in that: The device comprises: A data acquisition module is configured to acquire matrix information data and the number of items to be clustered, wherein the matrix information data represents the user's rating of items or the interaction between users and items in the recommendation system, or represents the interaction information between users in social network analysis, or represents the co-occurrence relationship between word vectors and word vectors in natural language processing, or the position information and gray value information of each pixel after the image data is grayed out; The matrix construction module is configured to define four initialization sparse matrices, corresponding to the construction matrix E, the weight matrix U, the coefficient matrix V, and the assistance matrix Z, and set the initial hyperparameter value μ; An iterative solution module is configured to determine the target equation and the maximum number of iterations, and iteratively solve the construction matrix E, the weight matrix U, the coefficient matrix V, the assistance matrix Z, and the Lagrangian operators Y1 and Y2 according to the update formula; The clustering output module is configured to output the final coefficient matrix V to obtain the clustering result when the number of iterations is maximum or the iterations converge. 10 . A non-transitory computer-readable storage medium storing instructions, which, when executed by a processor, perform the method according to claim 1 .

Images