Clustering method, device and medium for graph regular non-negative matrix decomposition

Abstract

The present invention belongs to the field of image classification technology, and discloses a clustering method, device and medium of graph regular non-negative matrix decomposition, the method comprising: obtaining matrix information data and the number of to-be-clustered; defining four initialization sparse matrices, corresponding to the construction matrix E, weight matrix U, coefficient matrix V, and auxiliary matrix Z, respectively, and setting the initial hyperparameter value μ; determining the target equation and the maximum number of iterations, and iteratively solving the construction matrix E, weight matrix U, coefficient matrix V, auxiliary matrix Z, Lagrangian operators Y1 and Y2 according to the update formula; when the number of iterations is the maximum or the iterations converge, outputting the final coefficient matrix V to obtain the clustering result. The present invention improves the objective function construction method, adds better constraints to capture deep-level information features, thereby improving the robustness and accuracy of image classification and obtaining better clustering effects.

Description

Technical Field

[0001] The present invention relates to the technical field of image cluster analysis, and more specifically, to a clustering method, device and medium for graph regularized non-negative matrix decomposition. Background Art

[0002] Traditional NMF methods, which use the bi-norm to construct the initial matrix, have achieved good performance in many tasks. However, they still have some limitations. One major issue is the use of the bi-norm to construct the objective function, which is well known to be unstable with noise and outliers. This places higher demands on the input data to achieve good clustering results. However, noise and outliers are almost inevitable in the real world, and even with methods to address these anomalies, they can still affect clustering results. This study innovatively proposes a non-negative matrix factorization clustering method based on graph regularization. This method incorporates a more rigorous constraint paradigm through a carefully designed matrix optimization strategy, significantly enhancing the robustness of the algorithm. This improvement not only reduces the stringent requirements for initial input data quality but also effectively resists the interference of outliers on the feature extraction process even when the dataset is mixed with outliers, ensuring the stability and accuracy of the results. As a result, the algorithm demonstrates wider applicability and potential in various practical applications, particularly in image recognition tasks such as face recognition, where it maintains excellent performance and achieves high-precision recognition even with datasets containing some anomalies. Summary of the Invention

[0003] In order to solve the above-mentioned problems existing in the prior art, the present invention provides a clustering method, device and medium for graph regularized non-negative matrix decomposition. By improving the objective function construction method and adding better constraints to capture deep-level information features, the robustness and accuracy of image classification are improved, and better clustering effects are obtained.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] According to a first aspect of the present invention, a clustering method for graph regularized non-negative matrix decomposition is provided, the method comprising:

[0006] Obtaining matrix information data and the number of clusters to be clustered. The matrix information data represents user ratings of items or user-item interactions in a recommendation system, or represents user interaction information in social network analysis, or represents the co-occurrence relationship between word vectors in natural language processing, or the position and grayscale value information of each pixel in grayscale processed image data;

[0007] Define four initialization sparse matrices, corresponding to the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z, and set the initial hyperparameter value μ;

[0008] Determine the target equation and the maximum number of iterations, and iteratively solve the construction matrix E, weight matrix U, coefficient matrix V, assistance matrix Z, and Lagrangian operators Y1 and Y2 according to the update formula;

[0009] When the number of iterations is maximum or the iterations converge, the final coefficient matrix V is output to obtain the clustering result.

[0010] Furthermore, the matrix information data is represented as X∈R m×n , where each eigenvalue matrix X consists of n column vectors x i ∈R m Composition, R m Represents an m-dimensional space, m represents the dimension of the sample, and n represents the number of samples.

[0011] Furthermore, the number of clusters to be clustered is a positive integer not exceeding 10.

[0012] Furthermore, the initial hyperparameter value μ is set to 0.01.

[0013] Furthermore, the objective equation is expressed as:

[0014]

[0015] stZ≥0,Z=V,V T V=I,E=X-UV T .

[0016] Where, (·) Hx Expressed as ∑ i log(1+∥(·) i ∥2), i represents the number of rows in the matrix;

[0017] Tr() represents the trace of the matrix;

[0018] I represents the identity matrix;

[0019] α, λ1, and λ2 represent hyperparameters;

[0020] T represents matrix transpose;

[0021] Furthermore, the maximum number of iterations is determined based on the size of the data set, computational complexity, and computational resource limitations.

[0022] Furthermore, the update formulas for the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z are obtained through the following steps:

[0023] Expand the objective equation to obtain the Lagrangian function:

[0024]

[0025] stZ≥0,V T V=I

[0026] The Karush-Kuhn-Tucker condition is used to obtain the update formulas of the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z.

[0027] Furthermore, the update formula for constructing the matrix E is:

[0028]

[0029] Where, Q i represents the i-th row of matrix Q;

[0030] The update formula of the weight matrix U is:

[0031]

[0032] The update formula of the coefficient matrix V is:

[0033]

[0034] Where, P and W both represent construction matrices, W=(V T V+λ2I) -1 V T ;

[0035] The update formula of the assistance matrix Z is:

[0036]

[0037] Where, R represents a matrix constructed to simplify calculations, and the solution of Z is Z = max(R, 0);

[0038] The update formulas for the Lagrangian operators Y1 and Y2 and the hyperparameter value μ are:

[0039] Y1=Y1+μ(ZV),

[0040] Y2=Y2+μ(X-UV T -E)

[0041] μ=ρμ.

[0042] Where ρ represents the Lagrangian parameter in the optimization process, which indicates the degree of update of μ.

[0043] According to a second aspect of the present invention, there is provided a clustering device for graph regularized non-negative matrix decomposition, the device comprising:

[0044] A data acquisition module is configured to acquire matrix information data and the number of items to be clustered. The matrix information data represents user ratings of items or user-item interactions in a recommendation system, or represents user interaction information in social network analysis, or represents the co-occurrence relationship between word vectors in natural language processing, or the position and grayscale value information of each pixel in grayscale processed image data;

[0045] The matrix construction module is configured to define four initialization sparse matrices, corresponding to the construction matrix E, the weight matrix U, the coefficient matrix V, and the assistance matrix Z, and set the initial hyperparameter value μ;

[0046] An iterative solution module is configured to determine the target equation and the maximum number of iterations, and iteratively solve the construction matrix E, the weight matrix U, the coefficient matrix V, the assistance matrix Z, and the Lagrangian operators Y1 and Y2 according to the update formula;

[0047] The clustering output module is configured to output the final coefficient matrix V to obtain the clustering result when the number of iterations is maximum or the iterations converge.

[0048] According to a third aspect of the present invention, a readable storage medium is provided, wherein the readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method described above.

[0049] The present invention has at least the following beneficial effects:

[0050] The present invention improves the objective function construction method and adds better constraints to capture deep information features, thereby improving the robustness and accuracy of image classification and achieving better clustering effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A flowchart of a graph regularized non-negative matrix factorization clustering method according to an embodiment of the present invention is shown.

[0052] Figure 2 A structural diagram of a clustering device for graph regularized non-negative matrix decomposition according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0053] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments of the present invention are further described in detail below with reference to the accompanying drawings and specific embodiments, but are not intended to limit the present invention. For the various steps described herein, if there is no necessity for a contextual relationship between each other, the order in which they are described as examples herein should not be regarded as limiting, and those skilled in the art should know that they can be adjusted in order as long as the logic between them is not destroyed, resulting in the inability to implement the entire process.

[0054] Name explanation:

[0055] Karush-Kuhn-Tucker (KKT) conditions: A set of necessary conditions for determining whether a candidate solution is a local optimum in an optimization problem. It is a type of constraint used in nonlinear programming problems and is applicable to problems with equality and inequality constraints. KKT conditions combine the objective function with equality and inequality constraints to determine the possible optimal solution by examining the gradient and Lagrange multipliers.

[0056] The embodiment of the present invention proposes a clustering method based on graph regular non-negative matrix decomposition, please refer to Figure 1 , which is a flowchart of the method.

[0057] The graph regular non-negative matrix factorization clustering method includes the following steps:

[0058] Step 1: Obtain the matrix information data X∈R for cluster analysis m×n And the number of clusters required, where each eigenvalue matrix X consists of n column vectors x i ∈R m Composition, R m It is expressed as an m-dimensional space, where m represents the latitude of the sample and n represents the number of samples.

[0059] It should be noted that the matrix information data described in this article includes matrices representing relationships between observed objects. In different application scenarios, matrix information data represents different data. For example, in recommendation systems, matrix information data represents user ratings of items or user-item interactions; in social network analysis, matrix information data represents user interactions; and in natural language processing, matrix information data represents the co-occurrence relationships between words. In image recognition, matrix information data is the position and grayscale value information of each pixel in the grayscale-processed image data.

[0060] The number of clusters required is the number of clusters. This is an integer, typically determined based on the specific problem and data characteristics. To ensure the practicality of the results, it is recommended to set the R value to less than 10 when the data volume is small. This can ensure a moderate number of clusters, avoid the generation of excessive clusters, and improve the interpretability and practicality of the clustering results.

[0061] Step 2: Define four initialization sparse matrices, corresponding to the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z, and set the initial hyperparameter value μ.

[0062] For example, the initial hyperparameter value μ is set to 0.01.

[0063] Step 3: Determine the target equation and the maximum number of iterations, and iteratively solve the construction matrix E, weight matrix U, coefficient matrix V, assistance matrix Z, and Lagrangian operators Y1 and Y2 according to the update formula.

[0064] In an exemplary embodiment, in step 3, the objective equation is:

[0065]

[0066] stZ≥0,Z=V,V T V=I,E=X-UV T .

[0067] in(·) Hx Expressed as Σ i log(1+||(·) i ||2);

[0068] Tr() represents the trace of the matrix;

[0069] I represents the identity matrix;

[0070] α, λ1, and λ2 are represented as hyperparameters;

[0071] T represents matrix transpose;

[0072] In an exemplary embodiment, in step 3, when setting the maximum number of iterations, factors such as the size of the data set, the complexity of the algorithm, and the limitation of computing resources need to be considered.

[0073] In an exemplary embodiment, in step 3, the update formulas for constructing the matrix E, the weight matrix U, the coefficient matrix V, and the assistance matrix Z are obtained by the following steps:

[0074] The target equation can be expanded according to the Augmented Lagrange Multiplier (ALM) to obtain the Lagrange function:

[0075]

[0076] stZ≥0,V T V=I

[0077] Where;

[0078] The Karush-Kuhn-Tucker condition is used to obtain the update formulas of the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z.

[0079] In an exemplary embodiment, the update formula for constructing the matrix E is:

[0080]

[0081] Where, Q i represents the i-th row of matrix Q;

[0082] The update formula of the weight matrix U is:

[0083]

[0084] The update formula of the coefficient matrix V is:

[0085]

[0086] Where, P and W both represent construction matrices, W=(V T V+λ2I) -1 V T ;

[0087] The update formula of the assistance matrix Z is:

[0088]

[0089] Where, R represents a matrix constructed to simplify calculations, and the solution of Z is Z = max(R, 0);

[0090] The update formulas for the Lagrangian operators Y1 and Y2 and the hyperparameter value μ are:

[0091] Y1=Y1+μ(ZV),

[0092] Y2=Y2+μ(X-UV T -E)

[0093] μ=ρμ.

[0094] Where ρ represents the Lagrangian parameter in the optimization process, which indicates the degree of update of μ.

[0095] Step 4: After the number of iterations reaches a maximum or the iterations converge, the final coefficient matrix V is output after the solution is completed, that is, the clustering result is obtained.

[0096] The embodiment of the present invention also provides a clustering device for graph regular non-negative matrix decomposition, such as Figure 2 As shown, the device includes:

[0097] The data acquisition module 201 is configured to acquire matrix information data and the number of items to be clustered. The matrix information data represents user ratings of items or user-item interactions in a recommendation system, or represents user interaction information in social network analysis, or represents the co-occurrence relationship between word vectors in natural language processing, or the position information and grayscale value information of each pixel in grayscale processed image data.

[0098] The matrix construction module 202 is configured to define four initialization sparse matrices, corresponding to the construction matrix E, the weight matrix U, the coefficient matrix V, and the assistance matrix Z, and set the initial hyperparameter value μ;

[0099] Iterative solution module 203 is configured to determine the target equation and the maximum number of iterations, and iteratively solve the construction matrix E, the weight matrix U, the coefficient matrix V, the assistance matrix Z, and the Lagrangian operators Y1 and Y2 according to the update formula;

[0100] The clustering output module 204 is configured to output the final coefficient matrix V to obtain the clustering result when the number of iterations is maximum or the iterations converge.

[0101] In some embodiments, the matrix information data is represented as X∈R m×n , where each eigenvalue matrix X consists of n column vectors x i ∈R m Composition, R m Represents an m-dimensional space, m represents the dimension of the sample, and n represents the number of samples.

[0102] In some embodiments, the number of clusters to be clustered is a positive integer not exceeding 10.

[0103] In some embodiments, the initial hyperparameter value μ is set to 0.01.

[0104] In some embodiments, the objective equation is expressed as:

[0105]

[0106] stZ≥0,Z=V,V T V=I,E=X-UV T .

[0107] Where, (·) Hx Expressed as ∑ i log(1+∥(·) i ∥2), i represents the number of rows in the matrix;

[0108] Tr() represents the trace of the matrix;

[0109] I represents the identity matrix;

[0110] α, λ1, and λ2 represent hyperparameters;

[0111] T represents matrix transpose;

[0112] In some embodiments, the maximum number of iterations is determined based on the size of the data set, computational complexity, and limitations of computing resources.

[0113] In some embodiments, the update formulas for the construction matrix E, the weight matrix U, the coefficient matrix V, and the assistance matrix Z are obtained by the following steps:

[0114] Expand the objective equation to obtain the Lagrangian function:

[0115]

[0116] stZ≥0,V T V=I

[0117] The Karush-Kuhn-Tucker condition is used to obtain the update formulas of the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z.

[0118] In some embodiments, the update formula for constructing the matrix E is:

[0119]

[0120] Where, Q i represents the i-th row of matrix Q;

[0121] The update formula of the weight matrix U is:

[0122]

[0123] The update formula of the coefficient matrix V is:

[0124]

[0125] Where, P and W both represent construction matrices, W=(V T V+λ2I) -1 V T ;

[0126] The update formula of the assistance matrix Z is:

[0127]

[0128] Where, R represents a matrix constructed to simplify calculations, and the solution of Z is Z = max(R, 0);

[0129] The update formulas for the Lagrangian operators Y1 and Y2 and the hyperparameter value μ are:

[0130] Y1=Y1+μ(ZV),

[0131] Y2=Y2+μ(X-UV T -E)

[0132] μ=ρμ.

[0133] Where ρ represents the Lagrangian parameter in the optimization process, which indicates the degree of update of μ.

[0134] It should be noted that the various device structures described in this embodiment belong to the same technical concept as the method described previously, and achieve the same technical effects through the same principles, which will not be repeated here.

[0135] An embodiment of the present invention further provides a readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the method described in any of the above embodiments.

[0136] Furthermore, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present invention having equivalent elements, modifications, omissions, combinations (e.g., schemes where various embodiments intersect), adaptations, or changes. The elements in the claims are to be interpreted broadly based on the language employed in the claims and are not limited to the examples described in this specification or during the prosecution of this application, which examples are to be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, with the true scope and spirit being indicated by the following claims and the full scope of their equivalents.

[0137] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of their solutions) can be used in combination with each other. For example, those of ordinary skill in the art may use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the present invention. This should not be interpreted as an intention that a feature of an invention that is not claimed for protection is necessary for any claim. On the contrary, the subject matter of the present invention may be less than all the features of the embodiments of a particular invention. Thus, the following claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of the present invention should be determined with reference to the appended claims and the full scope of equivalents to which these claims are entitled.

Claims

1. A graph regular non-negative matrix decomposition clustering method, characterized in that: The method comprises: Obtaining matrix information data and the number of to-be-clustered pixels, wherein the matrix information data is the position information and gray value information of each pixel after the image data is grayed; Define four initialization sparse matrices, corresponding to the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z, and set the initial hyperparameter value μ; Determine the target equation and the maximum number of iterations, and iteratively solve the construction matrix E, weight matrix U, coefficient matrix V, assistance matrix Z, and Lagrangian operators Y1 and Y2 according to the update formula; When the number of iterations is maximum or the iterations converge, the final coefficient matrix V is output to obtain the clustering result; The matrix information data is represented by X∈R m×n , where each eigenvalue matrix X consists of n column vectors x i ∈R m Composition, R m Represents m-dimensional space, m represents the dimension of the sample, and n represents the number of samples; The update formulas of the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z are obtained through the following steps: Expand the objective equation to obtain the Lagrangian function: The Karush-Kuhn-Tucker condition is used to obtain the update formulas of the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z; The update formula for constructing the matrix E is: In the formula, Q=X-UV T +Y2 / μ Q i represents the i-th row of matrix Q; The update formula of the weight matrix U is: The update formula of the coefficient matrix V is: In the formula, P and W both represent construction matrices, W = (V T V+λ2I) -1 V T ; The update formula of the assistance matrix Z is: In the formula, R represents a matrix constructed to simplify calculations, and the solution of Z is Z = max(R, 0); The update formula of Lagrangian operators Y1 and Y2 and hyperparameter value μ is: Y1=Y1+μ(ZV), Y2=Y2+µ(X-UV T -BY) μ=ρμ. Where ρ represents the Lagrangian parameter in the optimization process, which indicates the degree of updating of μ.

2. The method according to claim 1, characterized in that The number to be clustered is a positive integer not exceeding 10.

3. The method according to claim 1, characterized in that The initial hyperparameter value μ is set to 0.

01.

4. The method according to claim 1, characterized in that The objective equation is expressed as: In the formula, (·) Hx Expressed as ∑ i log(1+∥(·) i ∥2), i represents the number of rows of the matrix; Tr() represents the trace of the matrix; I represents the identity matrix; α, λ1, λ2 represent hyper parameters; T stands for matrix transpose.

5. The method according to claim 1, characterized in that The maximum number of iterations is determined based on the size of the data set, computational complexity, and limitations of computing resources.

6. A clustering device for graph regular non-negative matrix decomposition, characterized in that: The device comprises: A data acquisition module is configured to acquire matrix information data and the number of items to be clustered, wherein the matrix information data represents the user's rating of items or the interaction between users and items in the recommendation system, or represents the interaction information between users in social network analysis, or represents the co-occurrence relationship between word vectors and word vectors in natural language processing, or the position information and gray value information of each pixel after the image data is grayed out; The matrix construction module is configured to define four initialization sparse matrices, corresponding to the construction matrix E, the weight matrix U, the coefficient matrix V, and the assistance matrix Z, and set the initial hyperparameter value μ; An iterative solution module is configured to determine the target equation and the maximum number of iterations, and iteratively solve the construction matrix E, the weight matrix U, the coefficient matrix V, the assistance matrix Z, and the Lagrangian operators Y1 and Y2 according to the update formula; The clustering output module is configured to output the final coefficient matrix V to obtain the clustering result when the number of iterations is maximum or the iterations converge; The update formulas of the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z are obtained through the following steps: Expand the objective equation to obtain the Lagrangian function: The Karush-Kuhn-Tucker condition is used to obtain the update formulas of the construction matrix E, weight matrix U, coefficient matrix V, and assistance matrix Z; The update formula for constructing the matrix E is: In the formula, Q=X-UV T +Y2 / μ Q i represents the i-th row of matrix Q; The update formula of the weight matrix U is: The update formula of the coefficient matrix V is: In the formula, P and W both represent construction matrices, W = (V T V+λ2I) -1 V T ; The update formula of the assistance matrix Z is: In the formula, R represents a matrix constructed to simplify calculations, and the solution of Z is Z = max(R, 0); The update formula of Lagrangian operators Y1 and Y2 and hyperparameter value μ is: Y1=Y1+μ(ZV), Y2=Y2+µ(X-UV T -BY) μ=ρμ. Where ρ represents the Lagrangian parameter in the optimization process, which indicates the degree of updating of μ. 7 . A non-transitory computer-readable storage medium storing instructions, which, when executed by a processor, executes the method according to claim 1 .

Images