Image fair clustering method and device based on symmetric non-negative matrix factorization and medium

Abstract

The invention discloses an image fair clustering method and device based on symmetric non-negative matrix factorization and a medium, and relates to the technical field of image clustering analysis. The method comprises the following steps: acquiring image data and a to-be-clustered number; defining three initialization matrixes, namely an adjacent matrix A, a clustering distribution membership matrix H and a graph Laplacian matrix L; determining a target equation and the maximum number of iterations, and iteratively solving a clustering distribution membership matrix H according to an updating formula; and outputting a final clustering distribution membership matrix H under the condition that the number of iterations is maximum or iteration converges, and obtaining a clustering result. According to the invention, by improving the objective function construction method, a balanced clustering effect with cohesive force is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image clustering analysis, and more specifically, to an image fair clustering method, device and medium based on symmetric non-negative matrix factorization. Background Art

[0002] Traditional NMF methods have achieved good performance in clustering tasks. However, it still has some limitations. The most important problem is that it overly relies on strict constraints to ensure clustering balance, resulting in a significant reduction in the internal cohesion of the clustering results. Moreover, the existing individual and group fairness methods in graph segmentation mainly rely on eigenvalue decomposition techniques, and their interpretability is generally insufficient. Summary of the Invention

[0003] To solve the above technical problems, the present invention provides an image fair clustering method, device and medium based on symmetric non-negative matrix factorization, which realizes the unity of clustering balance and cohesion by introducing an adjustable fairness regularization mechanism.

[0004] In a first aspect, the present invention provides an image fair clustering method based on symmetric non-negative matrix factorization, and the method includes:

[0005] Obtain image data and the number of clusters to be clustered; wherein, the image data is the position information and grayscale value information of each pixel point after grayscale processing of the picture data;

[0006] Define three initial matrices, namely an adjacency matrix A, a clustering assignment membership matrix H, and a graph Laplacian matrix L, and set the value of the regularization parameter λ;

[0007] Determine the objective equation and the maximum number of iterations, and iteratively solve the clustering assignment membership matrix H and the Lagrangian operator according to the update formula;

[0008] When the number of iterations reaches the maximum or the iteration converges, output the final clustering assignment membership matrix H to obtain the clustering result.

[0009] Further, the image data is represented as X ∈ R m×n , where each eigenvalue matrix X is composed of n column vectors x i ∈ R m , R m represents the m-dimensional space, m represents the dimension of the sample, and n represents the number of samples.

[0010] Further, the number of clusters to be clustered is a positive integer not exceeding 10.

[0011] Further, the objective equation is expressed as:

[0012]

[0013] such that \(H\geq0\)

[0014] Where \(\|\cdot\|_2\) represents the 2-norm; Tr represents the trace of a matrix; \(T\) represents the matrix transpose; s.t represents the constraint condition, and min represents taking the minimum value.

[0015] Furthermore, determine the maximum number of iterations according to the scale of the data set, the computational complexity, and the limitation of computing resources.

[0016] Furthermore, construct the graph Laplacian matrix \(L\) in the following way:

[0017] Construct a positive term matrix \(P\), expressed as:

[0018]

[0019] Where \(g\) i represents the class to which sample \(i\) belongs, and \(g\) j represents the class to which sample \(j\) belongs;

[0020] Construct a negative term matrix \(N\), expressed as:

[0021]

[0022] Define the first matrix \(C\), \(C = P - N\);

[0023] Construct the graph Laplacian matrix \(L\) based on the first matrix and the second matrix, \(L = D - C\), where the second matrix \(D\) is expressed as:

[0024]

[0025] Where \(D\) ii represents the value of the \(i\)-th row and \(i\)-th column of the second matrix \(D\), \(n\) represents the number of samples, and \(C\) ij represents the value of the \(i\)-th row and \(j\)-th column of the first matrix \(C\).

[0026] Furthermore, determine the update formula of the clustering assignment membership matrix \(H\) in the following way:

[0027] Expand the objective equation to obtain the Lagrangian function \(Lag\):

[0028] \(Lag=\|A - HH\) T \|+\lambda tr(H T LH)+tr(\alpha H T )

[0029] Where \(A\) is the adjacency matrix, \(H\) is the clustering assignment membership matrix, \(L\) is the graph Laplacian matrix, \(\lambda\) represents the regularization parameter, and \(\alpha\) represents the Lagrange multiplier;

[0030] The Karush-Kuhn-Tucker conditions are used to obtain the update formula for constructing the membership matrix H.

[0031] Furthermore, the update formula for constructing the membership matrix H is as follows:

[0032]

[0033] In the formula, to ensure non-negativity, the positive and negative elements in the Laplacian matrix L are processed separately, L = L + -L - ,, h ik is the value of the i-th row and k-th column of the matrix H; (AH) ik is the value of the i-th row and k-th column of the matrix AH; (L - H) ik is the value of the i-th row and k-th column of the matrix L - H; (HHTH) ik is the value of the i-th row and k-th column of the matrix (HH T H); (L + H) ik is the value of the i-th row and k-th column of the matrix (L + H).

[0034] In a second aspect, the present invention provides an image fair clustering device based on symmetric non-negative matrix factorization. The device includes:

[0035] A data acquisition module configured to acquire image data and the number of clusters to be clustered; wherein, the image data is the position information and gray value information of each pixel point after gray-scale processing of the picture data;

[0036] A matrix construction module configured to define three initial matrices, namely an adjacency matrix A, a clustering assignment membership matrix H, and a graph Laplacian matrix L, and set the value of the regularization parameter λ;

[0037] An iterative solution module configured to determine the objective equation and the maximum number of iterations, and iteratively solve the clustering assignment membership matrix H and the Lagrangian operator according to the update formula;

[0038] A clustering output module configured to output the final clustering assignment membership matrix H and obtain the clustering result when the number of iterations reaches the maximum or the iteration converges.

[0039] In a third aspect, the present invention provides a readable storage medium storing one or more programs, which can be executed by one or more processors to implement the method described above.

[0040] The present invention has at least the following beneficial effects:

[0041] By improving the construction method of the objective function and adding better constraint conditions to capture deep information features, the present invention enhances the robustness and accuracy of image classification and obtains better clustering results. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 FIG. shows a flowchart of a fair clustering method for symmetric non - negative matrix factorization according to an embodiment of the present invention.

[0043] Figure 2 FIG. shows a structural diagram of a clustering device for graph - regularized non - negative matrix factorization according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be described in detail below in conjunction with the drawings and specific embodiments. The embodiments of the present invention will be further described in detail below in conjunction with the drawings and specific examples, but shall not be construed as a limitation to the present invention. For the steps described herein, if there is no necessity for a sequential relationship between them, the order in which they are described as examples herein shall not be regarded as a limitation. Those skilled in the art should know that they can adjust the order as long as the logic between them is not destroyed and the entire process cannot be realized.

[0045] Glossary:

[0046] Karush - Kuhn - Tucker conditions: In an optimization problem, a set of necessary conditions for determining whether a candidate solution is a local optimal solution. It is a constraint condition for nonlinear programming problems and is applicable to problems with equality and inequality constraints. The KKT conditions combine the objective function, equality constraints, and inequality constraints, and determine possible optimal solutions by examining gradients and Lagrange multipliers.

[0047] An embodiment of the present invention provides an image fair clustering method based on symmetric non - negative matrix factorization. As Figure 1 shown, FIG. is a flowchart of an image fair clustering method based on symmetric non - negative matrix factorization. The method includes steps 1 to 4, which are introduced in detail as follows.

[0048] Step 1: Obtain the image data X ∈ R m×n for clustering analysis and the number of clusters to be formed. Each eigenvalue matrix X consists of n column vectors x i ∈ R m where R m represents an m - dimensional space, m represents the dimension of the samples, and n represents the number of samples.

[0049] It should be noted that the image data described in this article contains a matrix regarding the relationships between observation objects. In different application scenarios, the image data represents different data. For example, in a recommendation system, the image data represents the ratings of items by users or the interaction behaviors between users and items; in social network analysis, the image data represents the interactions between users; in natural language processing, the image data represents the co-occurrence relationships between words. In image recognition, the image data is the position information and grayscale value information of each pixel point after grayscale processing of the picture data.

[0050] The number of clusters to be clustered, i.e., the clustering quantity, is an integer, and is usually determined according to the specific problem and the characteristics of the data. To ensure the practicality of the results, when the amount of data is not large, it is recommended to set the R value within 10. This can make the clustering quantity appropriate, avoid generating too many clustering clusters, and at the same time improve the interpretability and practicality of the clustering results.

[0051] Step 2: Define three initial matrices, namely the adjacency matrix A, the clustering assignment membership matrix H, and the graph Laplacian matrix L, and set the value of the regularization parameter λ.

[0052] Exemplarily, set the initial hyperparameter value μ to 0.01.

[0053] In an exemplary embodiment, the graph Laplacian matrix L is constructed in the following manner:

[0054] Construct a positive term matrix P, expressed as:

[0055]

[0056] In the formula, g i represents the category to which sample i belongs, and g j represents the category to which sample j belongs;

[0057] Construct a negative term matrix N, expressed as:

[0058]

[0059] Define the first matrix C, C = P - N;

[0060] Based on the first matrix and the second matrix, construct the graph Laplacian matrix L, L = D - C, where the second matrix D is expressed as:

[0061]

[0062] In the formula, D ii represents the value of the i-th row and i-th column of the second matrix D, n represents the number of samples, and C ij represents the value of the i-th row and j-th column of the first matrix C.

[0063] Step 3: Determine the target equation and the maximum number of iterations, and iteratively solve according to the update formula to construct the membership matrix H and the Lagrange operator α.

[0064] In an exemplary embodiment, in Step 3, the target equation is:

[0065]

[0066] Where, ||||2 represents the 2-norm; Tr represents the trace of the matrix; T represents the matrix transpose; s.t represents the constraint condition, and min represents taking the minimum value.

[0067] In an exemplary embodiment, in Step 3, when setting the maximum number of iterations, factors such as the scale of the dataset, the complexity of the algorithm, and the limitations of computing resources need to be considered.

[0068] In an exemplary embodiment, in Step 3, the update formula of the clustering assignment membership matrix H is obtained through the following steps:

[0069] The target equation can be expanded according to the Augmented Lagrange Multiplier (ALM) to obtain the Lagrangian function L:

[0070] lag = ||A - HH T || + λtr(H T LH) + tr(αH T ) and use the Karush-Kuhn-Tucker conditions to obtain the update formula of the clustering assignment membership matrix H.

[0071] In an exemplary embodiment, the update formula of the clustering assignment membership matrix H is:

[0072]

[0073] Where, to ensure non-negativity, the positive and negative elements in the Laplacian matrix L are processed separately, L = L + -L - ,, h ik is the value of the i-th row and k-th column of the matrix H; (AH) ik is the value of the i-th row and k-th column of the matrix AH; (L - H) ik is the value of the i-th row and k-th column of the matrix L - H; (HH T H) ik is the value of the i-th row and k-th column of the matrix (HH T H); (L + H) ik is the value of the i-th row and k-th column of the matrix (L +The value of the i-th row and k-th column of (H).

[0074] Step 4: After the maximum number of iterations or when the iteration converges, perform argmax(H) on the clustering assignment membership matrix H, that is, find the maximum value among the membership degrees of each data point i in all clusters, and return the index of the maximum value to finally obtain the clustering result argmax(H).

[0075] Next, the embodiments of the present invention will prove the feasibility and progressiveness of this method by combining a specific numerical example. Taking a face image as an example, based on the input face image, face image clustering is achieved, that is, the function of face recognition is realized.

[0076] Simulation conditions:

[0077] The hardware test platform used in the simulation experiment of the present invention is: the processor is Inter Core i7, the main frequency is 2.20GHz, and the memory is 32GB; the software platform is: Windows 10 Home Edition 64-bit operating system, and Matlab R2022a is used for simulation testing.

[0078] Dataset:

[0079] The present invention uses the MTFL dataset (Multi-task Facial Landmark Dataset, MTFL) for experimental verification. The MTFL dataset contains 12,995 images for face recognition and key point detection, and each image is labeled with attributes such as gender, smile, whether wearing glasses, and head pose angle. The present invention takes whether wearing glasses as the protected attribute, and takes gender as the target attribute for evaluating the clustering effectiveness. In order to construct a balanced fair clustering dataset, the present invention randomly selects 1,000 images wearing glasses and 1,000 images not wearing glasses from the MTFL dataset, a total of 2,000 images, to construct an experimental dataset.

[0080] Evaluation metrics:

[0081] The present invention uses Balance and Entropy as evaluation metrics to measure the balance degree of the clustering result on the protected attribute. The Balance metric measures the distribution balance of each cluster in the protected attribute in the clustering result, and the larger the value, the fairer the clustering result; the Entropy metric measures the distribution entropy of each cluster in the protected attribute, and the larger the value, the more uniform the distribution of the protected attribute within the cluster, and the fairer the clustering result. The present invention evaluates the performance of the NMF and FairNMF methods in the fair clustering task through these two metrics.

[0082] Experimental settings:

[0083] The present invention proposes an image fair clustering method (FairNMF) based on symmetric non - negative matrix factorization and conducts a comparative experiment with the non - negative matrix factorization method (NMF). To comprehensively evaluate the performance of the method, each method runs 10 experiments, and the average value is taken to reduce the influence of randomness on the results. In the experiment, the parameter λ (controlling fairness) of the FairNMF method is adjusted to the optimal value through preliminary experiments.

[0084] Experimental results and analysis:

[0085] Table 1 shows the Balance and Entropy metrics of FairNMF and NMF on the MTFL dataset. It can be seen from Table 1 that FairNMF performs better than the NMF method in both the Balance and Entropy metrics. Specifically, the Balance values of FairNMF are higher than those of NMF, indicating that the distribution of FairNMF in protecting the attribute (whether wearing glasses) is more balanced; at the same time, the Entropy value of FairNMF is higher than that of NMF, indicating that the distribution of protected attributes within the clusters is more uniform and the clustering results are fairer.

[0086] Table 1 Index parameters

[0087]

[0088]

[0089] In summary, the experimental results of the method proposed by the present invention on the MTFL dataset show that it can significantly improve the fairness of clustering while maintaining the effectiveness of clustering, is superior to the traditional NMF method, and is applicable to image clustering tasks that require considering fairness constraints.

[0090] The embodiment of the present invention also provides an image fair clustering device based on symmetric non - negative matrix factorization, as Figure 2 shown. The image fair clustering device 200 based on symmetric non - negative matrix factorization includes:

[0091] A data acquisition module 201, configured to acquire image data and the number of clusters to be clustered; wherein, the image data includes a matrix about the relationship between observation objects;

[0092] A matrix construction module 202, configured to define three initial matrices, namely an adjacency matrix A, a clustering assignment membership matrix H, and a graph Laplacian matrix L, and set the value of the regularization parameter λ;

[0093] An iterative solution module 203, configured to determine the objective equation and the maximum number of iterations, and iteratively solve the clustering assignment membership matrix H and the Lagrangian operator according to the update formula;

[0094] The clustering output module 204 is configured to output the final clustering assignment membership matrix H and obtain the clustering result when the maximum number of iterations is reached or the iteration converges.

[0095] In some embodiments, the image data is represented as X ∈ R m×n , where each eigenvalue matrix X consists of n column vectors x i ∈ R m , R m represents an m-dimensional space, m represents the dimension of the sample, and n represents the number of samples.

[0096] In some embodiments, the number of clusters to be clustered is a positive integer not exceeding 10.

[0097] In some embodiments, the objective equation is expressed as:

[0098]

[0099] s.t H≥0

[0100] In the formula, ||||2 represents the 2-norm; Tr represents the trace of the matrix; T represents the matrix transpose; s.t represents the constraint condition, and min represents taking the minimum value.

[0101] In some embodiments, the maximum number of iterations is determined according to the scale of the data set, the computational complexity, and the limitation of computing resources.

[0102] In some embodiments, the matrix construction module is further configured to construct the graph Laplacian matrix L in the following manner:

[0103] Construct a positive term matrix P, expressed as:

[0104]

[0105] In the formula, g i represents the category to which sample i belongs, and g j represents the category to which sample j belongs;

[0106] Construct a negative term matrix N, expressed as:

[0107]

[0108] Define the first matrix C, C = P - N;

[0109] Construct the graph Laplacian matrix L based on the first matrix and the second matrix, L = D - C, where the second matrix D is expressed as:

[0110]

[0111] In the formula, D iirepresents the value of the i-th row and i-th column of the second matrix D, n represents the number of samples, and C ij represents the value of the i-th row and j-th column of the first matrix C..

[0112] In some embodiments, the iterative solution module is further configured to determine the update formula of the clustering assignment membership matrix H in the following manner:

[0113] Expand the objective equation to obtain the Lagrangian function Lag:

[0114] lag = ||A - HH T || + λtr(H T LH) + tr(αH T )

[0115] In the formula, A is the adjacency matrix, H is the clustering assignment membership matrix, L is the graph Laplacian matrix, λ represents the regularization parameter, and α represents the Lagrange multiplier;

[0116] Use the Karush-Kuhn-Tucker conditions to obtain the update formula for constructing the membership matrix H.

[0117] In some embodiments, the update formula for constructing the membership matrix H is expressed as:

[0118]

[0119] In the formula, to ensure non-negativity, we separate the positive and negative elements of the Laplacian matrix L, L = L + - L - , h ik is the value of the i-th row and k-th column of the matrix H; (AH) ik is the value of the i-th row and k-th column of the matrix AH; (L - H) ik is the value of the i-th row and k-th column of the matrix L - H; (HH T H) ik is the value of the i-th row and k-th column of the matrix (HH T H); (L + H) ik is the value of the i-th row and k-th column of the matrix (L + H).

[0120] It should be noted that the structures of the various image fair clustering devices based on symmetric non-negative matrix factorization described in this embodiment belong to the same technical concept as the previously described image fair clustering method based on symmetric non-negative matrix factorization, and achieve the same beneficial effects through the same principle, which will not be elaborated here.

[0121] An embodiment of the present invention further provides a readable storage medium storing one or more programs, which can be executed by one or more processors to implement the method described in any of the foregoing embodiments.

[0122] The above embodiments are only used to illustrate the present application, rather than limiting the present application. Those of ordinary skill in the relevant technical field can also make various changes and modifications without departing from the spirit and scope of the present application. Therefore, all equivalent technical solutions also belong to the scope of the present application. The patent protection scope of the present application shall be defined by the claims.

Claims

1. A fair image clustering method based on symmetric non-negative matrix decomposition, characterized in that: The method comprises: Obtain image data and the number of to-be-clustered images; wherein the image data is the position information and gray value information of each pixel after the image data is grayed; Define three initialization matrices, namely the adjacency matrix A, the cluster assignment membership matrix H and the graph Laplacian matrix L, and set the regularization parameter value λ; Determine the target equation and the maximum number of iterations, and iteratively solve the clustering assignment membership matrix H and the Lagrangian operator according to the update formula; When the number of iterations is maximum or the iterations converge, the final clustering assignment membership matrix H is output to obtain the clustering result.

2. The image fair clustering method based on symmetric non-negative matrix decomposition according to claim 1, characterized in that: The image data is represented as X∈R m×n , where each eigenvalue matrix X consists of n column vectors x i ∈R m Composition, R m Represents m-dimensional space, m represents the dimension of the sample, n represents the number of samples, and the adjacency matrix A is constructed according to the Gaussian kernel function. The specific formula is as follows: Where δ is the thermal nuclear parameter.

3. The image fair clustering method based on symmetric non-negative matrix decomposition according to claim 1, characterized in that: The number to be clustered is a positive integer not exceeding 10.

4. The image fair clustering method based on symmetric non-negative matrix decomposition according to claim 1, characterized in that: The objective equation is expressed as: In the formula, ||||2 represents the 2-norm; Tr represents the trace of the matrix; T represents the matrix transpose; st represents the constraint condition, and min represents the minimum value.

5. The image fair clustering method based on symmetric non-negative matrix decomposition according to claim 1, characterized in that: The maximum number of iterations is determined based on the size of the data set, computational complexity, and limitations of computing resources.

6. The image fair clustering method based on symmetric non-negative matrix decomposition according to claim 1, characterized in that: The graph Laplacian matrix L is constructed as follows: Construct the positive matrix P, expressed as: In the formula, g i Indicates the category to which sample i belongs, g j Indicates the category to which sample j belongs; Construct the negative matrix N, expressed as: Define the first matrix C, C = PN; The graph Laplacian matrix L is constructed based on the first matrix and the second matrix, L=DC, where the second matrix D is a diagonal matrix, expressed as: Where D ii represents the value of the i-th row and i-th column of the second matrix D, n represents the number of samples, C ij Represents the value of the i-th row and j-th column of the first matrix C.

7. The image fair clustering method based on symmetric non-negative matrix decomposition according to claim 4, characterized in that: The update formula of the cluster assignment membership matrix H is determined as follows: Expand the objective equation to obtain the Lagrangian function Lag: lag=||A-HH T ||+λtr(H T LH)+tr(αH T ) Where A is the adjacency matrix, H is the cluster assignment membership matrix, L is the graph Laplace matrix, λ represents the regularization parameter, and α represents the Lagrange multiplier. The Karush-Kuhn-Tucker condition is used to obtain the update formula for constructing the membership matrix H.

8. The image fair clustering method based on symmetric non-negative matrix decomposition according to claim 7, characterized in that: The update formula for constructing the membership matrix H is expressed as: In the formula, in order to ensure non-negativity, the positive elements and negative elements in the Laplace matrix L are processed separately, L = L + -L - ,h ik is the value of the i-th row and k-th column of matrix H; (AH) ik is the value of the i-th row and k-th column of matrix AH; (L - H) ik is the matrix L - The value of the i-th row and k-th column of H; (HH T H) ik For the matrix (HH T H) the value of the i-th row and k-th column; (L + H) ik is the matrix (L + H)'s value in the i-th row and k-th column.

9. A fair image clustering device based on symmetric non-negative matrix decomposition, characterized in that: The device comprises: A data acquisition module is configured to acquire image data and the number of to-be-clustered images; wherein the image data is the position information and gray value information of each pixel after the image data is grayed; The matrix construction module is configured to define three initialization matrices, namely, the adjacency matrix A, the cluster assignment membership matrix H, and the graph Laplacian matrix L, and set the regularization parameter value λ; An iterative solution module is configured to determine a target equation and a maximum number of iterations, and iteratively solve a clustering allocation membership matrix H and a Lagrangian operator according to an update formula; The clustering output module is configured to output the final clustering assignment membership matrix H to obtain the clustering result when the number of iterations is maximum or the iterations converge. 10 . A non-transitory computer-readable storage medium storing instructions, which, when executed by a processor, perform the method according to claim 1 .

Images