Image Fair Clustering Method, Device, and Medium Based on Symmetric Nonnegative Matrix Factorization

Abstract

This invention discloses a fair image clustering method, apparatus, and medium based on symmetric nonnegative matrix factorization, relating to the field of image clustering analysis technology. The method includes: acquiring image data and the number of cells to be clustered; defining three initialization matrices: an adjacency matrix A, a clustering membership matrix H, and a graph Laplacian matrix L; determining the objective function and the maximum number of iterations; iteratively solving the clustering membership matrix H according to the update formula; and outputting the final clustering membership matrix H when the number of iterations is maximized or the iterations converge, thus obtaining the clustering result. This invention achieves a balanced and cohesive clustering effect by improving the objective function construction method.

Description

Technical Field

[0001] This invention relates to the field of image clustering analysis technology, and more specifically, to an image fair clustering method, apparatus, and medium based on symmetric nonnegative matrix factorization. Background Technology

[0002] Traditional NMF methods have achieved good performance in clustering tasks. However, they still have some limitations. The most significant problem is their over-reliance on strict constraints to ensure cluster balance, which leads to a significant reduction in the internal cohesion of the clustering results. Moreover, existing methods for individual and group fairness in graph segmentation are mainly based on eigenvalue decomposition techniques, which generally lack interpretability. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides an image fair clustering method, apparatus, and medium based on symmetric nonnegative matrix factorization. By introducing an adjustable fairness regularization mechanism, it achieves a balance between clustering balance and cohesion.

[0004] In a first aspect, the present invention provides a fair image clustering method based on symmetric nonnegative matrix factorization, the method comprising:

[0005] Obtain image data and the number of cells to be clustered; wherein, the image data is the position information and grayscale value information of each pixel after grayscale processing of image data;

[0006] Define three initialization matrices: adjacency matrix A, cluster assignment membership matrix H, and graph Laplacian matrix L, and set the regularization parameter value λ.

[0007] Determine the objective equation and the maximum number of iterations, and iteratively solve the clustering assignment membership matrix H and the Lagrange operator according to the update formula;

[0008] If the number of iterations is maximized or the iterations converge, the final clustering membership matrix H is output, and the clustering result is obtained.

[0009] Furthermore, the image data is represented as X∈R m×n Each eigenvalue matrix X consists of n column vectors x i ∈R m Composition, R m Let m represent an m-dimensional space, where m represents the dimension of the sample and n represents the number of samples.

[0010] Furthermore, the number of clusters to be clustered is a positive integer not exceeding 10.

[0011] Furthermore, the objective equation is expressed as:

[0012]

[0013] st H≥0

[0014] In the formula, ||||2 represents the 2-norm; Tr represents the trace of the matrix; T represents the transpose of the matrix; st represents the constraint condition; and min represents taking the minimum value.

[0015] Furthermore, the maximum number of iterations is determined based on the size of the dataset, computational complexity, and limitations of computational resources.

[0016] Furthermore, the graph Laplacian matrix L is constructed as follows:

[0017] Construct a positive term matrix P, expressed as:

[0018]

[0019] In the formula, g i G represents the category to which sample i belongs. j Indicates the category to which sample j belongs;

[0020] Construct a negative term matrix N, denoted as:

[0021]

[0022] Define the first matrix C, C = PN;

[0023] Construct a graph Laplacian matrix L, L = DC, based on the first and second matrices, where the second matrix D is represented as:

[0024]

[0025] In the formula, D ii Let C represent the value in the i-th row and i-th column of the second matrix D, where n represents the number of samples. ij This represents the value in the i-th row and j-th column of the first matrix C.

[0026] Furthermore, the update formula for the clustering assignment membership matrix H is determined as follows:

[0027] Expanding the objective equation yields the Lagrange function Lag:

[0028] lag=||A-HH T ||+λtr(H T LH)+tr(αH T )

[0029] In the formula, A is the adjacency matrix, H is the clustering assignment membership matrix, L is the graph Laplacian matrix, λ represents the regularization parameter, and α represents the Lagrange multiplier;

[0030] The update formula for constructing the membership matrix H is obtained using the Karush-Kuhn-Tucker conditions.

[0031] Furthermore, the update formula for constructing the membership matrix H is:

[0032]

[0033] In the formula, to ensure non-negativity, the positive and negative elements of the Laplace matrix L are treated separately, L = L + -L - ,,h ik Let be the value in the i-th row and k-th column of matrix H; (AH) ik Let AH be the value in the i-th row and k-th column of matrix AH; (L - H) ik For matrix L - The value of H in the i-th row and k-th column; (HHTH) ik For matrix (HH) T The value of the i-th row and k-th column of H); (L + H) ik For matrix (L) + The value of the i-th row and k-th column of H).

[0034] Secondly, the present invention provides an image fair clustering device based on symmetric nonnegative matrix factorization, the device comprising:

[0035] The data acquisition module is configured to acquire image data and the number of data to be clustered; wherein, the image data is the position information and grayscale value information of each pixel after grayscale processing of image data;

[0036] The matrix construction module is configured to define three initialization matrices: an adjacency matrix A, a clustering assignment membership matrix H, and a graph Laplacian matrix L, and to set the regularization parameter value λ.

[0037] The iterative solution module is configured to determine the objective equation and the maximum number of iterations, and iteratively solve the clustering assignment membership matrix H and the Lagrange operator according to the update formula.

[0038] The clustering output module is configured to output the final clustering assignment membership matrix H when the number of iterations is maximized or the iterations converge, thus obtaining the clustering result.

[0039] Thirdly, the present invention provides a readable storage medium storing one or more programs that can be executed by one or more processors to implement the method described above.

[0040] The present invention has at least the following beneficial effects:

[0041] This invention improves the objective function construction method and adds better constraints to capture deeper information features, thereby enhancing the robustness and accuracy of image classification and achieving better clustering results. Attached Figure Description

[0042] Figure 1 A flowchart of a fair clustering method based on symmetric nonnegative matrix factorization according to an embodiment of the present invention is shown.

[0043] Figure 2 A structural diagram of a clustering apparatus for graph regular nonnegative matrix factorization according to an embodiment of the present invention is shown. Detailed Implementation

[0044] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and specific examples, but this is not intended to limit the present invention. If there is no necessary sequential relationship between the various steps described herein, the order in which they are described as examples should not be considered a limitation. Those skilled in the art should understand that the order can be adjusted, as long as it does not disrupt the logical consistency between them and render the entire process impossible.

[0045] Definitions:

[0046] The Karush-Kuhn-Tucker (KKT) conditions are a set of necessary conditions used in optimization problems to determine whether a candidate solution is a local optimum. They are a type of constraint used for nonlinear programming problems, applicable to problems with both equality and inequality constraints. The KKT conditions combine the objective function, equality constraints, and inequality constraints, determining possible optimal solutions by examining the gradient and Lagrange multipliers.

[0047] This invention provides a fair image clustering method based on symmetric nonnegative matrix factorization. For example... Figure 1 The diagram shows a flowchart of a fair image clustering method based on symmetric nonnegative matrix factorization. The method includes steps 1 to 4, which are described in detail below.

[0048] Step 1: Obtain image data X∈R for cluster analysis m×n And the number of clusters required, where each eigenvalue matrix X consists of n column vectors x i ∈R m Composition, R m It is represented as an m-dimensional space, where m represents the dimension of the sample and n represents the number of samples.

[0049] It should be noted that the image data described in this article includes matrices relating to the relationships between observed objects. Image data represents different data in different application scenarios. For example, in recommendation systems, image data represents user ratings of items or user-item interactions; in social network analysis, image data represents interactions between users; in natural language processing, image data represents co-occurrence relationships between words. In image recognition, image data consists of the location and grayscale value information of each pixel after grayscale processing of the image data.

[0050] The number of clusters required is called the cluster size, which is an integer and is usually determined based on the specific problem and the characteristics of the data. To ensure the practicality of the results, it is recommended to set the R value below 10 when the amount of data is small. This ensures a moderate number of clusters, avoids generating too many clusters, and improves the interpretability and practicality of the clustering results.

[0051] Step 2: Define three initialization matrices: adjacency matrix A, cluster assignment membership matrix H, and graph Laplacian matrix L, and set the regularization parameter value λ.

[0052] For example, the initial hyperparameter value μ is set to 0.01.

[0053] In an exemplary embodiment, the graph Laplacian matrix L is constructed as follows:

[0054] Construct a positive term matrix P, expressed as:

[0055]

[0056] In the formula, g i G represents the category to which sample i belongs. j Indicates the category to which sample j belongs;

[0057] Construct a negative term matrix N, denoted as:

[0058]

[0059] Define the first matrix C, C = PN;

[0060] Construct a graph Laplacian matrix L, L = DC, based on the first and second matrices, where the second matrix D is represented as:

[0061]

[0062] In the formula, D ii Let C represent the value in the i-th row and i-th column of the second matrix D, where n represents the number of samples. ij This represents the value in the i-th row and j-th column of the first matrix C.

[0063] Step 3: Determine the objective equation and the maximum number of iterations, and construct the membership matrix H and the Lagrange operator α by iteratively solving according to the update formula.

[0064] In an exemplary embodiment, in step 3, the objective equation is:

[0065]

[0066] In the formula, ||||2 represents the 2-norm; Tr represents the trace of the matrix; T represents the transpose of the matrix; st represents the constraint condition; and min represents taking the minimum value.

[0067] In one exemplary embodiment, when setting the maximum number of iterations in step 3, factors such as the size of the dataset, the complexity of the algorithm, and the limitations of computing resources need to be taken into account.

[0068] In an exemplary embodiment, in step 3, the update formula for the clustering assignment membership matrix H is obtained through the following steps:

[0069] The objective equation can be expanded using the Augmented Lagrange Multiplier (ALM) to obtain the Lagrangian function L:

[0070] lag=||A-HH T ||+λtr(H T LH)+tr(αH T The update formula for the cluster assignment membership matrix H is obtained using the Karush-Kuhn-Tucker conditions.

[0071] In an exemplary embodiment, the update formula for the clustering assignment membership matrix H is:

[0072]

[0073] In the formula, to ensure non-negativity, the positive and negative elements of the Laplace matrix L are treated separately, L = L + -L - ,,h ik Let be the value in the i-th row and k-th column of matrix H; (AH) ik Let AH be the value in the i-th row and k-th column of matrix AH; (L - H) ik For matrix L - The value of H in the i-th row and k-th column; (HH T H) ik For matrix (HH) T The value of the i-th row and k-th column of H); (L + H) ik For matrix (L) +The value of the i-th row and k-th column of H).

[0074] Step 4: After the maximum number of iterations or the iterations converge, perform argmax(H) processing on the clustering membership matrix H. That is, for each data point i, find the maximum value among all clusters and return the index of the maximum value to finally obtain the clustering result argmax(H).

[0075] The following embodiments of the present invention will demonstrate the feasibility and advancement of the method with a specific example. Taking a face image as an example, based on the input face image, the method achieves face image clustering, that is, realizes the function of face recognition.

[0076] Simulation conditions:

[0077] The hardware testing platform used in the simulation experiment of this invention is: an Intel Core i7 processor with a main frequency of 2.20GHz and 32GB of memory; the software platform is: Windows 10 Home Edition 64-bit operating system and Matlab R2022a for simulation testing.

[0078] Dataset:

[0079] This invention uses the MTFL (Multi-task Facial Landmark Dataset) for experimental verification. The MTFL dataset contains 12,995 images for face recognition and keypoint detection, each labeled with attributes such as gender, smile, whether glasses are worn, and head pose angle. This invention uses whether glasses are worn as a protected attribute and gender as the target attribute for evaluating clustering effectiveness. To construct a balanced and fair clustering dataset, this invention randomly selects 1,000 images of people wearing glasses and 1,000 images of people not wearing glasses from the MTFL dataset, totaling 2,000 images, to construct the experimental dataset.

[0080] Evaluation indicators:

[0081] This invention uses Balance and Entropy as evaluation metrics to measure the balance of protection attributes in clustering results. The Balance metric measures the distribution balance of protection attributes among clusters in the clustering results; a higher value indicates a fairer clustering result. The Entropy metric measures the distribution entropy of protection attributes among clusters; a higher value indicates a more uniform distribution of protection attributes within the cluster, and a fairer clustering result. This invention uses these two metrics to evaluate the performance of NMF and FairNMF methods on fair clustering tasks.

[0082] Experimental setup:

[0083] This invention proposes a fair image clustering method based on symmetric nonnegative matrix factorization (FairNMF) and compares it with the nonnegative matrix factorization method (NMF). To comprehensively evaluate the performance of each method, 10 experiments were run for each method, and the average value was taken to reduce the impact of randomness on the results. In the experiments, the parameter λ (controlling fairness) of the FairNMF method was adjusted to its optimal value through preliminary experiments.

[0084] Experimental Results and Analysis:

[0085] Table 1 shows the Balance and Entropy metrics of FairNMF and NMF on the MTFL dataset. As can be seen from Table 1, FairNMF outperforms the NMF method in both Balance and Entropy metrics. Specifically, FairNMF's Balance values ​​are higher than NMF's, indicating that FairNMF has a more balanced distribution of protective attributes (whether glasses are worn); simultaneously, FairNMF's Entropy values ​​are higher than NMF's, indicating a more uniform distribution of protective attributes within its clusters and fairer clustering results.

[0086] Table 1 Indicator Parameters

[0087]

[0088]

[0089] In summary, the experimental results of the proposed method on the MTFL dataset show that it can significantly improve the fairness of clustering while maintaining the effectiveness of clustering, which is superior to the traditional NMF method and is suitable for image clustering tasks that require consideration of fairness constraints.

[0090] This invention also provides an image fair clustering device based on symmetric nonnegative matrix factorization, such as... Figure 2 As shown, the image fair clustering device 200 based on symmetric nonnegative matrix factorization includes:

[0091] The data acquisition module 201 is configured to acquire image data and the number of clusters to be clustered; wherein the image data contains a matrix relating the relationships between the observed objects;

[0092] The matrix construction module 202 is configured to define three initialization matrices, namely the adjacency matrix A, the cluster assignment membership matrix H, and the graph Laplacian matrix L, and to set the regularization parameter value λ.

[0093] The iterative solution module 203 is configured to determine the objective equation and the maximum number of iterations, and iteratively solve the clustering assignment membership matrix H and the Lagrange operator according to the update formula.

[0094] The clustering output module 204 is configured to output the final clustering assignment membership matrix H when the number of iterations is maximized or the iterations converge, thus obtaining the clustering result.

[0095] In some embodiments, the image data is represented as X∈R m×n Each eigenvalue matrix X consists of n column vectors x i ∈R m Composition, R m Let m represent an m-dimensional space, where m represents the dimension of the sample and n represents the number of samples.

[0096] In some embodiments, the number of clusters to be clustered is a positive integer not exceeding 10.

[0097] In some embodiments, the objective equation is expressed as:

[0098]

[0099] st H≥0

[0100] In the formula, ||||2 represents the 2-norm; Tr represents the trace of the matrix; T represents the transpose of the matrix; st represents the constraint condition; and min represents taking the minimum value.

[0101] In some embodiments, the maximum number of iterations is determined based on the size of the dataset, computational complexity, and limitations of computational resources.

[0102] In some embodiments, the matrix construction module is further configured to construct the Graph Laplacian matrix L in the following manner:

[0103] Construct a positive term matrix P, expressed as:

[0104]

[0105] In the formula, g i G represents the category to which sample i belongs. j Indicates the category to which sample j belongs;

[0106] Construct a negative term matrix N, denoted as:

[0107]

[0108] Define the first matrix C, C = PN;

[0109] Construct a graph Laplacian matrix L, L = DC, based on the first and second matrices, where the second matrix D is represented as:

[0110]

[0111] In the formula, D iiLet C represent the value in the i-th row and i-th column of the second matrix D, where n represents the number of samples. ij Let represent the value in the i-th row and j-th column of the first matrix C.

[0112] In some embodiments, the iterative solution module is further configured to determine the update formula for the cluster assignment membership matrix H in the following manner:

[0113] Expanding the objective equation yields the Lagrange function Lag:

[0114] lag=||A-HH T ||+λtr(H T LH)+tr(αH T )

[0115] In the formula, A is the adjacency matrix, H is the clustering assignment membership matrix, L is the graph Laplacian matrix, λ represents the regularization parameter, and α represents the Lagrange multiplier;

[0116] The update formula for constructing the membership matrix H is obtained using the Karush-Kuhn-Tucker conditions.

[0117] In some embodiments, the update formula for constructing the membership matrix H is expressed as:

[0118]

[0119] In the formula, to ensure non-negativity, we treat the positive and negative elements of the Laplace matrix L separately, L = L + -L - ,h ik Let be the value in the i-th row and k-th column of matrix H; (AH) ik Let AH be the value in the i-th row and k-th column of matrix AH; (L - H) ik For matrix L - The value of H in the i-th row and k-th column; (HH T H) ik For matrix (HH) T The value of the i-th row and k-th column of H); (L + H) ik For matrix (L) + The value of the i-th row and k-th column of H).

[0120] It should be noted that the structures of the various image fair clustering devices based on symmetric nonnegative matrix factorization described in this embodiment belong to the same technical concept as the previously described image fair clustering method based on symmetric nonnegative matrix factorization, and achieve the same beneficial effects through the same principle, which will not be elaborated here.

[0121] This invention also provides a readable storage medium storing one or more programs that can be executed by one or more processors to implement the methods described in any of the above embodiments.

[0122] The above embodiments are only used to illustrate this application and are not intended to limit this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of this application. Therefore, all equivalent technical solutions also fall within the scope of this application, and the patent protection scope of this application should be defined by the claims.

Claims

1. A fair image clustering method based on symmetric nonnegative matrix factorization, characterized in that, The method includes: Obtain image data and the number of cells to be clustered; wherein, the image data is the position information and grayscale value information of each pixel after grayscale processing of image data; Define three initialization matrices: adjacency matrix A, cluster assignment membership matrix H, and graph Laplacian matrix L, and set the regularization parameter value λ. Determine the objective equation and the maximum number of iterations, and iteratively solve the clustering assignment membership matrix H and the Lagrange operator according to the update formula; If the number of iterations is maximized or the iterations converge, the final clustering membership matrix H is output to obtain the clustering result. The objective equation is expressed as: st H≥0 In the formula, ||||2 represents the 2-norm; Tr represents the trace of the matrix; T represents the transpose of the matrix; st represents the constraint condition; and min represents taking the minimum value. The update formula for the cluster assignment membership matrix H is determined as follows: Expanding the objective equation yields the Lagrange function Lag: lag=||A-HH T ||+λtr(H T LH)+tr(αH T ) In the formula, A is the adjacency matrix, H is the clustering assignment membership matrix, L is the graph Laplacian matrix, λ represents the regularization parameter, and α represents the Lagrange multiplier; The update formula for constructing the membership matrix H is obtained using the Karush-Kuhn-Tucker conditions; The update formula for constructing the membership matrix H is expressed as: In the formula, to ensure non-negativity, the positive and negative elements of the Laplace matrix L are treated separately, L = L + -L - h ik Let be the value in the i-th row and k-th column of matrix H; (AH) ik Let AH be the value in the i-th row and k-th column of matrix AH; (L - H) ik For matrix L - The value of H in the i-th row and k-th column; (HH T H) ik For matrix (HH) T The value of the i-th row and k-th column of H); (L + H) ik For matrix (L) + The value of the i-th row and k-th column of H).

2. The image fair clustering method based on symmetric nonnegative matrix factorization according to claim 1, characterized in that, The image data is represented as X∈R m×n Each eigenvalue matrix X consists of n column vectors x i ∈R m Composition, R m Let m represent an m-dimensional space, where m represents the dimension of the samples and n represents the number of samples. The adjacency matrix A is constructed using the Gaussian kernel function, as shown in the following formula: Where δ is the thermonuclear parameter.

3. The image fair clustering method based on symmetric nonnegative matrix factorization according to claim 1, characterized in that, The number of clusters to be clustered is a positive integer not exceeding 10.

4. The image fair clustering method based on symmetric nonnegative matrix factorization according to claim 1, characterized in that, The maximum number of iterations is determined based on the size of the dataset, computational complexity, and limitations of computing resources.

5. The image fair clustering method based on symmetric nonnegative matrix factorization according to claim 1, characterized in that, The graphical Laplacian matrix L is constructed as follows: Construct a positive term matrix P, expressed as: In the formula, g i G represents the category to which sample i belongs. j Indicates the category to which sample j belongs; Construct a negative term matrix N, denoted as: Define the first matrix C, C = PN; Construct a graph Laplacian matrix L, L = DC, based on the first and second matrices, where the second matrix D is a diagonal matrix, represented as: In the formula, D ii Let C represent the value in the i-th row and i-th column of the second matrix D, where n represents the number of samples. ij This represents the value in the i-th row and j-th column of the first matrix C.

6. An image fair clustering device based on symmetric nonnegative matrix factorization, characterized in that, Based on the method of any one of claims 1 to 5, the apparatus comprises: The data acquisition module is configured to acquire image data and the number of data to be clustered; wherein, the image data is the position information and grayscale value information of each pixel after grayscale processing of image data; The matrix construction module is configured to define three initialization matrices: an adjacency matrix A, a clustering assignment membership matrix H, and a graph Laplacian matrix L, and to set the regularization parameter value λ. The iterative solution module is configured to determine the objective equation and the maximum number of iterations, and iteratively solve the clustering assignment membership matrix H and the Lagrange operator according to the update formula. The clustering output module is configured to output the final clustering assignment membership matrix H when the number of iterations is maximized or the iterations converge, thus obtaining the clustering result.

7. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, perform the method according to any one of claims 1 to 5.

Images