Image data discrete discriminant clustering method based on unsupervised regression analysis
By optimizing the discrete label matrix using unsupervised regression analysis, the problems of low efficiency and information loss in spectral clustering are solved, achieving high-precision image data clustering and improving clustering performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-04
- Publication Date
- 2026-03-13
AI Technical Summary
Existing spectral clustering methods are inefficient and susceptible to noise when processing large-scale nonlinear data. Furthermore, existing improved methods have failed to effectively address the original graph cut problem and data discriminative exploration, leading to a decrease in clustering accuracy.
An unsupervised regression analysis-based approach is adopted to transform image data into a data matrix. The unsupervised weighted least squares regression model is equivalently derived into a trace maximization problem. A reweighting method is designed to optimize the discrete label matrix, directly obtaining the clustering results and avoiding information loss caused by relaxation optimization.
It achieves high-precision discrete discriminative clustering, improves clustering accuracy and efficiency, preserves key data information, and is equivalent to the discriminative K-means method, demonstrating strong discriminative learning capabilities.
Smart Images
Figure CN121661369A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image recognition and classification and pattern recognition, and specifically relates to a discrete discriminant clustering method for image data based on unsupervised regression analysis. Background Technology
[0002] With the rapid development of communication and sensing technologies, the amount of data generated is growing exponentially. While this data contains a wealth of valuable information, it generally suffers from scarce known labels, massive sample sizes, and complex data structures. As an unsupervised learning method that does not rely on manual annotation, clustering techniques are often used as a key preprocessing step in data analysis. In recent years, they have been widely applied in tasks such as image retrieval, image segmentation, and object recognition, aiming to uncover the potential cluster structures hidden within pixels. In practical engineering applications, K-means clustering is often used for large-scale preprocessing before downstream tasks such as recognition, detection, and tracking due to its simplicity and high computational efficiency, enabling rapid initial grouping. However, this method is only suitable for linearly separable data. When the sample distribution is complex or nonlinear boundaries exist, the clustering results are prone to distortion, affecting the accuracy of subsequent tasks. Therefore, graph-based clustering methods have emerged in recent years to address these issues. Among them, spectral clustering, as a representative example, effectively characterizes complex and diverse data structures and significantly improves clustering results through a three-step learning paradigm of "graph construction, spectral decomposition, and discretization." Nevertheless, spectral clustering methods have the following three problems: 1) The fixed graph similarity matrix is easily affected by noise in the original data, thus failing to obtain the optimal clustering solution; 2) The time cost of Laplacian matrix eigenvalue decomposition is proportional to the cube of the number of samples, which greatly reduces the efficiency of large-scale data processing; 3) Solving the relaxation problem of the label matrix and the discretization post-processing can easily cause the loss of key information, thus affecting clustering.
[0003] Therefore, to address the aforementioned issues, many researchers have made technical improvements to spectral clustering to enhance its performance. For example, Yang Xiaojun et al. (“A Spectral Clustering Method and Related Device Based on Anchor Graphs”, Guangdong Province: CN202311623680.8, 2024-02-23) proposed an anchor graph-based spectral clustering method. This method constructs a small-scale anchor graph by sparsely downsampling the data to learn the anchor label matrix. Subsequently, a discrete label regularization term is introduced to propagate the anchor labels to the entire sample through a bipartite graph to obtain the discrete labels of the samples. This method not only reduces the graph learning time overhead by introducing sparse anchor graphs but also directly obtains discrete labels through joint model optimization, thereby comprehensively improving the clustering effect. However, the introduction of sparse anchors can lead to the loss of original data information to some extent; at the same time, this method adopts a strategy of approximating discrete sample labels from anchor labels. If the sparse anchors are not selected properly, the accuracy of label propagation will be affected, ultimately resulting in a decrease in clustering performance.
[0004] Existing spectral clustering variations typically employ strategies such as spectral rotation and label propagation to approximate the discrete labels of samples. However, these methods are all improvements based on the relaxation problem of spectral clustering and do not fundamentally solve the original graph cut problem. Furthermore, existing methods often neglect the exploration of data discriminativeness, leading to a decrease in clustering accuracy. Therefore, there is an urgent need to explore spectral clustering methods that can directly solve the original discrete optimization problem, in order to achieve more accurate clustering results while preserving structural information. Summary of the Invention
[0005] Existing spectral clustering variations typically employ strategies such as spectral rotation and label propagation to approximate the discrete labels of samples. However, these methods are all improvements based on the relaxation problem of spectral clustering and do not fundamentally solve the original graph cut problem of spectral clustering. Furthermore, existing methods often neglect the exploration of data discriminativity, leading to a decrease in clustering accuracy. Therefore, inspired by the mapping relationship between latent subspace data and labels in regression analysis, this invention proposes a discrete discriminative clustering method for image data based on unsupervised regression analysis. This method starts from an unsupervised weighted least squares regression model and is equivalently derived as a trace maximization problem with respect to the discrete sample label matrix, aiming to maximize intra-cluster similarity and thus achieve accurate clustering. A reweighting method is designed for the proposed trace maximization problem to directly optimize the discrete label matrix and obtain the clustering results. The clustering method proposed in this invention has the following advantages: 1) It cleverly transforms the regression problem into a clustering problem. The unsupervised regression mapping relationship it implies makes the proposed method equivalent to the discriminative K-means clustering method, clarifying its strong discriminative learning ability; 2) It focuses on optimizing the discrete label matrix to directly obtain the clustering results, avoiding the problem of losing important information caused by "relaxation optimization followed by discrete approximation". In summary, the method proposed in this invention is inspired by the unsupervised regression analysis model, solves the problem of maximizing intra-cluster similarity, and achieves high-precision discrete discriminative clustering.
[0006] Technical solution A discrete discriminant clustering method for image data based on unsupervised regression analysis, characterized by the following steps: Step 1: Put The original image was stretched and integrated into a data matrix. ,in Number of images and These represent the number of pixels horizontally and vertically in the image, respectively. The total number of pixels is used as the dimension of the data matrix; Step 2: Through For data matrix Centralized processing is implemented, in which As the central matrix, for 1D identity matrix for A column vector of all 1s; Step 3: with As a similarity matrix, a tradeoff parameter is given for the regularization term. For the centralized data matrix A dense similarity matrix is constructed. ; Step 4: Obtain the similarity matrix The sparse similarity matrix is obtained by finding the m largest elements in each row. ,right The discrete sample label matrix is obtained by using the nearest neighbor hierarchical initialization method. The initialization result; Step 5: with As the sub-gradient for optimization, a reweighted algorithm is used for clustering to obtain the final clustering result. The process is as follows: Step (1) Update subgradients
[0007] Step (2) Update row by row using a coordinate descent strategy. ,fixed Solution: In calculating the increment of the objective function, the label index corresponding to the largest increment is selected as... New label after coordinate update; in, and For matrix and The List; Step (3) repeats step (2) and updates sequentially. Other lines; Step (4) Repeat steps (1) through (3) until the problem is solved: When the objective function converges, the output is... The final clustering results are obtained.
[0008] The derivation of the similarity matrix P is as follows: First, we present the unsupervised regression model:
[0009] in, From From the original feature space to the latent The linear projection matrix of the dimensional subspace, where This represents the actual number of clusters. The deviation vector describes Image samples about The non-homogeneous bias of each cluster during linear projection, i.e., the global offset responsible for the projected subspace; Let be a discrete sample label matrix, where Discrete label constraints are implemented through constraints. Each row contains only one value of 1, and the rest are 0; This is the trade-off parameter for the regularization term; because , The trivial solution, therefore for the discrete label matrix Perform weighting and obtain a weighted label matrix. The unsupervised weighted regression model is obtained as follows:
[0010] in, The nonsingularity of a matrix can force removal The non-zero column is used to avoid the above trivial solution; Dependent variable and Unconstrained, obtained by taking partial derivatives and The is:
[0011] Will and Substituting the expression into the unsupervised weighted regression model, we get:
[0012] Among them For data matrix The constructed dense similarity matrix.
[0013] If and only if the first i ( The ) sample belongs to the ) k ( When there are ) clusters ,otherwise The first term aims to minimize the sample size. To discrete labels The regression residuals, the second term is about Regularization terms are used to avoid pathological problems caused by high-dimensional raw data.
[0014] The discrete sample label matrix The process of obtaining it is as follows: Due to the matrix It is dense, making it impossible to obtain the nearest neighbor relationships between samples. Therefore, taking... Find the m largest elements in each row of the matrix, and obtain the m nearest neighbors of each sample to get a sparse similarity matrix representing the nearest neighbor relationship between samples. ,right The nearest neighbor level initialization method is adopted, that is... As input to the nearest neighbor hierarchical initialization method, which aims to directly predict labels for all samples by parsing the sample nearest neighbor relationships contained in the nearest neighbor graph, n discrete label matrices of image data are obtained. The initialization result is given, where the nearest neighbor number m is empirically taken as 20.
[0015] The process of step 4 is as follows: Due to the matrix It is dense, making it impossible to obtain the nearest neighbor relationships between samples. Therefore, taking... Find the m largest elements in each row of the matrix, and obtain the m nearest neighbors of each sample to obtain a sparse similarity matrix representing the nearest neighbor relationship between samples. ,right The nearest neighbor level initialization method is adopted, that is... As input to the nearest neighbor hierarchical initialization method, which aims to directly predict labels for all samples by parsing the sample nearest neighbor relationships contained in the nearest neighbor graph, n discrete label matrices of image data can be obtained in the end. The initialization result is given, where the nearest neighbor number m is empirically taken as 20.
[0016] Subgradient The result is: First, a general model for the reweighting method is given:
[0017] in, For about convex function, For about Arbitrary constraints; The solution exist The subgradient problem is solved by Sure exist The subgradient; let , , and define constraints Based on the above definitions, the reweighted optimization method is used to first solve the following subproblems:
[0018] Wherein, sub-gradient The calculation method is as follows: .
[0019] The solution exist The subgradient problem, by fixing Solve the subproblems renew .
[0020] Update row by row using coordinate descent strategy : When subgradient When fixed, the present invention solves the following problem: The matrix is updated row by row using a coordinate descent strategy. That is, during the update The OK When, other rows remain fixed; when After the update is complete, continue updating while keeping the other lines fixed. And so on to update All rows; therefore, with the first OK Let's take an example to illustrate the update strategy for a specific row: Assumption The previous tags were The updated tags are According to the following rules, the first Problem of calculating the coordinates of candidate labels Increment of the objective function :
[0021] Among them, when Time indicates The labels remain unchanged, therefore the objective function increment is 0. This indicates a change in the label, and the increment of the objective function is not zero. Ultimately, in Select the label index corresponding to the largest increment from the increments as... The new label after the coordinates are updated.
[0022] An electronic device, characterized in that it includes a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the steps of the image data discrete discriminant clustering method based on unsupervised regression analysis.
[0023] A computer program product is characterized by including computer-executable instructions, which, when executed, are used to implement a discrete discriminant clustering method for image data based on unsupervised regression analysis.
[0024] This invention proposes a discrete discriminative clustering method for image data based on unsupervised regression analysis. Starting from an unsupervised weighted least squares regression model, this method is equivalently derived as a trace maximization problem with respect to the discrete sample label matrix, aiming to maximize intra-cluster similarity and thus achieve accurate clustering. A reweighting method is designed to directly optimize the discrete label matrix and obtain the clustering results for the proposed trace maximization problem. The clustering method proposed in this invention has the following advantages: 1) It cleverly transforms the regression problem into a clustering problem, and the implied unsupervised regression mapping relationship makes the proposed method equivalent to the discriminative K-means clustering method, clarifying its strong discriminative learning ability; 2) It focuses on optimizing the discrete label matrix to directly obtain the clustering results, avoiding the problem of losing important information caused by "relaxation optimization followed by discrete approximation". In summary, the method proposed in this invention, inspired by unsupervised regression analysis models, solves the problem of maximizing intra-cluster similarity and achieves high-precision discrete discriminative clustering.
[0025] Its beneficial effects specifically include: (1) The clustering model of the present invention is equivalently derived from the unsupervised weighted least squares regression model. It cleverly utilizes the “sample-label” mapping relationship of regression analysis to perform data clustering, thereby improving the clustering accuracy.
[0026] (2) The model proposed in this invention is equivalent to the K-means discrimination method, which ensures its cluster discrimination ability.
[0027] (3) This invention focuses on the optimization of discrete label matrix. By solving the problem of maximizing intra-cluster similarity, the optimal discrete sample label matrix and the final clustering result are obtained directly. Compared with the approximation learning paradigm of "relaxation first and then discretization", key data information is preserved and clustering performance is effectively improved. Attached Figure Description
[0028] Figure 1 Flowchart of an image data discrete discriminant clustering method based on unsupervised regression analysis Specific Implementation The present invention will now be further described in conjunction with the embodiments and accompanying drawings: The basic flowchart of image data processing in this invention is as follows: Figure 1 As shown, the specific steps are as follows: Step 1: Put Large-scale image data stretched into a data matrix ,in Number of images and These represent the number of pixels horizontally and vertically in the image, respectively. The total number of pixels is used as the dimension of the data matrix. This step will... The original images are merged into a new data matrix to facilitate subsequent operations.
[0029] Specifically, the 1799 images were stretched and integrated into a data matrix. The matrix dimension of 256 represents the total number of pixels in a single image; Step 2: Process the data matrix Regarding the centering (normalization) of feature dimensions, that is... ,in As the central matrix, for 1D identity matrix for A column vector of all 1s.
[0030] Step 3: Specify the trade-off parameters And calculate the similarity matrix. ; Find the similarity matrix The expression is as follows: First, the following unsupervised least squares regression model is given: (1) in, From Dimensional original space to potential Linear projection matrix of a dimensional subspace ( (The actual number of clusters) The deviation vector describes Non-homogeneous bias of image samples during linear projection Let be a discrete sample label matrix, where Discrete label constraints are implemented through constraints. Each row contains only one value of 1, and the rest are 0. In other words, if and only if the first row contains a value of 1, the second row contains a value of 0. ( The ) sample belongs to the ) ( When there are ) clusters ,otherwise The first term aims to minimize the sample size. To discrete labels The regression residuals, the second term is about The regularization term is designed to avoid pathological problems caused by the raw data. This is the trade-off parameter for the regularization term. However, this problem exists. , The trivial solution, therefore for the discrete label matrix Perform weighting and obtain a weighted label matrix. The following unsupervised weighted least squares regression model is obtained: (2) in, The nonsingularity of a matrix can force removal The non-zero column thus avoids the above trivial solution.
[0031] Note the variables and Without constraints, the partial derivative is used to directly obtain the result. and The expression is as follows: (3) Substituting solution (3) back into problem (2), we can further simplify and obtain the following problem: (4) in, . Notice Since the constant is constant, problem (4) can be equivalently transformed into the following problem: (5) in, For data matrix A dense similarity matrix is constructed. Therefore, through this derivation process, the similarity matrix is obtained. .
[0032] To clarify the discriminative nature of model (5), the following correlation derivation is performed: The discriminant K-means method aims to achieve discriminative clustering by minimizing between-class divergence while maximizing overall divergence. Its model can be formally expressed as: (6) in, and Let be the overall scatter matrix and the inter-class scatter matrix, respectively. Substituting this expression into problem (6) yields the following problem: (7) This shows that model (5) is equivalent to the K-means discrimination method, proving the cluster discriminative property of the proposed method.
[0033] Step 4: Before optimizing problem (5), this step aims to initialize the discrete label matrix using a nearest neighbor hierarchical initialization algorithm. Perform initialization. First, obtain the similarity matrix. The K maximum similarities in each row are used to reveal the sparse nearest neighbor structure between samples, resulting in a sparse similarity matrix. Where K is empirically set to 20; then, the similarity matrix is... As input to the nearest neighbor hierarchical initialization algorithm, it directly outputs a discrete label matrix. The initialization result.
[0034] Specifically: Take the similarity matrix The 20 largest elements in each row yield a sparse similarity matrix. and will The input is fed into the nearest neighbor hierarchical initialization algorithm to obtain the discrete sample label matrix. The initialization result; Step 5: This step aims to calculate the sub-gradient. First, we present the general model for the reweighting method: (8) in, For about convex function, For about With arbitrary constraints, this problem can be solved in the following two steps: 1) By Sure exist 1) Subgradient; 2) Fixed Solve the subproblems renew Therefore, let , , and define constraints Based on the above definition, the reweighted optimization method is used to optimize problem (5), and the following subproblems are solved first: (9) Wherein, sub-gradient .
[0035] Step 6: This step aims to update the discrete label matrix. In fixed In this case, problem (9) is transformed into vector form: (10) in, and For matrix and The List.
[0036] This invention employs a coordinate descent strategy to update row by row. ,against The OK ,remember The previous tags were , for the first The objective function (10) increments are calculated for each candidate label. . With the first OK For example, suppose The previous tags were The updated tags are According to the following rules, the first The increment of the objective function (10) is calculated based on the coordinates of each candidate label. : (11) Among them, when Time indicates The labels remain unchanged, therefore the objective function increment is 0. Select the label index corresponding to the largest increment from the increments as... The new label after the coordinates are updated.
[0037] Step 7: Update sequentially according to the rules in Step 6. Other lines; Step 8: Repeat steps 5-7 until the objective function of problem (5) converges, and output the result. The final clustering results are obtained.
[0038] Let the trade-off parameters The proposed method achieves clustering accuracy (ACC), normalized mutual information (NMI), and adjusted Rand index (ARI) of 82.77%, 89.83%, and 80.08% respectively on the MSRA25 image dataset. ACC and NMI range from 0 to 1, while ARI ranges from -1 to 1; higher values indicate better clustering performance. Based on these three metrics, the K-means method achieves only 48.31%, 56.53%, and 34.55%, while spectral clustering achieves 53.91%, 65.59%, and 40.88%. The performance of mainstream spectral clustering variants on the MSRA25 dataset over the past three years has mostly fluctuated within the ranges of 57%–62%, 68%–75%, and 44%–55%, representing a slight improvement in clustering performance compared to traditional spectral clustering. However, the method proposed in this invention achieves a significant improvement in clustering results, verifying the superiority of discrete discriminative clustering.
[0039] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the scope of the technology disclosed in the present invention, and such modifications or substitutions should all be covered within the scope of protection of the present invention.
Claims
1. A discrete discriminant clustering method for image data based on unsupervised regression analysis, characterized in that... The steps are as follows: Step 1: Put The original image was stretched and integrated into a data matrix. ,in Number of images and These represent the number of pixels horizontally and vertically in the image, respectively. The total number of pixels is used as the dimension of the data matrix; Step 2: Through For data matrix Centralized processing is implemented, in which As the central matrix, for 1D identity matrix for A column vector of all 1s; Step 3: with As a similarity matrix, a tradeoff parameter is given for the regularization term. For the centralized data matrix A dense similarity matrix is constructed. ; Step 4: Obtain the similarity matrix The sparse similarity matrix is obtained by finding the m largest elements in each row. ,right The discrete sample label matrix is obtained by using the nearest neighbor hierarchical initialization method. The initialization result; Step 5: with As the sub-gradient for optimization, a reweighted algorithm is used for clustering to obtain the final clustering result. The process is as follows: Step (1) Update subgradients Step (2) Update row by row using a coordinate descent strategy. ,fixed Solution: In calculating the increment of the objective function, the label index corresponding to the largest increment is selected as... New label after coordinate update; in, and For matrix and The List; Step (3) repeats step (2) and updates sequentially. Other lines; Step (4) Repeat steps (1) through (3) until the problem is solved: When the objective function converges, the output is... The final clustering results are obtained.
2. The image data discrete discriminant clustering method based on unsupervised regression analysis according to claim 1, characterized in that: The derivation of the similarity matrix P is as follows: First, we present the unsupervised regression model: in, From From the original feature space to the latent The linear projection matrix of the dimensional subspace, where This represents the actual number of clusters. The deviation vector describes Image samples about The non-homogeneous bias of each cluster during linear projection, i.e., the global offset responsible for the projected subspace; Let be a discrete sample label matrix, where Discrete label constraints are implemented through constraints. Each row contains only one value of 1, and the rest are 0; This is the tradeoff parameter for the regularization term; because , The trivial solution, therefore for the discrete label matrix Perform weighting and obtain a weighted label matrix. The unsupervised weighted regression model is obtained as follows: in, The nonsingularity of a matrix can force removal The non-zero column is used to avoid the above trivial solution; Dependent variable and Unconstrained, obtained by taking partial derivatives and The is: Will and Substituting the expression into the unsupervised weighted regression model, we get: Among them For data matrix The constructed dense similarity matrix.
3. The image data discrete discriminant clustering method based on unsupervised regression analysis according to claim 2, characterized in that: If and only if the first i , The sample belongs to the first k , When there are clusters ,otherwise The first term aims to minimize the sample size. To discrete labels The regression residuals, the second term is about Regularization terms are used to avoid pathological problems caused by high-dimensional raw data.
4. The image data discrete discriminant clustering method based on unsupervised regression analysis according to claim 1, characterized in that: The discrete sample label matrix The process of obtaining it is as follows: Due to the matrix It is dense, making it impossible to obtain the nearest neighbor relationships between samples. Therefore, taking... Find the m largest elements in each row of the matrix, and obtain the m nearest neighbors of each sample to get a sparse similarity matrix representing the nearest neighbor relationship between samples. ,right The nearest neighbor level initialization method is adopted, that is... As input to the nearest neighbor hierarchical initialization method, which aims to directly predict labels for all samples by parsing the sample nearest neighbor relationships contained in the nearest neighbor graph, n discrete label matrices of image data are obtained. The initialization result is given, where the nearest neighbor number m is empirically taken as 20.
5. The image data discrete discriminant clustering method based on unsupervised regression analysis according to claim 1, characterized in that: The process of step 4 is as follows: Due to the matrix It is dense, making it impossible to obtain the nearest neighbor relationships between samples. Therefore, taking... Find the m largest elements in each row of the matrix, and obtain the m nearest neighbors of each sample to obtain a sparse similarity matrix representing the nearest neighbor relationship between samples. ,right The nearest neighbor level initialization method is adopted, that is... As input to the nearest neighbor hierarchical initialization method, which aims to directly predict labels for all samples by parsing the sample nearest neighbor relationships contained in the nearest neighbor graph, n discrete label matrices of image data are obtained. The initialization result is given, where the nearest neighbor number m is empirically taken as 20.
6. The image data discrete discriminant clustering method based on unsupervised regression analysis according to claim 1, characterized in that: Subgradient The result is: First, a general model for the reweighting method is given: solve exist The subgradient problem is solved by Sure exist The subgradient; let , , and define constraints Based on the above definitions, the reweighted optimization method is used to first solve the following subproblems: Wherein, sub-gradient The calculation method is as follows: 。 7. The image data discrete discriminant clustering method based on unsupervised regression analysis according to claim 6, characterized in that: The solution exist The subgradient problem, by fixing Solve the subproblems renew .
8. The image data discrete discriminant clustering method based on unsupervised regression analysis according to claim 1, characterized in that: Update row by row using coordinate descent strategy : When subgradient When fixed, the present invention solves the following problem: The matrix is updated row by row using a coordinate descent strategy. That is, during the update The OK When, other rows remain fixed; when After the update is complete, continue updating while keeping the other lines fixed. And so on to update All rows; therefore, with the first OK Let's take an example to illustrate the update strategy for a specific row: Assumption The previous tags were The updated tags are According to the following rules, the first Problem of calculating the coordinates of candidate labels Increment of the objective function : Among them, when Time indicates The labels remain unchanged, therefore the objective function increment is 0. When the label changes, the increment of the objective function is not zero; ultimately, in Select the label index corresponding to the largest increment from the increments as... The new label after the coordinates are updated.
9. An electronic device, characterized in that, It includes a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the steps of the image data discrete discriminative clustering method based on unsupervised regression analysis as described in any one of claims 1 to 8.
10. A computer program product, characterized in that... It includes computer-executable instructions, which, when executed, are used to implement the image data discrete discriminant clustering method based on unsupervised regression analysis as described in any one of claims 1 to 8.