A Deep Multi-View Clustering Method Based on Multi-Granularity Contrast Learning

Abstract

This invention provides a deep multi-view clustering method based on multi-granularity contrastive learning, belonging to the field of computer vision technology. It solves the technical problem that single-granularity feature representations are insufficient to comprehensively capture the multi-level semantic information of data and the quality differences between different views. The method includes the following steps: S10, designing an independent encoder for each view to extract latent embedding features; S20, mapping the attention-enhanced features of each view to the same semantic space; S30, dynamically assigning weights based on the similarity between the features of each view and the global representation; S40, granular-level contrastive learning can enhance the compactness of the cluster structure within the granular sphere and align the granular sphere structure between cross-view granular spheres; S50, performing K-means clustering on the final global features to obtain the final prediction result. This invention enhances the weight of key features by adding a feature attention enhancement module to the view features.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a deep multi-view clustering method based on multi-granularity contrastive learning. Background Technology

[0002] With the rapid development of multimedia applications, large amounts of data are collected from various sources or described with different attributes. Often, this data lacks labeling information. For example, in the case of video recording, images may be captured from different cameras, accompanied by different audio and text descriptions; in the case of image recording, images may be obtained from different angles.

[0003] To explore useful consistency and complementarity information among multiple views in an unsupervised manner, multi-view clustering aims to integrate data from different sources to gain a more comprehensive understanding of underlying phenomena. In the field of multi-view clustering, deep multi-view clustering methods have proven to achieve superior performance due to the powerful representation learning capabilities of deep neural networks. These methods employ view-specific encoders to learn salient features for each view. These learned views are then further fused to obtain more discriminative global features that can be divided into different categories based on the complementarity information of all views. While significant progress has been made in deep multi-view clustering in recent years, [Unsupervised Deep Embedding for Clustering Analysis (DEC)] utilizes autoencoders to extract data features and maps these features to a latent space for clustering. Its paradigm of network loss plus clustering loss inspired subsequent deep multi-view clustering work, [Self-Weighted Contrastive Fusion for Deep Multi-View Clustering], which establishes a hierarchical feature fusion framework that effectively separates the consistency objective from the reconstruction objective. It then achieves multi-view contrastive fusion by maximizing the consistency between view consensus representations and global representations, fully exploring the consistency and complementarity of views. However, some problems still exist: (1) Traditional contrastive learning mainly focuses on sample-level feature representation, ignoring the local structural information of the data. Single-granularity feature representation is difficult to fully capture the multi-level semantic information of the data. (2) Traditional methods use hard boundary partitioning, which cannot effectively handle the fuzziness and uncertainty of data distribution. (3) How to effectively integrate the information of different views is also a research focus. (4) Many previous studies have ignored the quality differences between different views. For example, low-quality views will weaken the clustering effect of high-quality views. Summary of the Invention

[0004] The purpose of this invention is to provide a deep multi-view clustering method based on multi-granularity contrastive learning. By adding a feature attention enhancement module to the view features, the weight of key features is enhanced, making fuller use of data information. At the same time, the weight of different views is assigned based on the attention mechanism, which can enhance the feature fusion effect and make the beneficial views account for a larger proportion in the clustering. Finally, sample-level and granular-level multi-granularity contrastive learning is performed to make fuller use of features at different levels and improve the clustering performance of multi-view images.

[0005] To achieve the above-mentioned objectives, the present invention employs the following technical solution: a deep multi-view clustering method based on multi-granularity contrastive learning, comprising the following steps: S10. Design an independent encoder for each view to extract latent embedding features, and design a feature attention enhancement module for these latent feature representations to enhance the weights of features that contribute to clustering. S20. Map the attention-enhanced features of each view to the same semantic space to learn cross-view features. Figure 1 It achieves consistent expression, enhances the ability to share knowledge among views, and integrates the enhanced features of all views to construct a global feature representation, capturing the overall structural features of the data; S30. In sample-level contrastive learning within multi-granularity contrastive learning, the global representation of the same sample is brought closer to that of each view. Figure 1 The distance between similar representations is increased, thus widening the distance between different sample representations and improving the discriminative power of feature representations. An adaptive view adjustment mechanism based on attention is also designed, dynamically allocating weights according to the similarity between each view feature and the global representation. S40. The concept of fuzzy logic is introduced in sphere-level contrastive learning, allowing samples to be assigned to multiple spheres with different membership degrees. Sphere-level contrastive learning can enhance the compactness of the cluster structure within spheres and align sphere structures across views. S50. Perform K-means clustering on the final global features to obtain the final prediction result.

[0006] As a deep multi-view clustering method based on multi-granularity contrastive learning provided by the present invention, step S10 includes the following steps: S11. Because autoencoders can learn low-dimensional representations of data, remove redundant information, and extract key features, they play a crucial role in clustering tasks. First, for each view... v Using an encoder Original features Mapped to a low-dimensional latent feature space: (1) In the formula, Represents the first in a multi-view datasetv The encoder for each view consists of multiple fully connected layers and ReLU activation functions. The neural network parameters of the encoder, Indicates the first v The latent feature representation of a view It is the dimension of the latent feature space; S12. To better highlight the key features of each view, a feature attention enhancement module was designed for each view. Similar to the channel attention mechanism, it can calculate the importance of each feature and enhance key features through weighted summation. First, global average pooling and global max pooling are calculated. Average pooling can capture the overall features while reducing noise interference, making the output smoother. Max pooling can extract the most critical features to highlight the key points and enhance the model's ability to recognize key patterns. The calculation formula is as follows: , (2) (3) In the formula The attention weights representing average pooling are obtained by passing through a small multilayer perceptron (MLP) and a sigmoid function. This represents the attention weights in max pooling. This represents the attention weights after combining average pooling and max pooling. Represents a Sigmoid function; S13. Multiply the latent features by the attention weights to obtain the weighted enhanced features. The specific calculation formula is shown in equation (4): (4) In the formula This indicates the enhanced features. This represents element-wise multiplication; S14. To preserve the information of the original features, a decoder is used to reconstruct the features. Each view has an independent decoder, attempting to extract features from the attention-enhanced features. Reconstruct the original input : (5) In the formula, These are the neural network parameters of the decoder. Indicates the first v Decoder for each view Represented as the first v The reconstruction features of each view, the structure of the decoder is symmetrical with the structure of the encoder; S15. In summary, measure the original input for each view. and reconstruction results The reconstruction loss can be expressed as mean squared error (MSE), calculated as follows: (6) In the formula, This represents the Frobenius norm.

[0007] As a deep multi-view clustering method based on multi-granularity contrastive learning provided by the present invention, step S20 includes the following steps: S21. To learn a consistent representation among views, attention-enhanced features are applied to each view. Mapped to a view via a shared mapping module CIM (Common Information Module). Figure 1 Consistent feature space, to obtain visual Figure 1 Characteristics of homogeneity: (7) In the formula, CIM is a simple linear layer, and normalize represents the L2 normalization operation, which ensures that the length of the feature vector is 1, thus obtaining consistent features. , For vision Figure 1 Dimensions of consistency characteristics; S22. In order to integrate information from multiple views, attention enhancement features are applied to all views. Connect them together, and obtain global features through a Feature Fusion Module (FFM). : (8) The formula FFM contains multiple linear layers and a ReLU activation function. This indicates the feature connection operation, and normalize indicates L2 normalization. The dimension of the global features.

[0008] As a deep multi-view clustering method based on multi-granularity contrastive learning provided by the present invention, step S30 includes the following steps: S31. The purpose of sample-level contrastive learning is to narrow the distance between the same sample in different views and to keep negative samples away. At the same time, this method takes into account that the quality of different views is different, which will affect the final clustering result. Therefore, this paper proposes an adaptive view weight allocation mechanism to give higher weight to high-quality views so that high-quality views can better dominate the clustering effect. This paper assigns weights to different views based on the attention mechanism. First, the query value is calculated. The specific calculation formula is shown in Equation (9): (9) In the formula, The fused feature matrix, N That is the number of samples. It is the global feature mean of all samples; S32. Then calculate the key value, as shown in formula (10): (10) in, It is the first v The feature mean of each view It is the first v Consistency feature matrix of each view; S33. Next, calculate the attention score for each view. The specific calculation formula is shown in equation (11): (11) in, It is a scalar, representing the first... v Individual View Figure 1 Attention score between consistent features and global features; S34. Finally, calculate the attention weight for each view. The specific calculation formula is shown in Equation (12): (12) in It is the Sigmoid activation function, used to normalize the attention score. It is the first v Attention weights for each view; S35. Calculate the sample-level contrast loss, the specific formula is shown in equation (13): (13) In the formula, yes InfoNCE Comparative losses: (14) In the formula, It is a temperature parameter learned through sample-level comparative learning. It is a conditional function. It is the similarity between two features.

[0009] As a deep multi-view clustering method based on multi-granularity contrastive learning provided by the present invention, step S40 includes the following steps: S41. Particle-level contrastive learning aims to compensate for the deficiency of sample-level contrastive learning in neglecting the local semantic structure between samples. First, K-means++ is used to randomly initialize the centers. S42 calculates the Euclidean distance from each sample to each center, which can be used to calculate fuzzy membership. The specific calculation formula is as follows: (15) In the formula, It is the first v Within the first view i Consistency feature vector of each sample It is the first v Within the first view j The center of each grain, It is the first v Samples within a view i To the j The distance between the centers; S43. Introduce Gaussian blur so that a sample can belong to multiple spheres simultaneously. The calculation formula is as follows: (16) It is the first j The radius of each sphere, It is the first v The first view i The nth sample pair j The membership degree of an individual sphere increases as it gets closer to the center. S44, For each ball k Select samples with a membership degree higher than a threshold as the sample set within the sphere: (17) In the formula, This is the membership threshold; S45. Calculate the contrast loss inside each particle. k The weighted contrast loss of its internal samples is calculated, with the goal of bringing the sample representations belonging to the same fuzzy particle closer together. The specific calculation formula is shown in Equation (18): (18) In the formula, and These are sample characteristics within the same sphere. It is a temperature parameter for particle-level contrastive learning, which controls the smoothness of the similarity distribution; The total intrasphere loss is expressed as: (19) S46. In order to enhance the consistency between different views, cross-view particle-level contrastive learning is introduced in particle-level contrastive learning. First, the particle center similarity matrix is ​​calculated to determine whether similar particles have consistent representations under different views. The calculation formula is shown in (20): (20) For the first i The first view under the first view p The center of each grain, For the first j The first view under the first view q The center of each grain, The similarity between the centers of the particles in the two views; S47. Next, calculate the membership overlap matrix. The larger the overlap, the more the pair of spheres captures the same sample structure in different views. The calculation formula is as follows: (twenty one) in, N Indicates the number of samples. Indicates the first n A sample pair view i granules p membership degree Indicates granules p , q The extent to which samples are shared across different views; Then normalize to a probability distribution: (twenty two) S48. Only positive samples are trained on samples with high overlap to obtain a weighted sum of the similarity of positive samples. Finally, the loss between cross-view particles is calculated. The specific calculation formula is shown in Equation (23): (twenty three) In the formula, The threshold is used to determine whether the membership degree of the overlap is greater than the threshold. If it is greater than the threshold, the value is 1; otherwise, it is 0. The final loss is the average between view pairs, as shown in equation (24): (twenty four) Meanwhile, the present invention proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed, it implements the steps of the method described in the present invention.

[0010] Furthermore, the present invention proposes a computer-readable storage medium having a computer program stored thereon, the computer program being configured to implement the steps of the method described in the present invention when invoked by a processor.

[0011] Finally, the present invention proposes a computer program product, including a computer program / instructions, characterized in that the computer program / instructions, when executed by a processor, implement the steps of the method described in the present invention.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) A multi-granularity contrastive learning strategy is proposed, which effectively integrates multi-level semantic information at the sample level and the sphere level, significantly improving clustering performance. This invention breaks through the limitation of most contrastive learning methods that only perform feature alignment at a single granularity, and introduces a multi-granularity contrastive learning mechanism that combines the sample level and the sphere level. At the sample level, the consistency of cross-view features is enhanced by comparing and constraining the potential representations of the same semantic sample under different views; at the sphere level, samples with similar semantic structures are aggregated into spheres, and contrastive learning is performed between spheres, thereby introducing a higher level of semantic constraints. This multi-granularity collaborative contrastive strategy can simultaneously take into account the local sample structure and the global semantic distribution, effectively narrowing the distance between positive samples and spheres with the same semantic meaning, and widening the distance between different semantic categories, thereby significantly enhancing the discriminativeness and stability of the clustering structure and improving the overall clustering accuracy and consistency.

[0013] (2) The concept of fuzzy membership is introduced to optimize the sample clustering boundary and improve the robustness of clustering. Existing clustering methods usually adopt a hard partitioning strategy, which makes it difficult to handle noisy samples between classes. This invention introduces the concept of fuzzy membership. By assigning different membership values ​​to each sample, samples can participate in multiple clusters with different weights at the same time, effectively characterizing the uncertainty of samples near the class boundary.

[0014] (3) An attention-based adaptive adjustment mechanism for view importance was designed to effectively address the problem of uneven view quality. Addressing the performance limitations of existing multi-view clustering methods that generally assume consistent view quality and equal contribution, this invention proposes an attention-based adaptive adjustment strategy for view importance. This strategy dynamically learns the weight allocation of each view during model training based on the discriminative ability and reliability of the information contained in different views. This automatically enhances the contribution of high-quality views to the clustering results, suppresses interference from noisy or redundant views, and effectively improves the rationality of multi-view information fusion and the accuracy of the clustering results.

[0015] (4) A feature attention enhancement module was designed for potential embedded features to strengthen the weight of effective features. By combining the weights of average pooling and global pooling, the weights of key features in each view are enhanced, enabling the model to focus more on useful features and further improve the overall clustering performance. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0017] Figure 1 This is a flowchart illustrating the overall data processing of deep multi-view clustering based on multi-granularity contrastive learning according to the present invention.

[0018] Figure 2 This is an overall flowchart of a deep multi-view clustering method based on multi-granularity contrastive learning according to the present invention.

[0019] Figure 3 This is a convergence analysis image of the MINST-USPS and Fashion datasets in this invention.

[0020] Figure 4 The images shown are tsne visualizations of the Caltech-5V dataset in this invention, before, during, and after the first and second rounds of comparative learning.

[0021] Figure 5 The images shown are tsne visualizations of the Fashion dataset in this invention, before, during, and after the first and final rounds of comparative learning. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1: As Figure 1 and Figure 2 As shown, this embodiment provides a deep multi-view clustering method based on multi-granularity contrastive learning, including the following steps: S1. Design an independent encoder for each view to extract latent embedding features, and design a feature attention enhancement module for these latent feature representations to enhance the weights of features that contribute to clustering. S2. Map the attention-enhanced features of each view to the same semantic space to learn cross-view features. Figure 1 It achieves consistent expression, enhances the ability to share knowledge among views, and integrates the enhanced features of all views to construct a global feature representation, capturing the overall structural features of the data; S3. In sample-level contrastive learning within multi-granularity contrastive learning, the global representation of the same sample is brought closer to that of each view. Figure 1 The distance between similar representations is increased, thus widening the distance between different sample representations and improving the discriminative power of feature representations. An adaptive view adjustment mechanism based on attention is also designed, dynamically allocating weights according to the similarity between each view feature and the global representation. S4. The concept of fuzzy logic is introduced in sphere-level contrastive learning, allowing samples to be assigned to multiple spheres with different membership degrees. Sphere-level contrastive learning can enhance the compactness of the cluster structure within spheres and align sphere structures across views. S5. Perform K-means clustering on the final global features to obtain the final prediction result.

[0024] Step S1 includes the following steps: S11. Because autoencoders can learn low-dimensional representations of data, remove redundant information, and extract key features, they play a crucial role in clustering tasks. First, for each view... v Using an encoder Original features Mapped to a low-dimensional latent feature space: (1) In the formula, Represents the first in a multi-view dataset v The encoder for each view consists of multiple fully connected layers and ReLU activation functions. The neural network parameters of the encoder, Indicates the first v The latent feature representation of a view It is the dimension of the latent feature space; S12. To better highlight the key features of each view, a feature attention enhancement module was designed for each view. Similar to the channel attention mechanism, it can calculate the importance of each feature and enhance key features through weighted summation. First, global average pooling and global max pooling are calculated. Average pooling can capture the overall features while reducing noise interference, making the output smoother. Max pooling can extract the most critical features to highlight the key points and enhance the model's ability to recognize key patterns. The calculation formula is as follows: , (2) (3) In the formula The attention weights representing average pooling are obtained by passing through a small multilayer perceptron (MLP) and a sigmoid function. This represents the attention weights in max pooling. This represents the attention weights after combining average pooling and max pooling. Represents a Sigmoid function; S13. Multiply the latent features by the attention weights to obtain the weighted enhanced features. The specific calculation formula is shown in equation (4): (4) In the formula This indicates the enhanced features. This represents element-wise multiplication; S14. To preserve the information of the original features, a decoder is used to reconstruct the features. Each view has an independent decoder, attempting to extract features from the attention-enhanced features. Reconstruct the original input : (5) In the formula, These are the neural network parameters of the decoder. Indicates the first v Decoder for each view Represented as the first v The reconstruction features of each view, the structure of the decoder is symmetrical with the structure of the encoder; S15, Measure the raw input for each view. and reconstruction results The reconstruction loss can be expressed as mean squared error (MSE), calculated as follows: (6) In the formula, This represents the Frobenius norm.

[0025] Step S includes the following steps: S21. To learn a consistent representation among views, attention-enhanced features are applied to each view. Mapped to a view via a shared mapping module CIM (Common Information Module). Figure 1 Consistent feature space, to obtain visual Figure 1 Characteristics of homogeneity: (7) In the formula, CIM is a simple linear layer, and normalize represents the L2 normalization operation, which ensures that the length of the feature vector is 1, thus obtaining consistent features. , For vision Figure 1 Dimensions of consistency characteristics; S22. In order to integrate information from multiple views, attention enhancement features are applied to all views. Connect them together, and obtain global features through a Feature Fusion Module (FFM). : (8) In the formula, the feature fusion module FFM contains multiple linear layers and the ReLU activation function. This indicates the feature connection operation, and normalize indicates L2 normalization. The dimension of the global features.

[0026] As a deep multi-view clustering method based on multi-granularity contrastive learning provided by the present invention, step S3 includes the following steps: Step S3 includes the following steps: S31. The purpose of sample-level contrastive learning is to narrow the distance between the same sample in different views and to keep negative samples away. At the same time, this method takes into account that the quality of different views is different, which will affect the final clustering result. Therefore, this paper proposes an adaptive view weight allocation mechanism to give higher weight to high-quality views so that high-quality views can better dominate the clustering effect. This paper assigns weights to different views based on the attention mechanism. First, the query value is calculated. The specific calculation formula is shown in Equation (9): (9) In the formula, The fused feature matrix, N It is the number of samples. It is the global feature mean of all samples; S32. Then calculate the key value, as shown in formula (10): (10) in, It is the first v The feature mean of each view It is the first v Consistency feature matrix of each view; S33. Next, calculate the attention score for each view. The specific calculation formula is shown in equation (11): (11) It is a scalar, representing the first... v Individual View Figure 1 Attention score between consistent features and global features; S34. Finally, calculate the attention weight for each view. The specific calculation formula is shown in Equation (12): (12) in It is the Sigmoid activation function, used to normalize the attention score. It is the first v Attention weights for each view; S35. Calculate the sample-level contrast loss, the specific formula is shown in equation (13): (13) In the formula, yes InfoNCE Comparative losses: (14) In the formula, It is a temperature parameter learned through sample-level comparative learning. It is a conditional function. It is the similarity between two features.

[0027] Step S4 includes the following steps: S41. Particle-level contrastive learning aims to compensate for the deficiency of sample-level contrastive learning in neglecting the local semantic structure between samples. First, K-means++ is used to randomly initialize the centers. S42 calculates the Euclidean distance from each sample to each center, which can be used to calculate fuzzy membership. The specific calculation formula is as follows: (15) In the formula, It is the first v Within the first view i Consistency feature vector of each sample It is the first v Within the first view j The center of each grain, It is the first v Samples within a view i To the j The distance between the centers; S43. Introduce Gaussian blur so that a sample can belong to multiple spheres simultaneously. The calculation formula is as follows: (16) It is the first j The radius of each sphere, It is the first v The first view i The nth sample pair j The membership degree of an individual sphere increases as it gets closer to the center. S44, For each ball k Select samples with a membership degree higher than a threshold as the sample set within the sphere: (17) In the formula, This is the membership threshold; S45. Calculate the contrast loss inside each particle. k The weighted contrast loss of its internal samples is calculated, with the goal of bringing the sample representations belonging to the same fuzzy particle closer together. The specific calculation formula is shown in Equation (18): (18) In the formula, and These are sample characteristics within the same sphere. It is a temperature parameter for particle-level contrastive learning, which controls the smoothness of the similarity distribution; The total intrasphere loss is expressed as: (19) S46. In order to enhance the consistency between different views, cross-view particle-level contrastive learning is introduced in particle-level contrastive learning. First, the particle center similarity matrix is ​​calculated to determine whether similar particles have consistent representations under different views. The calculation formula is shown in (20): (20) For the first i The first view under the first view p The center of each grain, For the first j The first view under the first view q The center of each grain, The similarity between the centers of the particles in the two views; S47. Next, calculate the membership overlap matrix. The larger the overlap, the more the pair of spheres captures the same sample structure in different views. The calculation formula is as follows: (twenty one) in N Indicates the number of samples. Indicates the first n A sample pair view i granules p membership degree Indicates granules p , q The extent to which samples are shared across different views; Then normalize to a probability distribution: (twenty two) S48. Only positive samples are trained on samples with high overlap to obtain a weighted sum of the similarity of positive samples. Finally, the loss between cross-view particles is calculated. The specific calculation formula is shown in Equation (23): (twenty three) In the formula, The threshold is used to determine whether the membership degree of the overlap is greater than the threshold. If it is greater than the threshold, the value is 1; otherwise, it is 0. The final loss is the average between view pairs, as shown in equation (24): (twenty four).

[0028] Example 2: To address the above problems, this example employs the deep multi-view clustering method based on multi-granularity contrastive learning from Example 1. Experimental results on multiple multi-view datasets demonstrate that, compared with existing methods, this invention exhibits significant effectiveness and superiority. Experimental results for some datasets are shown in Tables 1 and 2: Table 1 shows the results on four public datasets. Bold text indicates the best results, and underlined text indicates the second-best results.

[0029] Table 2 shows the results on the Caltech dataset with different numbers of views. Bold text indicates the best results, and underlined text indicates the second-best results.

[0030] Example 3: As Figure 2 As shown, to demonstrate the effectiveness of the deep multi-view clustering method based on multi-granularity contrastive learning, the training processes of the MINST-USPS and Fashion datasets are presented. Their loss values ​​exhibit a monotonically decreasing trend during training and tend to converge as the number of epochs increases. Meanwhile, key metrics such as accuracy (ACC) and normalized mutual information (NMI) show a gradually increasing trend during training, reaching a peak and then fluctuating slightly within a narrow range.

[0031] Example 4: Figure 4 and Figure 5 As shown, to intuitively study the proposed multi-granularity contrastive learning, the contrastive training process is visualized using T-SNE, with examples from the Caltech-5V and Fashion datasets. This visualization shows that when the autoencoder pre-training is complete but multi-granularity contrastive learning training has not yet begun, the discriminative power of the features is very low, the clustering structure is still scattered, and the clusters cannot reflect the true clustering structure. In the subsequent multi-granularity contrastive learning process, the clustering structure with embedded features becomes increasingly clear until finally, each sample can be effectively assigned to its corresponding cluster, thus confirming the effectiveness of this method.

[0032] Example 5: This example proposes an electronic system, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method steps of the present invention.

[0033] Example 6: This example proposes a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the method described in this invention, which will not be repeated here.

[0034] Example 7: This example proposes a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, they implement the steps of the method described in this invention, which will not be repeated here.

[0035] It should be noted that the processing flow of embodiments 5-7 corresponds to the specific steps of the method provided in embodiment 1 of the present invention, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the method provided in embodiment 1 of the present invention.

[0036] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0037] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A deep multi-view clustering method based on multi-granularity contrastive learning, characterized in that, Includes the following steps: S1. Design an independent encoder for each view to extract latent embedding features, and design a feature attention enhancement module for these latent embedding feature representations to enhance the weights of features that contribute to clustering. S2. Map the attention-enhanced features of each view to the same semantic space, learn cross-view consistent representations, improve the knowledge sharing ability between views, and integrate the enhanced features of all views to construct a global feature representation and capture the overall structural features of the data. S3. In sample-level contrastive learning in multi-granularity contrastive learning, the discriminativeness of feature representation is improved by narrowing the distance between the global representation of the same sample and the consistent representation of each view. An adaptive view adjustment mechanism based on attention mechanism is designed to dynamically allocate weights according to the similarity between the features of each view and the global representation. S4. The concept of fuzzy logic is introduced into particle-level contrastive learning, which allows samples to be assigned to multiple particles with different membership degrees. Particle-level contrastive learning enhances the compactness of the cluster structure within the particle and aligns the particle structure between particles across views. S5. Perform K-means clustering on the final global features to obtain the final prediction result.

2. The deep multi-view clustering method based on multi-granularity contrastive learning according to claim 1, characterized in that, S1 includes the following steps: S11, First, for each view v Using an encoder Original features Mapped to a low-dimensional latent feature space: （1）； In the formula, Represents the first in a multi-view dataset v The encoder for each view consists of multiple fully connected layers and ReLU activation functions. The neural network parameters of the encoder, Indicates the first v The latent feature representation of a view It is the dimension of the latent feature space; S12. A feature attention enhancement module was designed for each view. Similar to the channel attention mechanism, it calculates the importance of each feature and enhances key features through weighted summation. First, global average pooling and global max pooling are calculated. Average pooling obtains the overall features while reducing noise interference, making the output smooth. Max pooling extracts key features to highlight the key points and enhances the model's ability to recognize key patterns. The calculation formula is as follows: ； In the formula The attention weights representing average pooling are obtained by passing through a small multilayer perceptron (MLP) and a sigmoid function. This represents the attention weights in max pooling. This represents the attention weights after combining average pooling and max pooling. Represents a Sigmoid function. and These represent average pooling and max pooling, respectively. S13. Multiply the latent features by the attention weights to obtain the weighted enhanced features. The calculation formula is shown in equation (4): （4）； In the formula This indicates the enhanced features. This represents element-wise multiplication; S14. Reconstruct features using a decoder. Each view has its own independent decoder. Attempt to reconstruct features from attention-enhanced features. Reconstruct the original input : （5）； In the formula, These are the neural network parameters of the decoder. Indicates the first v Decoder for each view Represented as the first v The reconstruction features of each view, the structure of the decoder is symmetrical with the structure of the encoder; S15, Measure the raw input for each view. and reconstruction results The reconstruction loss between the two is calculated using the mean squared error (MSE), as shown in the following formula: （6）； In the formula, This represents the Frobenius norm.

3. The deep multi-view clustering method based on multi-granularity contrastive learning according to claim 2, characterized in that, S2 includes the following steps: S21. Enhance the attention feature of each view. View consistency features are obtained by mapping a shared mapping module CIM to a view consistency feature space: （7）； In the formula, CIM is a simple linear layer, and normalize represents the L2 normalization operation, which ensures that the length of the feature vector is 1, thus obtaining consistent features. , Dimensions for view consistency features; S22, Enhance attention feature for all views Connect them together, and obtain global features through a feature fusion module (FFM). : （8）； In the formula, the feature fusion module FFM contains multiple linear layers and the ReLU activation function. This indicates the feature connection operation, and normalize indicates L2 normalization. The dimension of the global features.

4. The deep multi-view clustering method based on multi-granularity contrastive learning according to claim 3, characterized in that, S3 includes the following steps: S31. The purpose of sample-level contrastive learning is to narrow the distance between the same sample in different views and to make negative samples move away. Based on the attention mechanism, different views are assigned weights. First, the query value is calculated. The calculation formula is shown in equation (9): （9）； In the formula, The fused feature matrix, N It is the sample size. It is the global feature mean of all samples; S32. Then calculate the key value, as shown in formula (10): （10）； in, It is the first v The feature mean of each view It is the first v Consistency feature matrix of each view; S33. Next, calculate the attention score for each view, as shown in formula (11): （11）； in, It is a scalar, representing the first... v Attention score between view consistency features and global features; S34. Calculate the attention weight for each view. The calculation formula is shown in Equation (12): （12）； in It is the Sigmoid activation function, used to normalize the attention score. It is the first v Attention weights for each view; S35. Calculate the sample-level contrast loss, as shown in formula (13): （13）； In the formula, yes InfoNCE Comparative losses: （14）； In the formula, It is a temperature parameter learned through sample-level comparative learning. It is a conditional function. It is the similarity between two features.

5. The deep multi-view clustering method based on multi-granularity contrastive learning according to claim 4, characterized in that, S4 includes the following steps: S41. Particle-level contrastive learning is designed to compensate for the deficiency of sample-level contrastive learning in ignoring the local semantic structure between samples. First, K-means++ is used to randomly initialize the centers. S42. Calculate the Euclidean distance from each sample to each center, used for calculating fuzzy membership. The calculation formula is as follows: （15）； In the formula, It is the first v Within the first view i Consistency feature vector of each sample It is the first v Within the first view j The center of each grain, It is the first v Samples within a view i To the j The distance between the centers; S43. Introduce Gaussian blur so that a sample belongs to multiple spheres simultaneously. The calculation formula is as follows: （16）； in, It is the first j The radius of each sphere, It is the first v The first view i The nth sample pair j The membership degree of each sphere is determined by the sphere radius, which is a learnable parameter. An initial value is set, and it will be optimized and updated with backpropagation to automatically adjust the degree of fuzziness to adapt to the density structure inside different spheres. The closer to the center, the higher the membership degree. S44, For each ball k Select samples with a membership degree higher than a threshold as the sample set within the sphere: （17）； In the formula, This is the membership threshold; S45. Calculate the contrast loss inside each particle. k The weighted contrast loss of its internal samples is calculated, with the goal of bringing the sample representations belonging to the same fuzzy particle closer together. The calculation formula is shown in Equation (18): （18）； In the formula, and These are sample characteristics within the same sphere. It is a temperature parameter for particle-level contrastive learning, which controls the smoothness of the similarity distribution; The total intrasphere loss is expressed as: （19）； S46. Introducing cross-view particle-level contrastive learning into particle-level contrastive learning, firstly, the particle center similarity matrix is ​​calculated to determine whether similar particles have consistent representations under different views. The calculation formula is shown in (20): （20）； in, For the first i The first view under the first view p The center of each grain, For the first j The first view under the first view q The center of each grain, The similarity between the centers of the particles in the two views; S47. Calculate the membership overlap matrix. The larger the overlap, the more likely the pair of spheres have captured the same sample structure in different views. The calculation formula is as follows: （21）； in, N Indicates the number of samples. Indicates the first n A sample pair view i granules p membership degree Indicates granules p , q The extent to which samples are shared across different views; Then normalize to a probability distribution: （22）； S48. Only perform positive sample training on samples with high overlap to obtain the weighted sum of the similarity of positive samples. Finally, calculate the loss between cross-view particles. The calculation formula is shown in Equation (23): （23）； In the formula, The threshold is used to determine whether the membership degree of the overlap is greater than the threshold. If it is greater than the threshold, the value is 1; otherwise, it is 0. The final loss is the average between view pairs, as shown in equation (24): （24）。 6. The deep multi-view clustering method based on multi-granularity contrastive learning according to claim 5, characterized in that, S5 includes the following steps: S51. Calculate the total loss of the entire model: （25）； S52. Apply the K-means clustering algorithm to the global feature G to obtain the final clustering result Y.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed, it implements the steps of the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is configured to implement the steps of the method according to any one of claims 1 to 6 when invoked by a processor.

9. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 6.

Images