Asymmetric multi-modal clustering method based on hyperbolic uncertainty modeling

Abstract

The invention belongs to the technical field of deep learning and data mining, and discloses an asymmetric multi-modal clustering method based on hyperbolic uncertainty modeling, and the method comprises the steps: obtaining a multi-modal data set, and carrying out the preprocessing of the multi-modal data set, and obtaining a processed multi-modal data set; inputting the multi-modal data into an encoder network to obtain hidden layer features, and projecting the hidden layer features to obtain hyperbolic embedding features; calculating the distance between the hyperbolic embedding feature and the clustering prototype, calculating the prediction entropy, and constructing an asymmetric modal alignment mechanism, so that the low-entropy modal unidirectionally guides the high-entropy modal to perform expression alignment; fusing the multi-modal soft distribution probability to obtain a global consensus probability and a pseudo tag, and updating a clustering prototype; and performing iterative training on the modal independent automatic encoder network to obtain a trained modal independent automatic encoder network, and outputting a clustering result of the multi-modal data. According to the method and the device, the accuracy and the robustness of multi-modal clustering in a noisy and information-missing environment are remarkably improved.

Description

Technical Field

[0001] This application belongs to the field of deep learning and data mining technology, specifically relating to an asymmetric multimodal clustering method based on hyperbolic uncertainty modeling. Background Technology

[0002] With the rapid development of multi-sensor and multimodal data acquisition technologies, multimodal data has become a major form of information in the real world. Multimodal clustering aims to extract unified semantic consensus from these heterogeneous and multi-source data under unsupervised conditions, so as to achieve automatic grouping and pattern discovery of data.

[0003] However, existing deep multimodal clustering methods mainly face two fundamental bottlenecks:

[0004] First, there is the problem of geometric mismatch. Real-world multimodal data typically contains complex hierarchical structures and varying degrees of uncertainty, while most existing methods map multimodal features to Euclidean space with zero curvature. The volume of Euclidean space grows polynomially, which not only lacks sufficient boundary capacity to separate highly discriminative clusters, but also fails to naturally provide geometric isolation between the determinism and ambiguity of the data. This makes fuzzy and noisy features highly susceptible to interfering with the overall clustering structure.

[0005] Second, rigid alignment leads to noise propagation. Traditional multimodal contrastive learning or alignment strategies typically employ symmetric distance minimization. When one modality contains rich information while another is severely contaminated or missing, this rigid symmetric alignment forces features from the high-quality modality to drift towards the noisy modality, resulting in severe degradation of the representation learned by the model.

[0006] Therefore, there is an urgent need for a novel geometric mapping model and alignment mechanism that can adaptively perceive uncertainties in multimodal data and cut off the negative impact of noisy modes on high-quality modes. Summary of the Invention

[0007] To address the aforementioned technical issues, this application provides an asymmetric multimodal clustering method based on hyperbolic uncertainty modeling. This method utilizes a Poincaré spherical manifold to replace the traditional Euclidean space and innovatively proposes an uncertainty-aware unidirectional asymmetric alignment mechanism, achieving robust fusion and accurate clustering of complex multi-source data.

[0008] To achieve the above objectives, this application employs the following technical solution:

[0009] This application discloses an asymmetric multimodal clustering method based on hyperbolic uncertainty modeling. The asymmetric multimodal clustering method specifically includes the following steps:

[0010] Step 1: Obtain the multimodal dataset, preprocess the multimodal dataset to obtain the processed multimodal dataset;

[0011] Step 2: Input the multimodal data from the multimodal dataset into the pre-established modality-independent autoencoder network, and output the hidden layer features. Use the hidden layer features as vectors in the tangent space of the Poincaré sphere origin, and project the vectors in the tangent space of the Poincaré sphere origin to the hyperbolic space through exponential mapping to output the hyperbolic embedding features.

[0012] Step 3: Based on the hyperbolic embedding features obtained in Step 2, obtain the initial clustering results and extract the cluster prototype. Calculate the hyperbolic distance between the hyperbolic embedding features and the cluster prototype to obtain the soft assignment probability and calculate the prediction entropy to quantify the uncertainty. Based on the prediction entropy, construct a dynamic gating mechanism to generate asymmetric guiding weights.

[0013] Step 4: Calculate the uncertainty perception weight based on the prediction entropy of each modality, fuse the multimodal soft assignment probability, output the global consensus probability and pseudo-label, use a confidence-based masking mechanism to screen reliable samples, use the distance of reliable samples in the tangent space of the Poincaré sphere origin to calculate the candidate cluster prototype, and update the cluster prototype through the momentum strategy.

[0014] Step 5: Using the obtained global consensus probability and pseudo-labels as clustering prediction results, the modality-independent autoencoder network is iteratively trained using reconstruction loss, asymmetric alignment loss, and hyperbolic cross-entropy clustering loss to obtain the trained modality-independent autoencoder network, and the clustering results of the multimodal data are output.

[0015] A further improvement of this application is that step 2 specifically includes the following steps:

[0016] Step 2.1: Transfer multimodal data The input is fed into a pre-established modality-independent encoder network, and the output is the hidden layer features;

[0017] Step 2.2: Use the hidden layer features output in Step 2.1 as the tangent space of the Poincaré sphere origin. vectors in ,in, Describes the dimension of hyperbolic space. Indicates the sample number. Indicates the modal number;

[0018] Step 2.3: Utilizing exponential mapping Cut space from the origin of the Poincaré ball. vectors in Projected onto a hyperbolic space with a curvature of negative one In the middle, output hyperbolic embedding features :

[0019]

[0020] To ensure numerical stability, a space-cutting decoding strategy is adopted, which involves cutting the space of the Poincaré sphere origin. vectors in The input is fed into a modality-independent decoder network, which outputs reconstructed multimodal data. Hyperbolic embedding features Used only for clustering.

[0021] A further improvement of this application is that step 3 specifically includes the following steps:

[0022] Step 3.1: Using the K-means algorithm based on hyperbolic embedding features Obtain the initial clustering results and extract the clustering prototype. Calculate hyperbolic embedding features With clustering prototype hyperbolic distance between :

[0023]

[0024] The soft assignment probability is obtained using the Sinkhorn-Knopp algorithm. Calculate the prediction entropy to quantify uncertainty;

[0025] Step 3.2: Based on the soft assignment probability obtained in Step 3.1 Calculate the prediction entropy for each sample in each modality. :

[0026]

[0027] in, This represents the total number of categories in a multimodal dataset. Indicates the category number;

[0028] Step 3.3: Based on the obtained predicted entropy , and construct a dynamic gating mechanism.

[0029] A further improvement in this application is that, in step 3.3, the dynamic gating mechanism includes asymmetric guided weights and an asymmetric aligned loss function, specifically including the following steps:

[0030] Step 3.3.1: First, calculate the asymmetric guided weights. When the source mode The predicted entropy value is lower than the target mode. hour, ;on the contrary, , Indicates the first The first sample Predictive entropy for each modality;

[0031] Step 3.3.2: Utilize asymmetric guided weights The alignment loss is weighted, and the weighted asymmetric alignment loss function is:

[0032] in, This represents the total number of samples in the multimodal dataset. This represents the total number of modes in the multimodal dataset. Indicates the first The sample at the th Hyperbolic embedding features in each modality Indicates the first The first sample Hyperbolic embedding features in each modality Represents hyperbolic distance. This represents the gradient truncation operation during neural network training.

[0033] A further improvement in this application is that step 4 specifically includes the following steps:

[0034] Step 4.1: Calculate the uncertainty-aware weights based on the prediction entropy of each modality. :

[0035]

[0036] Fusion soft allocation probability The output yields the global consensus probability. and pseudo-tags :

[0037]

[0038] ;

[0039] Step 4.2: Based on global consensus probability From all samples in the multimodal dataset, the top 50% of samples, ranked from highest to lowest confidence, are selected to participate in the clustering loss calculation. The masked hyperbolic cross-entropy clustering loss is:

[0040]

[0041] in, This is a binary mask vector. The top 50% of samples, sorted by confidence level from highest to lowest, have a binary mask of 1; otherwise, they have a binary mask of 0. Indicates and The closest cluster prototype, For the first A clustering prototype;

[0042] Step 4.3: Using the top 50% of samples ranked from highest to lowest confidence, calculate the candidate cluster prototypes in the tangent space of the Poincaré sphere origin using the K-means algorithm. Update candidate clustering prototypes using momentum strategies Corresponding hyperbolic space Tangent space clustering prototype The momentum strategy update formula is as follows:

[0043] in, and They are the first Second and third The th iteration A prototype for scissor space clustering. It is the momentum update coefficient, representing the proportion of the prototype updated in each iteration;

[0044] Step 4.4: Cluster the prototype of the tangent space using exponential mapping. Reprojected back into hyperbolic space As the final clustering prototype .

[0045] A further improvement in this application is that, in step 5, the modality-independent autoencoder network is iteratively trained using reconstruction loss, asymmetric alignment loss function, and hyperbolic cross-entropy clustering loss, specifically including the following steps:

[0046] Step 5.1: Minimize reconstruction loss The preheating process for initializing the modal independent autoencoder network is completed, enabling the tangent space features, i.e., the tangent space of the Poincaré sphere origin, to be realized. vectors in Initial capture of the intrinsic structure of multimodal data;

[0047] Step 5.2: Enter the joint training phase and jointly optimize the reconstruction loss. Asymmetric alignment loss function under hyperbolic distance and hyperbolic cross-entropy clustering loss The pre-established modality-independent autoencoder network is iteratively trained to obtain the total loss:

[0048]

[0049] The modality-independent autoencoder network is trained through iterative training, and finally, the clustering results of the multimodal data are output.

[0050] This application discloses an asymmetric multimodal clustering system based on hyperbolic uncertainty modeling, comprising:

[0051] The preprocessing module is used to acquire the multimodal dataset, preprocess the multimodal dataset, and obtain the processed multimodal dataset.

[0052] The feature extraction and projection module is used to input multimodal data into a pre-established modality-independent encoder network and output hidden layer features. The hidden layer features are used as vectors in the tangent space of the Poincaré sphere origin and projected into hyperbolic space through exponential mapping to output hyperbolic embedding features.

[0053] like Figure 3 As shown, the alignment calculation module is used to calculate the hyperbolic distance between the hyperbolic embedded features and the cluster prototype, obtain the soft assignment probability, and calculate the prediction entropy to quantify the uncertainty. Based on the prediction entropy, a dynamic gating mechanism is constructed to generate asymmetric alignment weights.

[0054] The prototype update module calculates the uncertainty perception weight based on the prediction entropy of each modality, integrates the multimodal soft assignment probability, outputs the global consensus probability and pseudo-label, uses a confidence-based masking mechanism to screen reliable samples, uses the distance of reliable samples in the tangent space of the Poincaré sphere origin to calculate the candidate clustering prototype, and updates the clustering prototype through the momentum strategy.

[0055] The training and clustering module uses the obtained global consensus probability and pseudo-labels as clustering prediction results. It iteratively trains the modality-independent autoencoder network using reconstruction loss, asymmetric alignment loss function and hyperbolic cross-entropy clustering loss function to obtain the trained modality-independent autoencoder network and output the clustering results of multimodal data.

[0056] The beneficial effects of this application are: the asymmetric multimodal clustering method based on hyperbolic uncertainty modeling proposed in this application has the characteristics of good separability, strong robustness, and strong generalization ability, specifically:

[0057] This application solves the geometric mismatch problem and improves cluster separability. It fully utilizes the exponential volume growth characteristic of the hyperbolic Poincaré sphere model to push high-confidence discriminative samples towards the boundary, significantly increasing inter-cluster distance; simultaneously, it anchors fuzzy samples near the origin, effectively isolating noise interference.

[0058] This application overcomes the feature degradation problem caused by rigid alignment and has strong robustness. The innovative asymmetric modality alignment mechanism proposed in this application dynamically evaluates modality quality through instance-level prediction entropy, realizing unidirectional knowledge transfer with superior modalities leading inferior ones, fundamentally preventing the negative pull of damaged modalities on high-quality representations.

[0059] This application demonstrates strong generalization ability. By combining a consensus-aware global pseudo-label filtering mechanism, this method maintains excellent clustering accuracy even in extremely harsh data environments such as large-scale data noise and high missing rate, significantly outperforming existing advanced multimodal clustering schemes. Attached Figure Description

[0060] Figure 1 This is a flowchart of the asymmetric multimodal clustering method of this application.

[0061] Figure 2 This is a schematic diagram of the network structure of the feature extraction and projection module of this application.

[0062] Figure 3 This is a schematic diagram of the model structure of the alignment calculation module in this application. Detailed Implementation

[0063] The embodiments of the present invention will be disclosed below with reference to the drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the present invention. That is, in some embodiments of the present invention, these practical details are not essential. In addition, for the sake of simplicity, some conventional structures and components will be shown in the drawings in a simple schematic manner.

[0064] like Figure 1 As shown, this application presents an asymmetric multimodal clustering method based on hyperbolic uncertainty modeling, which specifically includes the following steps:

[0065] Step 1: Obtain the multimodal dataset, preprocess the multimodal dataset to obtain the processed multimodal dataset.

[0066] Step 2: Input the multimodal data from the multimodal dataset into a pre-established modality-independent autoencoder network, and output the hidden layer features. Treat these hidden layer features as vectors in the tangent space of the Poincaré sphere origin, and project these vectors into hyperbolic space using an exponential mapping, outputting hyperbolic embedding features. The modality-independent autoencoder network consists of two parts: a modality-independent encoder network and a modality-independent decoder network. Specifically, it includes the following steps:

[0067] Step 2.1: Transfer multimodal data The input is fed into a pre-established modality-independent encoder network, and the output is the hidden layer features;

[0068] Step 2.2: Use the hidden layer features output in Step 2.1 as the tangent space of the Poincaré sphere origin. vectors in ,in, Describes the dimension of hyperbolic space. Indicates the sample number. Indicates the modal number;

[0069] Step 2.3: Utilizing exponential mapping Cut space from the origin of the Poincaré ball. vectors in Projected onto a hyperbolic space with a curvature of negative one In the middle, output hyperbolic embedding features :

[0070]

[0071] Through this projection, the norm of the hyperbolic embedding implicitly encodes the confidence level of the data. To ensure numerical stability, a tangent space decoding strategy is adopted, that is, tangent space is used to the origin of the Poincaré sphere. vectors in The input is fed into a modality-independent decoder network, which outputs reconstructed multimodal data. Hyperbolic embedding features Used only for clustering.

[0072] Step 3: Based on the hyperbolic embedding features obtained in Step 2, obtain the initial clustering results and extract the cluster circles. Calculate the hyperbolic distance between the hyperbolic embedding features and the cluster prototypes to obtain the soft assignment probability and calculate the prediction entropy to quantify uncertainty. Based on the prediction entropy, construct a dynamic gating mechanism to generate asymmetric guiding weights, enabling the low-entropy mode to unidirectionally guide the high-entropy mode for representation alignment. Specifically, this includes the following steps:

[0073] Step 3.1: Using the K-means algorithm based on hyperbolic embedding features Obtain the initial clustering results and extract the clustering prototype. Calculate hyperbolic embedding features With clustering prototype hyperbolic distance between :

[0074]

[0075] The soft assignment probability is obtained using the Sinkhorn-Knopp algorithm. Calculate the prediction entropy to quantify uncertainty;

[0076] Based on the geometric interpretation of the Poincaré sphere, samples in modes with higher confidence are pushed toward the boundary ( ), with lower prediction entropy; while samples in more ambiguous or noisy multimodal datasets remain at the origin ( It has a high prediction entropy.

[0077] Step 3.2: Based on the soft assignment probability obtained in Step 3.1 Calculate the prediction entropy for each sample in each modality. :

[0078]

[0079] in, This represents the total number of categories in a multimodal dataset. Indicates the category number, i.e., the first One category;

[0080] Step 3.3: Based on the geometric interpretation of the Poincaré sphere and the calculated predicted entropy A dynamic gating mechanism is constructed, which includes asymmetric guided weights and asymmetric aligned loss functions. The specific steps are as follows:

[0081] Step 3.3.1: First, calculate the asymmetric guided weights. When the source mode The predicted entropy value is lower than the target mode. hour, ;on the contrary, , Indicates the first The first sample Predictive entropy for each modality;

[0082] Step 3.3.2: Utilize asymmetric guided weights Weighting the alignment loss allows modes with lower entropy to unidirectionally guide modes with higher entropy, while modes with higher entropy do not affect modes with lower entropy. Here, "low" and "high" are relative to the high and low values ​​in the two multimodal datasets. The weighted asymmetric alignment loss function is:

[0083] in, This represents the total number of samples in the multimodal dataset. This represents the total number of modes in the multimodal dataset. Indicates the first The sample at the th Hyperbolic embedding features in each modality Indicates the first The first sample Hyperbolic embedding features in each modality Represents hyperbolic distance. This represents the gradient truncation operation during neural network training.

[0084] Step 4: Calculate the uncertainty-aware weights based on the prediction entropy of each modality, fuse the multimodal soft assignment probabilities, and output the global consensus probability and pseudo-labels. Use a confidence-based masking mechanism to screen reliable samples, calculate candidate cluster prototypes using the distances of reliable samples in the tangent space of the Poincaré sphere origin, and update the cluster prototypes using a momentum strategy. Specifically, this includes the following steps:

[0085] Step 4.1: Calculate the uncertainty-aware weights based on the prediction entropy of each modality. :

[0086]

[0087] Fusion soft allocation probability The output yields the global consensus probability. and pseudo-tags :

[0088]

[0089] ;

[0090] Step 4.2: Based on global consensus probability From all samples in the multimodal dataset, the top 50% of samples, ranked from highest to lowest confidence, are selected to participate in the clustering loss calculation. The masked hyperbolic cross-entropy clustering loss is:

[0091]

[0092] in, This is a binary mask vector. The top 50% of samples, sorted by confidence level from highest to lowest, have a binary mask of 1; otherwise, they have a binary mask of 0. Indicates and The closest cluster prototype, For the first A clustering prototype;

[0093] Step 4.3: Using the top 50% of samples ranked from highest to lowest confidence, calculate the candidate cluster prototypes in the tangent space of the Poincaré sphere origin using the K-means algorithm. Update candidate clustering prototypes using momentum strategies Corresponding hyperbolic space Tangent space clustering prototype The momentum strategy update formula is as follows:

[0094] in, and They are the first Second and third The th iteration A prototype for scissor space clustering. It is the momentum update coefficient, representing the proportion of the prototype updated in each iteration;

[0095] Step 4.4: Cluster the prototype of the tangent space using exponential mapping. Reprojected back into hyperbolic space As the final clustering prototype .

[0096] Step 5: Using the obtained global consensus probability and pseudo-labels as clustering prediction results, the modality-independent autoencoder network is iteratively trained using reconstruction loss, asymmetric alignment loss, and hyperbolic cross-entropy clustering loss to obtain the trained modality-independent autoencoder network, and the clustering results of the multimodal data are output.

[0097] In this step, the modality-independent autoencoder network is iteratively trained using reconstruction loss, asymmetric alignment loss function, and hyperbolic cross-entropy clustering loss, specifically including the following steps:

[0098] Step 5.1: Minimize reconstruction loss The preheating process for initializing the modal independent autoencoder network is completed, enabling the tangent space features, i.e., the tangent space of the Poincaré sphere origin, to be realized. vectors in Initial capture of the intrinsic structure of multimodal data;

[0099] Step 5.2: Enter the joint training phase and jointly optimize the reconstruction loss. Asymmetric alignment loss function under hyperbolic distance and hyperbolic cross-entropy clustering loss The pre-established modality-independent autoencoder network is iteratively trained to obtain the total loss:

[0100]

[0101] The modality-independent autoencoder network achieves co-evolution of feature representation and clustering structure during iterative training, resulting in a trained modality-independent autoencoder network. Finally, it outputs the clustering results of multimodal data, realizing the co-evolution of feature representation and clustering structure.

[0102] like Figure 2 As shown, this application also provides an asymmetric multimodal clustering system based on hyperbolic uncertainty modeling, comprising:

[0103] The preprocessing module is used to acquire the multimodal dataset, preprocess the multimodal dataset, and obtain the processed multimodal dataset.

[0104] The feature extraction and projection module is used to input multimodal data into a pre-established modality-independent encoder network and output hidden layer features. The hidden layer features are used as vectors in the tangent space of the Poincaré sphere origin and projected into hyperbolic space through exponential mapping to output hyperbolic embedding features.

[0105] The alignment calculation module is used to calculate the hyperbolic distance between hyperbolic embedded features and cluster prototypes, obtain soft assignment probabilities, and calculate prediction entropy to quantify uncertainty. Based on the prediction entropy, a dynamic gating mechanism is constructed to generate asymmetric alignment weights, so that the low-entropy mode unidirectionally guides the high-entropy mode for representation alignment.

[0106] The prototype update module calculates the uncertainty perception weight based on the prediction entropy of each modality, integrates the multimodal soft assignment probability, outputs the global consensus probability and pseudo-label, uses a confidence-based masking mechanism to screen reliable samples, uses the distance of reliable samples in the tangent space of the Poincaré sphere origin to calculate the candidate clustering prototype, and updates the clustering prototype through the momentum strategy.

[0107] The training and clustering module uses the obtained global consensus probability and pseudo-labels as clustering prediction results. It iteratively trains the modality-independent autoencoder network using reconstruction loss, asymmetric alignment loss function and hyperbolic cross-entropy clustering loss function to obtain the trained modality-independent autoencoder network and output the clustering results of multimodal data.

[0108] To verify this application, the following embodiments will further illustrate this application:

[0109] This embodiment compares with other current state-of-the-art multimodal clustering methods on the Caltech-101, CUB, LandUse-21, NoisyMNIST, Reuters, and Scene-15 datasets.

[0110] Table 1. Performance comparison of multimodal clustering methods on various datasets.

[0111]

[0112] This application conducted experiments on six datasets and compared the results with previous state-of-the-art methods. The comparison results in Table 1 with current leading multimodal clustering methods demonstrate that the multimodal clustering performance of this application surpasses that of the other compared methods, illustrating the effectiveness of this application.

[0113] To verify the effectiveness of the model proposed in this application, a series of ablation experiments were conducted. The main focus was on the effects of hyperbolic space mapping, dynamic gating mechanisms, uncertainty-aware weights, and binary masks. The results of these ablation studies explain the fundamental role of each component in the model's performance.

[0114] Table 2 shows the impact of different modules on each dataset.

[0115]

[0116] As shown in Table 2 above, the best results can be obtained when all modules are used. Removing any module will lead to a decrease in multimodal clustering performance, indicating that multimodal clustering is meaningful for any module.

[0117] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. An asymmetric multimodal clustering method based on hyperbolic uncertainty modeling, characterized in that: The asymmetric multimodal clustering method specifically includes the following steps: Step 1: Obtain the multimodal dataset, preprocess the multimodal dataset to obtain the processed multimodal dataset; Step 2: Input the multimodal data from the multimodal dataset into the pre-established modality-independent autoencoder network, and output the hidden layer features. Use the hidden layer features as vectors in the tangent space of the Poincaré sphere origin, and project the vectors into the hyperbolic space through exponential mapping to output the hyperbolic embedding features. Step 3: Based on the hyperbolic embedding features output in Step 2, obtain the initial clustering results and extract the clustering prototype. Calculate the hyperbolic distance between the hyperbolic embedding features and the clustering prototype to obtain the soft assignment probability and calculate the prediction entropy to quantify the uncertainty. Based on the prediction entropy, construct a dynamic gating mechanism to generate asymmetric guiding weights. Step 4: Calculate the uncertainty perception weight based on the prediction entropy of each modality, fuse the multimodal soft assignment probability, output the global consensus probability and pseudo-label, use a confidence-based masking mechanism to screen reliable samples, use the distance of reliable samples in the tangent space of the Poincaré sphere origin to calculate the candidate cluster prototype, and update the cluster prototype through the momentum strategy. Step 5: Using the obtained global consensus probability and pseudo-labels as clustering prediction results, the modality-independent autoencoder network is iteratively trained using reconstruction loss, asymmetric alignment loss, and hyperbolic cross-entropy clustering loss to obtain the trained modality-independent autoencoder network, and the clustering results of the multimodal data are output.

2. The asymmetric multimodal clustering method based on hyperbolic uncertainty modeling according to claim 1, characterized in that: Step 2 specifically includes the following steps: Step 2.1: Extract the multimodal data from the multimodal dataset. The input is fed into a pre-established modality-independent encoder network, and the output is the hidden layer features; Step 2.2: Use the hidden layer features output in Step 2.1 as the tangent space of the Poincaré sphere origin. vectors in ,in, Describes the dimension of hyperbolic space. Indicates the sample number. Indicates the modal number; Step 2.3: Utilizing exponential mapping Cut space from the origin of the Poincaré ball. vectors in Projected onto a hyperbolic space with a curvature of negative one In the middle, output hyperbolic embedding features : To ensure numerical stability, a space-cutting decoding strategy is adopted, which involves cutting the space of the Poincaré sphere origin. vectors in The input is fed into a modality-independent decoder network, which outputs reconstructed multimodal data. Hyperbolic embedding features Used only for clustering.

3. The asymmetric multimodal clustering method based on hyperbolic uncertainty modeling according to claim 2, characterized in that: Step 3 specifically includes the following steps: Step 3.1: Using the K-means algorithm based on hyperbolic embedding features Obtain the initial clustering results and extract the clustering prototype. Calculate hyperbolic embedding features With clustering prototype hyperbolic distance between : The soft assignment probability is obtained using the Sinkhorn-Knopp algorithm. Calculate the prediction entropy to quantify uncertainty; Step 3.2: Based on the soft assignment probability obtained in Step 3.1 Calculate the prediction entropy for each sample in each modality. : in, This represents the total number of categories in a multimodal dataset. Indicates the category number; Step 3.3: Based on the obtained predicted entropy , and construct a dynamic gating mechanism.

4. The asymmetric multimodal clustering method based on hyperbolic uncertainty modeling according to claim 3, characterized in that: In step 3.3, the dynamic gating mechanism includes asymmetric guided weights and asymmetric aligned loss functions, specifically including the following steps: Step 3.3.1: First, calculate the asymmetric guided weights. When the source mode The predicted entropy value is lower than the target mode. hour, ,on the contrary, , Indicates the first The first sample Predictive entropy for each modality; Step 3.3.2: Utilize asymmetric guided weights The alignment loss is weighted, and the weighted asymmetric alignment loss function is: in, This represents the total number of samples in the multimodal dataset. This represents the total number of modes in the multimodal dataset. Indicates the first The sample at the th Hyperbolic embedding features in each modality Indicates the first The first sample Hyperbolic embedding features in each modality Represents hyperbolic distance. This represents the gradient truncation operation during neural network training.

5. The asymmetric multimodal clustering method based on hyperbolic uncertainty modeling according to claim 3, characterized in that: Step 4 specifically includes the following steps: Step 4.1: Calculate the uncertainty-aware weights based on the prediction entropy of each modality. : Fusion soft allocation probability The output yields the global consensus probability. and pseudo-tags : ； Step 4.2: Based on global consensus probability From all samples in the multimodal dataset, the top 50% of samples, ranked from highest to lowest confidence, are selected to participate in the clustering loss calculation. The masked hyperbolic cross-entropy clustering loss is: in, This is a binary mask vector. The top 50% of samples, sorted by confidence level from highest to lowest, have a binary mask of 1; otherwise, they have a binary mask of 0. Indicates and The closest cluster prototype, For the first A clustering prototype; Step 4.3: Using the top 50% of samples ranked from highest to lowest confidence, calculate the candidate cluster prototypes in the tangent space of the Poincaré sphere origin using the K-means algorithm. Update candidate clustering prototypes using momentum strategies Corresponding hyperbolic space Tangent space clustering prototype The momentum strategy update formula is as follows: in, and They are the first Second and third The th iteration A prototype for scissor space clustering. It is the momentum update coefficient, representing the proportion of the prototype updated in each iteration; Step 4.4: Cluster the prototype of the tangent space using exponential mapping. Reprojected back into hyperbolic space As the final clustering prototype .

6. The asymmetric multimodal clustering method based on hyperbolic uncertainty modeling according to claim 5, characterized in that: In step 5, the modality-independent autoencoder network is iteratively trained using reconstruction loss, asymmetric alignment loss function, and hyperbolic cross-entropy clustering loss, specifically including the following steps: Step 5.1: Minimize reconstruction loss The preheating process for initializing the modal independent autoencoder network is completed, enabling the tangent space features, i.e., the tangent space of the Poincaré sphere origin, to be realized. vectors in Initial capture of the intrinsic structure of multimodal data; Step 5.2: Enter the joint training phase and jointly optimize the reconstruction loss. Asymmetric alignment loss function under hyperbolic distance and hyperbolic cross-entropy clustering loss The pre-established modality-independent autoencoder network is iteratively trained to obtain the total loss: The modality-independent autoencoder network is trained through iterative training, and finally, the clustering results of the multimodal data are output.

7. The asymmetric multimodal clustering method based on hyperbolic uncertainty modeling according to claim 1, characterized in that: The asymmetric multimodal clustering method is implemented through an asymmetric multimodal clustering system, which includes: The preprocessing module is used to acquire the multimodal dataset, preprocess the multimodal dataset, and obtain the processed multimodal dataset. The feature extraction and projection module is used to input multimodal data into a pre-established modality-independent encoder network and output hidden layer features. The hidden layer features are used as vectors in the tangent space of the Poincaré sphere origin and projected into hyperbolic space through exponential mapping to output hyperbolic embedding features. The alignment calculation module is used to calculate the hyperbolic distance between hyperbolic embedded features and cluster prototypes, obtain soft assignment probabilities, and calculate prediction entropy to quantify uncertainty. Based on the prediction entropy, a dynamic gating mechanism is constructed to generate asymmetric alignment weights. The prototype update module calculates the uncertainty perception weight based on the prediction entropy of each modality, integrates the multimodal soft assignment probability, outputs the global consensus probability and pseudo-label, uses a confidence-based masking mechanism to screen reliable samples, uses the distance of reliable samples in the tangent space of the Poincaré sphere origin to calculate the candidate clustering prototype, and updates the clustering prototype through the momentum strategy. The training and clustering module uses the obtained global consensus probability and pseudo-labels as clustering prediction results. It iteratively trains the modality-independent autoencoder network using reconstruction loss, asymmetric alignment loss function and hyperbolic cross-entropy clustering loss function to obtain the trained modality-independent autoencoder network and output the clustering results of multimodal data.

Images