Unified modal perception noise dynamic editing method for multi-modal sentiment analysis

Abstract

The invention belongs to the technical field of multi-modal model training, and discloses a unified modal perception noise dynamic editing method for multi-modal sentiment analysis. A unified mode is adopted to perceive a dynamic noise editing model, blocking, de-noising intensity calculation, noise editing and alignment are carried out in sequence, and de-noising interpretation is carried out; the de-noising sub-module is used for dynamically editing noise at an input coding end; the noise-driven contrast learning module promotes the unified modal perception noise dynamic editing model to learn and discriminate noise information according to positive and negative samples formed by de-noising features and original features; and the noise suppression cross attention module suppresses noise information according to the de-noising intensity information obtained in the de-noising sub-module during cross-modal interaction according to the de-noising intensity of the corresponding region. The model aims to form a unified capability model with both accuracy and robustness, promotes multi-modal emotion understanding to be applied to large-scale and sustainable industrial-level application from laboratory verification, and supports human-computer interaction and intelligent decision-making of key scenes such as customer service, education and medical treatment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multimodal model training, in particular to a unified modal perception noise dynamic editing method for multimodal sentiment analysis. BACKGROUND

[0002] Multimodal sentiment analysis (MSA) refers to joint modeling in heterogeneous data such as text, vision and audio, and recognizing and understanding the emotional state of individuals or scenes. Compared with methods relying only on a single modality, MSA can integrate language content, non-language clues and acoustic prosody to obtain more comprehensive and more robust emotional understanding ability. This technology has shown significant application value in human-computer interaction, online education, psychological health auxiliary evaluation, customer service quality inspection, media content understanding and social public opinion analysis.

[0003] Early work largely progressed from the "alignment-fusion" paradigm. TFN (Zadeh A, Chen M, Poria S, et al. Tensor Fusion Network for Multimodal Sentiment Analysis[C] Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.2017: 1103-1114.) constructs tensor representations that simultaneously contain unimodal, first-order pairwise interactions, and trimodal higher-order interactions through outer products, thus fully capturing cross-modal correlations. LMF (Liu Z, Shen Y, Lakshminarasimhan VB, et al. Efficient low-rank multimodal fusion with modality-specific factors [C] Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018: 2247-2256.) performs low-rank decomposition on the fusion weight tensor, achieving an approximation of "rank-1 outer product summation" with several rank-R modality-specific factors. This reduces the complexity from a product of dimensions to the sum of dimensions × R, improving training stability and generalization ability. Subsequently, Transformer-based cross-modal attention methods have also made progress in alignment quality and expressive power. MulT (Tsai YHH, Bai S, Liang PP, et al. Multimodal transformer for unaligned multimodal language sequences[C]Proceedings of the conference. Association for computational linguistics.Meeting. 2019, 2019: 6558.) proposed directional pairwise cross-modal attention for "unaligned" multimodal language sequences. In end-to-end training, it learns alignment and interaction simultaneously with soft alignment, and can capture long-range dependencies across modalities without forced alignment. It has become a representative work of the cross-modal attention paradigm.MMML (Wu Z, Gong Z, Koo J, et al. Multimodal multi-loss fusion network for sentiment analysis[C] Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: LongPapers). 2024: 3588-3602.) starts from systems engineering and training signal design, combining multimodal encoders and employing multi-loss collaborative optimization in the fusion layer. The system compares various fusion strategies and introduces contextual information, improving robustness and generalization. It is often used as the backbone and strong baseline in modern MSA frameworks. In addition, the approach of using external knowledge to guide attention to augmented representations has also been applied. KuDA (Feng, Xinyu, Lin, Yuming, He, Lihua, et al. Knowledge-Guided Dynamic Modality Attention Fusion Framework for Multimodal Sentiment Analysis[C] Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2024: 14755-14766) is a knowledge-guided dynamic modality attention fusion framework. By introducing sentiment priors to guide multi-level features, the model can adaptively determine and amplify the "dominant modality" at the sample / time level, further strengthening the modality's contribution to the final representation.

[0004] Modal analysis reveals a significant amount of noise in multimodal sentiment data. This misleading noise severely interferes with the model's representation learning, leading to performance degradation. A simple and direct solution to modal noise is to train a separate model from scratch for each type of noise. TFR-Net (Yuan Z, Li W, Xu H, et al. Transformer-based feature reconstruction network for robust multimodalsentiment analysis[C] Proceedings of the 29th ACM international conference onmultimedia. 2021: 4400-4407.) uses a Transformer as its backbone, first extracting element-level representations using intra-modal / cross-modal attention, and then explicitly generating missing modal features through a reconstruction module to mitigate noise interference. TPFN (Li B, LiC, Duan F, et al. TPFN: Applying outer product along time to multimodalsentiment analysis fusion on incomplete data[C] European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 431-447.) performs an outer product along time on the multimodal features of adjacent steps within a time window to capture higher-order temporal interactions and obtain robust features. JOSFD (Jiang X, Xu X, Lu H, et al. Joint objective and subjectivefuzziness denoising for multimodal sentiment analysis[J]. IEEE Transactions on Fuzzy Systems, 2024, 33(1): 15-27.) decomposes sentiment uncertainty into objective fuzziness introduced by expression and signal noise and subjective fuzziness caused by annotation subjectivity and psychological complexity. It introduces uncertainty estimation and fuzzification / denoising modules in an end-to-end framework and simultaneously suppresses both types of fuzziness with a joint objective, thereby obtaining a more robust joint representation and performance improvement on common benchmarks.t-HNE (Li Z, Li L. t-HNE: A Text-guided HierarchicalNoise Eliminator for Multimodal Sentiment Analysis[C] Proceedings of the 31st International Conference on Computational Linguistics. 2025: 2834-2844.) constructs a hierarchical denoising and feature recovery process anchored to text. It injects textual information into visual and acoustic representations through attention to reduce intramodal noise, followed by cross-modal denoising and feature recovery / contrast constraints to obtain a cleaner fused representation.

[0005] Early multimodal sentiment analysis techniques such as TFN, LMF, MulT, and MMML ignored the impact of noise. While these early representative methods propelled the development of multimodal sentiment analysis, they still suffer from systemic shortcomings in handling noise and uncertainty in real-world scenarios. Firstly, they typically assume clean, aligned, and complete inputs, failing to explicitly characterize pragmatic noise such as visual occlusion / low light, audio reverberation / accordion, text ASR errors, and irony. This makes TFN and LMF prone to combining and amplifying local noise, MulT susceptible to attention drift due to spurious correlations, and MMML amplifying the impact of noisy labels when uncertainty weights are lacking. Secondly, these methods are sensitive to temporal misalignment and modality quality fluctuations, lacking sample-level dynamic modality selection, cross-modal compensation, or feature reconstruction mechanisms. Furthermore, they lack invariance and consistency regularization under cross-distribution drift, leading to unstable generalization. Simultaneously, they lack sufficient interpretability and struggle to provide uncertainty estimation and causal evidence.

[0006] Although TFR-Net and TPFN have begun to address modal noise—the former reconstructing missing features using Transformer and the latter improving fusion robustness through time-series outer products and low-rank constraints—they essentially still treat noise as a latent variable learned along with the representation, lacking explicit characterization, estimation, and suppression mechanisms for noise. While JOSFD and t-HNE introduce denoising approaches, different modalities exhibit highly heterogeneous and non-uniform noise sources, forms, and intensity distributions. A uniform global denoising intensity cannot simultaneously satisfy the local requirements of "strong noise requiring strong suppression, and weak noise requiring gentle fidelity preservation," easily leading to over-suppression or under-suppression, excessive information smoothing, and cross-modal mismatch. Therefore, a single, static, or one-size-fits-all strategy for the entire modality is insufficient to handle complex noise patterns across modalities and scenarios. Fine-grained noise estimation and uncertainty weighting based on samples are urgently needed to achieve an interpretable optimal balance between noise suppression and information preservation.

[0007] In summary, current multimodal sentiment analysis (MSA) research largely focuses on direct modality integration and simple modality denoising, failing to adequately consider the dual challenges of intermodal heterogeneity and intramodal noise non-uniformity. MSA lacks a method that can both adaptively segment based on modal characteristics and dynamically assign differentiated intensities to different local regions / segments. Based on this situation, there is an urgent need for a general technical solution for real-world scenarios that allows for fine-grained dynamic noise editing. This solution should effectively suppress interference from noisy regions without sacrificing key information, while avoiding over-smoothing of weak / effective regions, thereby improving the robustness and generalization ability of multimodal sentiment analysis in complex environments. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of existing multimodal sentiment analysis in handling modal noise, such as ignoring modal noise and applying a one-size-fits-all approach to denoising the entire modality. This invention proposes a unified modality-aware local-adaptive noise editing model, MoLAN+, for multimodal sentiment analysis in real-world scenarios. This invention can stably improve the accuracy and consistency of sentiment recognition without relying on complex engineering or extensive prior knowledge, effectively suppressing attention contamination and reducing false positives on critical information. MoLAN+ aims to form a unified capability model that combines accuracy and robustness, driving multimodal sentiment understanding from laboratory validation to large-scale, sustainable industrial applications, supporting human-computer interaction and intelligent decision-making in key scenarios such as customer service, education, and healthcare.

[0009] The technical solution of the present invention is as follows: A unified modality-aware noise dynamic editing method for multimodal sentiment analysis, which adopts a unified modality-aware noise dynamic editing model, and uses block segmentation, noise reduction intensity calculation, noise editing, and alignment as the main lines for noise reduction interpretation;

[0010] The unified modality-aware noise dynamic editing model includes a denoising submodule, a noise-driven contrastive learning module, and a noise suppression cross-attention module. The denoising submodule performs dynamic noise editing at the input encoding end. The noise-driven contrastive learning module enables the unified modality-aware noise dynamic editing model to learn to distinguish noise information based on positive and negative samples composed of denoised features and original features. The noise suppression cross-attention module suppresses noise information according to the denoising intensity of the corresponding region during cross-modal interaction based on the denoising intensity information obtained from the denoising submodule.

[0011] After passing through the noise-suppressed cross-attention module, information from different modalities can be effectively interacted and integrated, thereby achieving the initial fusion of multimodal features. The interacted representations are then input into the self-attention mechanism to further model deep intramodal dependencies. Information from different modalities is integrated into a unified representation for sentiment analysis or sentiment classification tasks.

[0012] Given text, visual, and audio inputs, features are extracted using the corresponding modal encoders; text features Visual features Audio features The three elements are denoised using a denoising submodule; different modal features are segmented based on modality perception, with audio using one-dimensional segmentation, visuals using two-dimensional segmentation, and text not segmented; based on the optimal segmentation parameters, each modal feature is transformed into a feature block:

[0013] For each feature block, the denoising intensity is calculated; the text feature is used as the main modality, and the similarity between the other modality feature blocks and the text feature is calculated as the denoising intensity; denoising is performed according to the denoising intensity of different blocks, and noise is dynamically edited to obtain the clean features after denoising.

[0014] The visual feature segmentation process is as follows:

[0015]

[0016] This indicates the optimal block size for the partition; Represents the first feature dimension factor, Indicates the second feature dimension factor, Represents the set of features of the first dimension of the modality. Represents the set of features of the second dimension of the modality. Represents the first feature dimension of the modality, This represents the second feature dimension of the modality;

[0017] Based on the optimal block parameters, visual features are transformed into feature blocks:

[0018]

[0019] Indicates the first dimension block number, Indicates the second dimension block number, Indicates the optimal block size in the first dimension. This represents the optimal block size in the second dimension.

[0020] The similarity between the feature blocks of the visual modality and the text features is used as the denoising strength. :

[0021]

[0022] in, This indicates similarity calculation.

[0023] The noise-driven contrastive learning module forms positive sample pairs between the denoised clean features and their semantic counterparts, and forms negative sample pairs between the corresponding high-noise original features, cross-domain perturbation fragments, and regions inconsistent with the text. The training objective is to minimize the distance between positive sample pairs and maximize the discriminative power between positive and negative samples in the representation space, enabling the unified modality-aware noise dynamic editing model to focus on stable factors related to emotion while downplaying interference factors unrelated to emotion. The specific loss design is as follows:

[0024]

[0025] in This represents the total learning loss compared to the previous one. Indicates visual modal contrast loss. This represents the audio modal contrast loss. This indicates the expected calculation. , Represents the denoised visual and audio modal features. , Represents the original modal features. , Indicates other sample features;

[0026] The noise-driven contrastive learning module adaptively adjusts the weights and temperature of the samples: the stronger the denoising intensity, the looser the constraints and the lower the weights; the weaker the denoising intensity, the stricter the constraints and the higher the weights.

[0027] A noise-suppressed cross-attention module is introduced in the cross-modal interaction stage; the weights of attention are suppressed and redistributed using the denoising intensity as a constraint signal.

[0028] The noise suppression cross-attention module calculates the denoising intensity based on the similarity between the block and the text anchor point, and sets a threshold θ; blocks with a denoising intensity higher than θ are considered high-noise regions.

[0029] First, the noise mask matrix M is reorganized and merged, and aggregated according to spatial or temporal adjacency to obtain the complete mask matrix. This is used to uniformly constrain the contribution of the region to attention; a negative bias is applied to the log probability of attention at the masked position to reduce its probability of being noticed; on the other hand, amplitude gating is applied to the value vector of the masked position to limit the influence of the noise region on the output of the fused representation; and the adjustment of cross attention based on the noise mask matrix effectively avoids excessive suppression of effective information.

[0030]

[0031]

[0032] This represents a threshold function that filters regions based on a threshold. Let R represent the visual or audio feature matrix, and R denote the matrix recombination operation; channel suppression is applied to attention scoring based on the mask matrix.

[0033]

[0034] in Indicates the modal characteristics of the query. Modal characteristics representing keys and values, This indicates cross-attention calculation. This represents the modal features after passing through the noise-suppressed cross-attention module; This represents the noise mask matrix for the query modality features.

[0035] The beneficial effects of this invention are as follows: This invention proposes a unified modality-aware noise dynamic editing method for multimodal sentiment analysis (MSA), and designs a unified modality-aware noise dynamic editing model, MoLAN+, for MSA. The denoising submodule addresses the problems of undersuppression of strong noise, oversuppression of weak noise, and misinterpretation of key information caused by a one-size-fits-all approach to denoising the entire modality. It divides the visual / audio signal into patches and time segments, estimates the noise intensity coefficient of each local segment using text similarity, and employs soft masking and microstepping editing to limit and suppress high-noise blocks while preserving the fidelity of low-noise blocks. Ultimately, it achieves both noise suppression and effective information preservation, significantly reducing misjudgments and information loss, and improving the performance of the MSA task.

[0036] Secondly, this invention constructs a noise-driven contrastive learning model. It transforms the "noise level" into a learning signal, guiding the model to focus more on emotion-related invariant factors and downplay emotion-irrelevant interfering factors. Under conditions of weak labeling, cross-domain processing, and varying data distribution, it effectively reduces the risks of mismatch and overfitting, improves generalization ability and transferability, while simultaneously reducing labeling and iteration costs.

[0037] Finally, this invention introduces noise-suppressed cross-attention. This suppresses noise-induced false saliency and mismatches during the modal interaction and fusion stages, strengthening the model's focus on genuine sentiment evidence. This improves the consistency and interpretability of the final prediction, reduces the risk of misjudgment, and meets industry compliance requirements for auditability and traceability. Together, these three elements form a closed loop of input cleansing, representation learning, and decision fusion, giving MoLAN+ a comprehensive advantage of accuracy, robustness, interpretability, and deployability. Attached Figure Description

[0038] Figure 1 is a schematic diagram of the unified modality-aware noise dynamic editing model;

[0039] Figure 2 Here is a detailed schematic diagram of the noise reduction submodule;

[0040] Figure 3 This is a flowchart of the method. Detailed Implementation

[0041] This invention mainly comprises three parts: a denoising submodule, noise-driven contrastive learning, and noise-suppressed cross-attention. The denoising submodule dynamically edits noise at the input encoding end, noise-driven contrastive learning converts noise levels into discriminative learning signals, and noise-suppressed cross-attention suppresses the influence of noise information during cross-modal interactions.

[0042] Given a three-modal input consisting of text, visual, and audio, the text features are represented as follows: Visual features are represented as Audio features are represented as This invention first designs a denoising submodule. Since the noise distribution differs across modalities and regions within a modality, the modal features are first segmented. Because audio features exhibit significant temporal characteristics, and visual features exhibit significant spatial characteristics, this invention uses modality-aware segmentation. Audio features are segmented in one dimension, visual features in two dimensions, and text features are not segmented. The segmentation process is illustrated using visual features as an example:

[0043]

[0044] in This represents the optimal block size. Using a factor closest to the square root as the basis for block partitioning achieves balanced segmentation, avoiding information loss or noise residue due to overly large blocks, and semantic loss due to overly small blocks. Based on the optimal block parameters, modal features are transformed into feature blocks:

[0045]

[0046] Next, the noise intensity is calculated for each feature block. This invention uses text features as the primary modality and calculates the similarity between other modality feature blocks and text features as the denoising intensity.

[0047]

[0048] By determining the denoising intensity based on the noise intensity of different blocks, dynamic noise editing is achieved. Ultimately, this invention yields clean features after denoising.

[0049] After obtaining the denoised features, this invention introduces a noise-driven contrastive learning module. Positive sample pairs are formed between the denoised modal features and their semantic counterparts, while negative sample pairs are formed between the corresponding high-noise original features, cross-domain perturbation fragments, and regions inconsistent with the text. The training objective is to minimize the distance between positive sample pairs and maximize the discriminative power between positive and negative samples in the representation space, enabling the unified modality-aware noise dynamic editing model to focus on stable factors related to emotion while downplaying interference factors unrelated to emotion. The specific loss design is as follows:

[0050]

[0051] in This represents the total learning loss compared to the previous one. Indicates visual modal contrast loss. This represents the audio modal contrast loss. This indicates the expected calculation. , Represents the denoised visual and audio modal features. , Represents the original modal features. , Indicates other sample features;

[0052] To avoid overfitting to difficult samples and highlight the supervisory contribution of clean evidence, the noise-driven contrastive learning module uses a noise intensity index λ as the driver to adaptively adjust the sample weights and temperature: the stronger the noise, the looser the constraints and the lower the weights; the weaker the noise, the stricter the constraints and the higher the weights. Thus, the encoder is continuously guided to learn a discriminative subspace that is insensitive to noise but sensitive to emotional signals, resulting in more compact representations of similar emotions across modalities and more dispersed representations of dissimilar emotions.

[0053] To further enhance noise suppression capabilities, this invention introduces noise-suppressed cross-attention during the cross-modal interaction stage. The denoising intensity is used as a constraint signal to suppress and redistribute the weights of the attention. Specifically, this invention first constructs the denoising intensity of blocks and text anchors based on cross-modal similarity and sets a threshold θ. Based on this, blocks with denoising intensities higher than θ are considered high-noise regions and aggregated according to spatial or temporal adjacency to form a noise mask matrix M, which is used to uniformly constrain the contribution of this region to the attention.

[0054]

[0055]

[0056] Subsequently, channel suppression is applied to the attention score using a mask matrix M:

[0057]

[0058] On the one hand, a negative bias is applied to the log-probability of attention at the masked location to reduce its probability of being noticed; on the other hand, amplitude gating is applied to its value vector to limit the influence of noisy regions on the output of the fused representation. By adjusting cross-attention based on the noise masking matrix, excessive suppression of effective information is effectively avoided, and smoothing terms are added between adjacent sub-blocks. Through these mechanisms, noise-suppressed cross-attention can effectively block the cascading propagation of noise during the fusion stage, stabilize cross-modal alignment, strengthen the model's focus on sentiment-related evidence, and reduce mismatches and misjudgments.

[0059] By introducing a cross-attention mechanism, information from different modalities can effectively interact and integrate, thus achieving initial fusion of multimodal features. Subsequently, the interacted representation is input into a self-attention mechanism to further model deep intramodal dependencies. Finally, information from different modalities is integrated into a unified representation for sentiment analysis tasks. For sentiment regression tasks, this invention employs L1 loss. For sentiment classification tasks, this invention uses standard cross-entropy loss to optimize the model.

[0060] This invention was comprehensively evaluated on four MSA datasets. As shown in Tables 1 and 2, MoLAN+ consistently achieved state-of-the-art performance on all datasets, surpassing existing baseline models and recent solutions based on Multimodal Large Language Models (MLLM). This superior performance demonstrates that the noise-suppressed cross-attention and noise-driven contrastive learning modules effectively enhance the model's cross-modal discrimination and semantic alignment capabilities. By highlighting sentiment-related multimodal representations during the feature extraction stage, the model can mitigate the impact of noise information and maintain stable performance under diverse conditions. These findings indicate a strong synergistic effect between the noise-suppressed cross-attention, noise-driven contrastive learning, and denoising submodules, collectively contributing to MoLAN+'s enhanced adaptability and effectiveness in MSA tasks.

[0061] This invention also compares MoLAN+ with two representative denoising MSA models, JOSFD and t-HNE. Experimental results show that the performance improvement of MoLAN+ is statistically significant. This suggests that traditional denoising strategies often over-filter multimodal signals, unintentionally removing key semantic information, thus leading to a decline in model performance. In contrast, MoLAN+ employs a dynamic noise editing mechanism, selectively suppressing irrelevant noise while preserving emotion-related features, thus achieving higher accuracy on multiple datasets.

[0062] Table 1 Experimental results of MOLAN+ on CMU-MOSI and CMU-MOSEI

[0063]

[0064] Table 2 Experimental results of MOLAN+ on CH-SIMS and IEMOCAP

[0065]

Claims

1. A unified modality-aware noise dynamic editing method for multimodal sentiment analysis, characterized in that, A unified modal sensing noise dynamic editing model is adopted, with block segmentation, noise intensity calculation, noise editing, and alignment as the main lines for noise interpretation; The unified modality-aware noise dynamic editing model includes a denoising submodule, a noise-driven contrastive learning module, and a noise suppression cross-attention module. The denoising submodule performs dynamic noise editing at the input encoding end; the noise-driven contrastive learning module enables the unified modality-aware noise dynamic editing model to learn to distinguish noise information based on positive and negative samples composed of denoised features and original features. The noise suppression cross-attention module suppresses noise information according to the denoising intensity information obtained from the denoising submodule during cross-modal interaction, based on the denoising intensity information of the corresponding region. After passing through the noise suppression cross-attention module, information from different modalities can be effectively interacted and integrated, thereby achieving the initial fusion of multimodal features; The representations after the interaction are fed into the self-attention mechanism to further model deep intra-modal dependencies; Information from different modalities is integrated into a unified representation for sentiment analysis or sentiment classification tasks.

2. The unified modality-aware noise dynamic editing method for multimodal sentiment analysis according to claim 1, characterized in that, Given text, visual, and audio inputs, features are extracted using the corresponding modal encoders; text features Visual features Audio features The three elements are denoised using a denoising submodule; different modal features are segmented based on modality perception, with audio using one-dimensional segmentation, visuals using two-dimensional segmentation, and text not segmented; based on the optimal segmentation parameters, each modal feature is transformed into a feature block: For each feature block, the denoising intensity is calculated; the text feature is used as the main modality, and the similarity between the other modality feature blocks and the text feature is calculated as the denoising intensity; denoising is performed according to the denoising intensity of different blocks, and noise is dynamically edited to obtain the clean features after denoising.

3. The unified modality-aware noise dynamic editing method for multimodal sentiment analysis according to claim 2, characterized in that, The visual feature segmentation process is as follows: ； This indicates the optimal block size for the partition; Represents the first feature dimension factor, Indicates the second feature dimension factor, Represents the set of features of the first dimension of the modality. Represents the set of features of the second dimension of the modality. Represents the first feature dimension of the modality, This represents the second feature dimension of the modality; Based on the optimal block parameters, visual features are transformed into feature blocks: ； Indicates the first dimension block number, Indicates the second dimension block number, Indicates the optimal block size in the first dimension. This represents the optimal block size in the second dimension; The similarity between the feature blocks of the visual modality and the text features is used as the denoising strength. : ； in, This indicates similarity calculation.

4. The unified modality-aware noise dynamic editing method for multimodal sentiment analysis according to claim 2, characterized in that, The noise-driven contrastive learning module forms positive sample pairs for the denoised clean features and their semantic counterparts, and forms negative sample pairs for the corresponding high-noise original features, cross-domain perturbation fragments, and regions inconsistent with the text. The training objective is to minimize the distance between positive sample pairs and maximize the discriminant difference with negative samples in the representation space, enabling the unified modality-aware noise dynamic editing model to focus on stable factors related to sentiment while downplaying interfering factors unrelated to sentiment. The specific loss design is as follows: ； in This represents the total learning loss compared to the previous one. Indicates visual modal contrast loss. This represents the audio modal contrast loss. This indicates the expected calculation. , Represents the denoised visual and audio modal features. , Represents the original modal features. , Indicates other sample features; The noise-driven contrastive learning module adaptively adjusts the weights and temperature of the samples: the stronger the denoising intensity, the looser the constraints and the lower the weights; the weaker the denoising intensity, the stricter the constraints and the higher the weights.

5. The unified modality-aware noise dynamic editing method for multimodal sentiment analysis according to claim 2, characterized in that, A noise-suppressed cross-attention module is introduced in the cross-modal interaction stage; the weights of attention are suppressed and redistributed using the denoising intensity as a constraint signal.

6. The unified modality-aware noise dynamic editing method for multimodal sentiment analysis according to claim 5, characterized in that, The noise suppression cross-attention module calculates the denoising intensity based on the similarity between the block and the text anchor point, and sets a threshold θ; blocks with a denoising intensity higher than θ are considered high-noise regions. First, the noise mask matrix M is reorganized and merged, and aggregated according to spatial or temporal adjacency to obtain the complete mask matrix. This is used to uniformly constrain the contribution of the region to attention; a negative bias is applied to the log probability of attention at the masked position to reduce its probability of being noticed; on the other hand, amplitude gating is applied to the value vector of the masked position to limit the influence of the noise region on the output of the fused representation; and the adjustment of cross attention based on the noise mask matrix effectively avoids excessive suppression of effective information.

7. The unified modality-aware noise dynamic editing method for multimodal sentiment analysis according to claim 6, characterized in that, ； ； This represents a threshold function that filters regions based on a threshold. Let R represent the visual or audio feature matrix, and R denote the matrix recombination operation; channel suppression is applied to attention scoring based on the mask matrix. ； in Indicates the modal characteristics of the query. Modal characteristics representing keys and values, This indicates cross-attention calculation. This represents the modal features after passing through the noise-suppressed cross-attention module; This represents the noise mask matrix for the query modality features.

Images