Multi-modal data inconsistency detection and cleaning method and system based on adaptive energy guidance

Abstract

The present application relates to a multi-modal data inconsistency detection and cleaning method and system based on adaptive energy guidance, belonging to the field of artificial intelligence and data mining technology. The present application proposes a multi-modal data inconsistency detection method based on adaptive energy guidance. First, a multi-expert uncertainty estimation framework is constructed to effectively mine multi-dimensional modal semantic correlation information such as prediction uncertainty, cross-modal consistency and teacher-student difference. Second, according to the multi-expert evaluation signal, an adaptive energy scoring mechanism is designed to effectively aggregate and quantify the sample modal consistency by constructing a comprehensive energy function and a weighted function based on Boltzmann distribution. Finally, a dynamic data cleaning and screening strategy based on energy score is constructed to improve the quality of the data set and the robustness of the downstream model, and better achieve the purpose of data service.

Description

Technical Field

[0001] This invention belongs to the field of data mining technology and relates to a logic-anchored multimodal processing for inconsistency detection (LAMP-Detect) and system based on adaptive energy-guided multimodal data inconsistency detection. To effectively quantify and evaluate inconsistency, an adaptive multi-expert energy scoring (AMES) architecture is proposed based on multi-expert game theory. Background Technology

[0002] In recent years, with the development of the mobile internet, accurately acquiring high-quality data from massive and noisy multimodal data has become a research hotspot. Multimodal inconsistency detection and cleaning aims to identify and correct semantic conflicts and noisy labels in the original data, constructing high-quality datasets, which are of great value in tasks such as multimodal pre-training and cross-modal retrieval. However, although multimodal data cleaning technology has made some progress, existing methods still have some shortcomings, mainly in the following aspects: First, the implicit conflict information is not fully utilized. Existing methods fail to make full use of semantic conflict clues such as irony in multimodal data. These clues are crucial for accurately identifying "image-text mismatch" samples, but existing methods often only focus on explicit object mismatches, resulting in low accuracy and recall. The second shortcoming lies in the difficulty of fine-grained reliability quantification, that is, there are rich intermediate states between samples ranging from completely consistent to completely conflicting. Existing methods struggle to effectively capture the relative quality differences between samples, often employing a "black and white" hard threshold strategy, leading to the incorrect discarding of hard negative samples. The third point is the poor generalization ability of the detection mechanism. Some existing methods rely on domain-specific prior rules or large-scale manually labeled data to improve accuracy. While these models improve accuracy to some extent, their generalization ability is poor, making them difficult to apply to large-scale data in open domains.

[0003] Currently, multimodal data cleaning technology is mainly applied in the following three areas. In social media analytics, inconsistency detection is crucial for analyzing users' true emotions and uncovering reversals in public opinion. For example, by cleaning satirical tweets (positive images paired with negative text) on social media, it's possible to more accurately analyze public sentiment, thereby determining the true direction of public opinion and supporting public opinion monitoring and crisis early warning. In e-commerce, image-text consistency detection helps uncover discrepancies between product descriptions and displayed images, analyzing whether merchants are engaging in false advertising. In multimodal content generation, high-quality data cleaning helps build well-aligned training corpora. By effectively removing semantically conflicting image-text pairs, the relevance and logic of the generated content can be guaranteed.

[0004] The limitations of existing methods lie in their reliance on shallow matching scores for text-image pairs. These methods neglect the complex semantic interactions within samples and suffer from high false positive rates when dealing with semantically ambiguous samples. Furthermore, they are constrained by limitations in their data distribution assumptions, making it difficult to effectively capture semantically plausible "long-tail" samples that, while deviating from the distribution, still possess reasonable semantics. Secondly, existing Transformer-based methods remain limited in the absence of explicit supervision signals (i.e., the lack of manually labeled inconsistencies), making it difficult to effectively learn deep logical conflict information. Summary of the Invention

[0005] To address the shortcomings of existing methods, this invention proposes an adaptive energy-guided method and system for multimodal data inconsistency detection and cleaning. This method effectively mines explicit and implicit semantic conflict associations in multimodal data by constructing an adaptive multi-expert energy scoring architecture. Simultaneously, a reliability quantification mechanism based on Boltzmann distribution is designed to adaptively learn the local and global reliability weights of samples. Furthermore, an optimization objective function combining the joint task and energy regularization is constructed to improve the detection efficiency and cleaning quality of the method.

[0006] This invention is achieved through the following technical solution: a method for detecting and cleaning multimodal data inconsistencies based on adaptive energy guidance, the steps of which are as follows: Step 1: Dataset preparation: Select a dataset as the evaluation baseline and perform standardized preprocessing on the data in the dataset. The dataset contains social media tweets with sentiment annotations.

[0007] Dataset preparation includes acquiring the raw multimodal image and text data and performing standardization preprocessing on the dataset. For example, this paper selects the widely used baseline dataset MVSA-Single to evaluate LAMP-Detect. This selection is solely for better method evaluation and has no other impact on the dataset itself. The relevant data is then passed to other modules, which ultimately obtain the results.

[0008] The specific dataset information is as follows: MVSA-Single. Proposed by Niu et al., the MVSA-Single dataset is a multimodal sentiment analysis dataset built on a social media platform. This dataset collects tweets containing images and text, labeled with sentiment tendencies, constructing a rich corpus of image-text text containing semantic conflicts. It addresses the problem that existing datasets struggle to simulate the high-noise (approximately 15%-25% label noise) and modal inconsistency (such as irony) environments of real social media. In recent years, it has been widely used in multimodal inconsistency detection and robust learning research.

[0009] Step 2: Extract high-dimensional feature representations of images and text using the multimodal feature extraction and representation module. Specifically, to address the feature heterogeneity between images and text, a pre-trained dual-tower architecture is used to establish a cross-modal semantic alignment foundation. A residual convolutional neural network ResNet-18 is used as a visual encoder to extract visual features, and a distilled bidirectional transformer DistilBERT is used as a text encoder to extract text features. Through deep convolution operations and multi-head self-attention mechanisms, visual spatial structure information and long-dependency semantics of text sequences are captured respectively, obtaining multimodal representations rich in semantic interactions.

[0010] This paper employs a Residual Neural Network (ResNet-18) and a Distilled Bidirectional Encoder Representations from Transformers (DistilBERT) as visual and text encoders, respectively. These serve as a classic computer vision backbone network and a lightweight variant of BERT. Modeling is based on deep convolutional operations and a multi-head self-attention mechanism. Large-scale pre-training and knowledge distillation strategies are used to learn multimodal contextual information rich in semantic interactions. For input image-text pairs... The characteristic representation of each mode is shown in Equation (1).

[0011] (1)

[0012] in, For image The visual feature vectors are extracted by ResNet-18; For text The text feature vector is obtained from the [CLS] tag output of DistilBERT; and These represent the visual encoder and the text encoder, respectively.

[0013] Step 3: Mining multi-dimensional uncertainty signals through an adaptive multi-expert energy scoring module to construct a comprehensive energy score. Specifically, to uncover multi-dimensional semantic inconsistency cues, an adaptive multi-expert energy scoring framework is constructed. This framework combines multi-expert game theory to establish a single-modal expert evaluation mechanism based on prediction uncertainty and a bimodal expert evaluation mechanism based on cross-modal alignment. Through a multi-dimensional uncertainty measurement function, explicit and implicit cues are transformed into sample-level reliability scores, with the expert set represented as a single-modal expert set. and cross-modal expert ensemble .

[0014] Unstructured multimodal data contains a large number of multidimensional semantic inconsistencies, and these clues are usually scattered and hidden. Effectively mining this information is crucial for improving the quality of multimodal datasets and the robustness of downstream models.

[0015] Multimodal inconsistency cues are mainly divided into explicit and implicit inconsistency cues. Explicit inconsistency cues can be directly expressed by explicit evidence, mainly including direct factual conflicts between image content and text description (such as a picture of a "cat" paired with text about a "dog"). Implicit inconsistency cues need to be obtained through external common sense or deep reasoning, mainly including irony, metaphor, and emotional polarity reversal. Among these, unimodal prediction entropy has been proven to be a fundamental and crucial uncertainty measure, effectively revealing the difficulty level of samples. However, single-dimensional entropy signals often ignore the complex interactive conflict information between different modalities. For example, the texts "smiling emoji" and "bad day" contain both positive signals at the visual level and negative signals at the text level, constituting an implicit ironic conflict. To address this, this paper constructs a unimodal expert evaluation mechanism based on prediction uncertainty and a bimodal expert evaluation mechanism based on cross-modal alignment.

[0016] In constructing the energy scoring system, to fully explore multimodal inconsistency semantic dependency clues, a multi-dimensional uncertainty measurement function was proposed based on multi-expert game theory to construct sample-level reliability scoring relationships. Furthermore, by combining deep-seated relationships such as knowledge differences between teachers and students and cross-modal semantic alignment, an adaptive multi-expert energy scoring framework was constructed, which is represented by a single-modal expert set. and cross-modal expert ensemble .by For example, it includes a Predictive Uncertainty Expert, which assesses the confidence level of the teacher model in predicting the current sample. It includes a Cross-Modal Consistency Expert to evaluate the semantic alignment between image and text modalities.

[0017] Step 3-1: Expert signal calculation and measurement function; Energy scoring, based on probability distribution and information entropy rules, constructs a quantitative link for semantic conflicts between modalities, expressing sample reliability information in numerical form. The study found that modal inconsistency in determining whether a sample contributes to noise mainly stems from the disorder of the model's predicted distribution and the distance between modalities. However, in traditional uncertainty estimation, relying solely on a single prediction entropy often leads to ambiguity, and different types of uncertainty (such as data uncertainty and model uncertainty) contain varying degrees of noise cues. Considering only a single indicator easily incorporates irrelevant and redundant information, affecting the interpretability of the method. Therefore, this paper proposes a multidimensional expert signal metric function to extract the optimal inconsistency features between samples.

[0018] In this method, a set of multidimensional uncertainty signals between image-text pairs is extracted. ,in Given the number of expert signals, such as m=4, construct the computational sequence for each signal in the signal set. For each computational sequence, the relative entropy and Shannon entropy methods from information theory are introduced to construct a serialized representation for each expert. Expert serialization is represented as ,in To predict the probability distribution of the model, For label or dual mode distribution.

[0019] The matching pattern is represented by the sequential distribution of predictions from the teacher and student models. To mine the temporal relationship features between modalities, an expert feature sequence table is constructed based on the expert role table, including: Forecasting uncertainty experts: Using teacher models to predict the entropy of distributions to measure the fundamental difficulty of a sample; Cross-modal consistency expert: Calculate the KL divergence between single-modal and multimodal predictions to measure the degree of modal semantic deviation; Prediction Entropy Expert: Calculates the entropy value of the fused prediction distribution to measure the confusion of the final decision; Teacher-student difference expert: Calculating the difference in the predicted distributions of teacher and student models measures the resistance to knowledge transfer; Based on the constructed expert signal calculation sequence and expert feature table, a weighted summation function from statistical learning methods is introduced to construct a comprehensive energy feature matching pattern, and the inner product of each expert signal and the learnable weight sequence is calculated. Using the obtained inner product, the comprehensive energy feature score of the sample is further obtained, and the energy representation containing the richest inconsistency information is selected. Through the proposed multi-expert signal metric function method, the optimal inconsistency metric at the sample level can be obtained. If modal conflict exists within a sample, its corresponding energy score is significantly increased. Specifically, as shown in formula (2).

[0020] (2)

[0021] in, For the number of expert signal categories, Learnable adaptive weight parameters For the first An uncertainty signal calculated by an expert; Step 3-2: Synthesis of integrated energy fractions and aggregation of heterogeneous signals; Using the expert signal metric function method in step 3-1, we construct sample-level multidimensional uncertainty dependency associations to mine cross-modal inconsistencies, focusing on intra-model predictions and inter-modal semantic alignment associations. Within the cross-modal semantic inconsistency association information, we focus on signal aggregation in the following three dimensions: (1) Model Confidence Correlation. Although there is noise interference in unimodal prediction, the entropy value of the prediction distribution can still characterize the inherent difficulty of the sample. By constructing this kind of correlation information, the ambiguity of model decision-making can be effectively resolved, thereby capturing the global uncertainty characteristics of prediction. If the prediction distribution tends to be uniform (high entropy), then the signal value of this dimension is made to approach the maximum entropy value; otherwise, it is made to approach 0.

[0022] (2) Modal Alignment. Modal alignment is usually defined as the consistency of semantic space. If there is irony or text-image discrepancy in the sample, this paper uses the distribution difference between modalities (KL divergence) as the criterion. If there is such semantic deviation between modalities, the signal value will increase significantly; otherwise, it will be low.

[0023] (3) Knowledge Transfer Discrepancy. By constructing and comparing the predicted distributions of the student model with those of the teacher model, information loss or cognitive discrepancies during the knowledge distillation process can be effectively captured. If such a prediction difference exists between the two models, the difference signal value is increased, indicating that the sample is in a knowledge blind spot; otherwise, it is 0.

[0024] By mining single-modal uncertainty and cross-modal inconsistency information, the multi-expert comprehensive energy score of the sample can be expressed by formula (2). For the number of expert signal categories, These are learnable, adaptive weight parameters. For the first The uncertainty signal value calculated by an expert.

[0025] Step 4: Use the energy-based sample reliability quantification and data cleaning module to convert energy into reliability weights and execute a dynamic cleaning strategy.

[0026] To effectively aggregate and learn the correlation information of energy scores, a reliability mapping and dynamic filtering mechanism is proposed. Based on the adaptive multi-expert energy scoring results and combined with the local energy features of the samples, a Boltzmann-like distribution mapping mechanism is used to fully learn the local reliability semantic information between samples; a hierarchical filter based on curriculum learning is constructed to capture the global quality distribution information of the dataset.

[0027] Step 4-1: Sample-level reliability mapping and feature enhancement; Based on the constructed comprehensive energy score information, we found that the sample energy value is closely related to the noise and conflict probability it contains. If the sample energy value is high, it indicates that the sample is likely to be a modally inconsistent sample (such as irony or mislabeling). If the sample has a low energy value, it indicates that the sample has high semantic consistency and label reliability. For samples in different energy ranges, their contribution to model training varies greatly, and simple linear truncation cannot meet the requirements for handling complex noise distributions. Therefore, combined with the good nonlinear mapping ability of the Boltzmann distribution, this paper introduces a temperature adjustment coefficient and designs a reliability quantization layer to better learn and represent local reliability features. The reliability quantization layer is shown in formula (8): (8) (9) in, This represents the sample obtained in the previous section. Reliability weight, This represents the overall energy score of the sample. Indicates the dynamic temperature coefficient; The calculation formula is as follows: The decay function, The decay rate is used; by introducing a temperature annealing mechanism, the model has a high tolerance for noise in the initial stage (high). Later, the requirements gradually became stricter (lower). ), The logical boundary of the normalization operation is represented; the weighted sample features are represented as: (10) (11) in, Represents the original cross-modal features. Let represent the weighted feature learning objective. Using the formula above, the local reliability embedding of a sample can be expressed as: (12) in, For feature splicing operations, MLP stands for Multilayer Perceptron; Step 4-2: Global distributed sensing and dynamic cleaning strategy; A three-layer hierarchical filter is constructed to aggregate local and global distribution information. In the first layer, a global statistical aggregation mechanism based on energy distribution is constructed to fully learn the global quality distribution information at the dataset level. Taking the energy distribution-dependent cleaning mechanism as an example, when determining whether a sample is "noise," the mean and variance of the energy of the entire batch are obtained to capture the sample's relative quality position in the global dataset. Meanwhile, to avoid truncation errors caused by a single threshold and reduce sensitivity to outliers, the filter employs a Gaussian distribution assumption strategy to learn global statistical features.

[0028] The global energy distribution statistics of the dataset can be expressed as: and ,in , Let these represent the mean and standard deviation of the current batch energy, respectively. The dynamic threshold, guided by distribution statistics, is defined as: (13) (14) (15) (16) in, Hyperparameters representing the degree of control over the cleaning process. Indicates an indicator function, The generated binary cleaning mask (1 indicates retention, 0 indicates cleaning) is then used to aggregate the cleaning decision vectors of all samples. (17) (18) (19) (20) (twenty one) Where Aggregate represents the global average pooling operation; This scheme achieves sample denoising by constructing a global energy distribution statistic and a dynamic cleaning mechanism. First, the energy value of the current batch is calculated using (13) and (14). mean difference with standard deviation This serves as a global quality benchmark for the dataset within this batch. Based on this, hyperparameters are introduced to control the rigor of the cleaning process. a Construct dynamic thresholds And generate a binary cleaning mask in conjunction with an indicator function. ,in This indicates that samples should be retained; otherwise, cleaning should be performed. To quantify the cleaning effect, a sample discard rate is defined. This represents the ratio of the number of discarded samples to the total number of samples. During the feature aggregation stage, the weighted features after masking are processed using the global average pooling operation (Aggregate). By fusing the data, robust global distribution features can be obtained. This guides the model to focus on highly reliable semantic information in the multi-task joint loss optimization.

[0029] In the second layer, the local reliability and global distribution features of the retained samples are aggregated to obtain robust semantic representation vectors. A curriculum learning regulation mechanism is introduced to enhance the training stability between them. (18) (19) As shown in formula (19), Indicates the training of the first Adjustment vector during the cycle.

[0030] (20) (twenty one) (twenty two) (twenty three) (twenty four) As shown in formulas (20) to (24), This represents the predicted retention probability. This represents the final sample weights after fusion. In the last layer of the filter, adaptive threshold truncation and standard normalization operations are used to obtain the final cleaned dataset. (25) (26) in, This indicates the final cleaning score. This is the final output threshold. The LayerNorm operation for each layer uses residual connections to obtain the final reliability state for all samples.

[0031] Step 4-3: Multi-task joint loss optimization module; In multimodal data inconsistency detection and cleaning tasks, the lack of large-scale, high-quality manual annotations means that treating all samples equally during training significantly reduces method efficiency and generalization performance after cleaning. Therefore, it is necessary to consider the impact of the reliability weight of each sample on the main task. The loss is expressed as: (27) Let each be defined as a reliability-weighted classification operation. Wherein The reliability weights obtained in step 3 are used to focus the model on learning modally consistent "clean" samples. The cross-modal attention distribution for each image-text pair is fed into a logistic alignment layer. If the student model and teacher model differ in their attention regions, the label is inconsistent and further corrected. The logistic distillation loss can be expressed as: (28) in This represents the KL divergence, designed to force student models to mimic the cross-modal reasoning logic of the teacher.

[0032] (29)

[0033] Meanwhile, to prevent the energy score from collapsing or exploding due to lack of constraints in the early stages of training, an energy regularization term as shown in formula (29) is introduced to ensure that only real modal conflict samples will generate high energy.

[0034] (30)

[0035] Finally, the objective function for multi-task joint optimization is calculated using formula (30) to determine the overall loss of inconsistency detection and cleaning.

[0036] An adaptive energy-guided multimodal data inconsistency detection and cleaning system includes a multimodal feature extraction and representation module, an adaptive multi-expert energy scoring module, and an energy-based sample reliability quantification and data cleaning module. Multimodal feature extraction and representation module: The original image and text data are modeled using a pre-trained dual-tower architecture. Visual features are extracted using a residual network, and text features are extracted using a distilled version of Transformer. By performing cross-modal feature fusion, a multimodal representation rich in semantic interaction is obtained. Adaptive multi-expert energy scoring module: Constructs four expert signals: prediction uncertainty, cross-modal consistency, prediction entropy, and teacher-student differences, to comprehensively evaluate the degree of inconsistency of samples in terms of semantic alignment, model confidence, and knowledge differences, and weights and synthesizes the multi-dimensional signals into a comprehensive energy score; The energy-based sample reliability quantification and data cleaning module realizes the probabilistic mapping and cleaning of sample reliability based on energy scores. It introduces a Boltzmann-like distribution function to transform the energy scores into normalized reliability weights, which are used as the basis for data screening. By setting a dynamic threshold, high-energy modal inconsistency samples are removed, and a structured dataset is output.

[0037] The beneficial effects of this invention are as follows: (1) Improved the accuracy and coverage of modal inconsistency identification: By constructing a multi-expert uncertainty estimation framework, this invention can not only identify explicit noise in image content and text description, but also effectively capture deep semantic conflicts such as irony, metaphor and emotional polarity reversal, solving the problem of insufficient utilization of implicit conflict information in existing methods.

[0038] (2) Achieved fine-grained and robust sample quality assessment: An adaptive scoring mechanism based on energy distribution dependence was introduced, changing the traditional hard threshold screening mode. By calculating the global energy mean and variance and combining it with the Gaussian distribution assumption, the system can capture the relative quality position of the sample in the dataset, effectively distinguishing high-difficulty samples from real noise and reducing truncation error.

[0039] (3) Reduced dependence on manually labeled data: This method, through adaptive energy guidance and logical distillation loss, can still effectively mine deep logical conflicts and correct them in the absence of large-scale high-quality manual annotation, thereby reducing data preparation costs and improving cleaning efficiency. Attached Figure Description

[0040] Figure 1 Example of a multimodal data inconsistency detection and cleaning process.

[0041] Figure 2 Overall architecture diagram of the LAMP-Detect model.

[0042] Figure 3 Adaptive multi-expert energy scoring module architecture diagram.

[0043] Figure 4 Reliability quantification layer.

[0044] Figure 5 It is a hierarchical aggregator.

[0045] Figure 6 This is a module diagram for the implementation method. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0047] A method for detecting and cleaning multimodal data inconsistencies based on adaptive energy guidance, comprising the following steps: Step 1: Dataset preparation: Select a dataset as the evaluation baseline and perform standardized preprocessing on the data in the dataset. The dataset contains social media tweets with sentiment annotations.

[0048] Dataset preparation includes acquiring the raw multimodal image and text data and performing standardization preprocessing on the dataset. For example, this paper selects the widely used baseline dataset MVSA-Single to evaluate LAMP-Detect. This selection is solely for better method evaluation and has no other impact on the dataset itself. The relevant data is then passed to other modules, which ultimately obtain the results.

[0049] The specific dataset information is as follows: MVSA-Single. Proposed by Niu et al., the MVSA-Single dataset is a multimodal sentiment analysis dataset built on a social media platform. This dataset collects tweets containing images and text, labeled with sentiment tendencies, constructing a rich corpus of image-text text containing semantic conflicts. It addresses the problem that existing datasets struggle to simulate the high-noise (approximately 15%-25% label noise) and modal inconsistency (such as irony) environments of real social media. In recent years, it has been widely used in multimodal inconsistency detection and robust learning research.

[0050] Step 2: Extract high-dimensional feature representations of images and text using the multimodal feature extraction and representation module. Specifically, to address the feature heterogeneity between images and text, a pre-trained dual-tower architecture is used to establish a cross-modal semantic alignment foundation. A residual convolutional neural network ResNet-18 is used as a visual encoder to extract visual features, and a distilled bidirectional transformer DistilBERT is used as a text encoder to extract text features. Through deep convolution operations and multi-head self-attention mechanisms, visual spatial structure information and long-dependency semantics of text sequences are captured respectively, obtaining multimodal representations rich in semantic interactions.

[0051] This paper employs a Residual Neural Network (ResNet-18) and a Distilled Bidirectional Encoder Representations from Transformers (DistilBERT) as visual and text encoders, respectively. These serve as a classic computer vision backbone network and a lightweight variant of BERT. Modeling is based on deep convolutional operations and a multi-head self-attention mechanism. Large-scale pre-training and knowledge distillation strategies are used to learn multimodal contextual information rich in semantic interactions. For input image-text pairs... The characteristic representation of each mode is shown in Equation (1).

[0052] (1)

[0053] in, For image The visual feature vectors are extracted by ResNet-18; For text The text feature vector is obtained from the [CLS] tag output of DistilBERT; and These represent the visual encoder and the text encoder, respectively.

[0054] Step 3: Mining multi-dimensional uncertainty signals through an adaptive multi-expert energy scoring module to construct a comprehensive energy score. Specifically, to uncover multi-dimensional semantic inconsistency cues, an adaptive multi-expert energy scoring framework is constructed. This framework combines multi-expert game theory to establish a single-modal expert evaluation mechanism based on prediction uncertainty and a bimodal expert evaluation mechanism based on cross-modal alignment. Through a multi-dimensional uncertainty measurement function, explicit and implicit cues are transformed into sample-level reliability scores, with the expert set represented as a single-modal expert set. and cross-modal expert ensemble .

[0055] Unstructured multimodal data contains a large number of multidimensional semantic inconsistencies, and these clues are usually scattered and hidden. Effectively mining this information is crucial for improving the quality of multimodal datasets and the robustness of downstream models.

[0056] Multimodal inconsistency cues are mainly divided into explicit and implicit inconsistency cues. Explicit inconsistency cues can be directly expressed by explicit evidence, mainly including direct factual conflicts between image content and text description (such as a picture of a "cat" paired with text about a "dog"). Implicit inconsistency cues need to be obtained through external common sense or deep reasoning, mainly including irony, metaphor, and emotional polarity reversal. Among these, unimodal prediction entropy has been proven to be a fundamental and crucial uncertainty measure, effectively revealing the difficulty level of samples. However, single-dimensional entropy signals often ignore the complex interactive conflict information between different modalities. For example, the texts "smiling emoji" and "bad day" contain both positive signals at the visual level and negative signals at the text level, constituting an implicit ironic conflict. To address this, this paper constructs a unimodal expert evaluation mechanism based on prediction uncertainty and a bimodal expert evaluation mechanism based on cross-modal alignment.

[0057] In constructing the energy scoring system, to fully explore multimodal inconsistency semantic dependency clues, a multi-dimensional uncertainty measurement function was proposed based on multi-expert game theory to construct sample-level reliability scoring relationships. Furthermore, by combining deep-seated relationships such as knowledge differences between teachers and students and cross-modal semantic alignment, an adaptive multi-expert energy scoring framework was constructed, which is represented by a single-modal expert set. and cross-modal expert ensemble .by For example, it includes a Predictive Uncertainty Expert, which assesses the confidence level of the teacher model in predicting the current sample. It includes a Cross-Modal Consistency Expert to evaluate the semantic alignment between image and text modalities. The overall architecture diagram of the LAMP-Detect model is shown below. Figure 2 As shown.

[0058] Step 3-1: Expert signal calculation and measurement function; Energy scoring, based on probability distribution and information entropy rules, constructs a quantitative link for semantic conflicts between modalities, expressing sample reliability information in numerical form. The study found that modal inconsistency in determining whether a sample contributes to noise mainly stems from the disorder of the model's predicted distribution and the distance between modalities. However, in traditional uncertainty estimation, relying solely on a single prediction entropy often leads to ambiguity, and different types of uncertainty (such as data uncertainty and model uncertainty) contain varying degrees of noise cues. Considering only a single indicator easily incorporates irrelevant and redundant information, affecting the interpretability of the method. Therefore, this paper proposes a multidimensional expert signal metric function to extract the optimal inconsistency features between samples.

[0059] In this method, a set of multidimensional uncertainty signals between image-text pairs is extracted. ,in Given the number of expert signals, such as m=4, construct the computational sequence for each signal in the signal set. For each computational sequence, the relative entropy and Shannon entropy methods from information theory are introduced to construct a serialized representation for each expert. Expert serialization is represented as ,in To predict the probability distribution of the model, For label or dual mode distribution.

[0060] The matching pattern is represented by the sequential distribution of predictions from the teacher and student models. To mine the temporal relationship features between modalities, an expert feature sequence table is constructed based on the expert role table, including: Forecasting uncertainty experts: Using teacher models to predict the entropy of distributions to measure the fundamental difficulty of a sample; Cross-modal consistency expert: Calculate the KL divergence between single-modal and multimodal predictions to measure the degree of modal semantic deviation; Prediction Entropy Expert: Calculates the entropy value of the fused prediction distribution to measure the confusion of the final decision; Teacher-student difference expert: Calculating the difference in the predicted distributions of teacher and student models measures the resistance to knowledge transfer; Based on the constructed expert signal calculation sequence and expert feature table, a weighted summation function from statistical learning methods is introduced to construct a comprehensive energy feature matching pattern, and the inner product of each expert signal and the learnable weight sequence is calculated. Using the obtained inner product, the comprehensive energy feature score of the sample is further obtained, and the energy representation containing the richest inconsistency information is selected. Through the proposed multi-expert signal metric function method, the optimal inconsistency metric at the sample level can be obtained. If modal conflict exists within a sample, its corresponding energy score is significantly increased. Specifically, as shown in formula (2).

[0061] (2)

[0062] in, For the number of expert signal categories, Learnable adaptive weight parameters For the first An uncertainty signal calculated by an expert; Step 3-2: Synthesis of integrated energy fractions and aggregation of heterogeneous signals; Using the expert signal metric function method in step 3-1, we construct sample-level multidimensional uncertainty dependency associations to mine cross-modal inconsistencies, focusing on intra-model predictions and inter-modal semantic alignment associations. Within the cross-modal semantic inconsistency association information, we focus on signal aggregation in the following three dimensions: (1) Model Confidence Correlation. Although there is noise interference in unimodal prediction, the entropy value of the prediction distribution can still characterize the inherent difficulty of the sample. By constructing this kind of correlation information, the ambiguity of model decision-making can be effectively resolved, thereby capturing the global uncertainty characteristics of prediction. If the prediction distribution tends to be uniform (high entropy), then the signal value of this dimension is made to approach the maximum entropy value; otherwise, it is made to approach 0.

[0063] (2) Modal Alignment. Modal alignment is usually defined as the consistency of semantic space. If there is irony or text-image discrepancy in the sample, this paper uses the distribution difference between modalities (KL divergence) as the criterion. If there is such semantic deviation between modalities, the signal value will increase significantly; otherwise, it will be low.

[0064] (3) Knowledge Transfer Discrepancy. By constructing and comparing the predicted distributions of the student model with those of the teacher model, information loss or cognitive discrepancies during the knowledge distillation process can be effectively captured. If such a prediction difference exists between the two models, the difference signal value is increased, indicating that the sample is in a knowledge blind spot; otherwise, it is 0.

[0065] By mining single-modal uncertainty and cross-modal inconsistency information, the multi-expert comprehensive energy score of the sample can be expressed by formula (2). For the number of expert signal categories, These are learnable, adaptive weight parameters. For the first The uncertainty signal value calculated by an expert. Figure 1 For example, the constructed adaptive multi-expert energy scoring architecture is as follows: Figure 3 As shown.

[0066] Step 4: Use the energy-based sample reliability quantification and data cleaning module to convert energy into reliability weights and execute a dynamic cleaning strategy.

[0067] To effectively aggregate and learn energy score correlation information, a reliability mapping and dynamic screening mechanism is proposed, such as... Figure 3 As shown. Based on the adaptive multi-expert energy scoring results and combined with the local energy features of the samples, a Boltzmann-like distribution mapping mechanism is used to fully learn the semantic information of local reliability between samples; a hierarchical filter based on curriculum learning is constructed to capture the global quality distribution information of the dataset.

[0068] Step 4-1: Sample-level reliability mapping and feature enhancement; Based on the constructed comprehensive energy score information, we found that the sample energy value is closely related to the noise and conflict probability it contains. A high sample energy value indicates that the sample is likely to be a modally inconsistent sample (such as irony or mislabeling). A low sample energy value indicates that the sample has high semantic consistency and label reliability. For samples in different energy ranges, their contribution to model training varies significantly, and simple linear truncation cannot meet the requirements for handling complex noise distributions. Therefore, combining the good nonlinear mapping capability of the Boltzmann distribution, this paper introduces a temperature adjustment coefficient and designs a reliability quantization layer to better learn and represent local reliability features. The reliability quantization layer is as follows: Figure 4 As shown. The reliability quantification layer is shown in formula (8): (8) (9) in, This represents the sample obtained in the previous section. Reliability weight, This represents the overall energy score of the sample. Indicates the dynamic temperature coefficient; The calculation formula is as follows: The decay function, The decay rate is used; by introducing a temperature annealing mechanism, the model has a high tolerance for noise in the initial stage (high). Later, the requirements gradually became stricter (lower). ), The logical boundary of the normalization operation is represented; the weighted sample features are represented as: (10) (11) in, Represents the original cross-modal features. Let represent the weighted feature learning objective. Using the formula above, the local reliability embedding of a sample can be expressed as: (12) in, For feature splicing operations, MLP stands for Multilayer Perceptron; Step 4-2: Global distributed sensing and dynamic cleaning strategy; Build as Figure 5 The three-layer hierarchical filter shown aggregates local and global distribution information. In the first layer, a global statistical aggregation mechanism based on energy distribution is constructed to fully learn the global quality distribution information at the dataset level. Taking the energy distribution-dependent cleaning mechanism as an example, when determining whether a sample is "noise," the mean and variance of the energy of the entire batch are obtained to capture the sample's relative quality position in the global dataset. Meanwhile, to avoid truncation errors caused by a single threshold and reduce sensitivity to outliers, the filter employs a Gaussian distribution assumption strategy to learn global statistical features.

[0069] The global energy distribution statistics of the dataset can be expressed as: and ,in , Let these represent the mean and standard deviation of the current batch energy, respectively. The dynamic threshold, guided by distribution statistics, is defined as: (13) (14) (15) (16) in, Hyperparameters representing the degree of control over the cleaning process. Indicates an indicator function, The generated binary cleaning mask (1 indicates retention, 0 indicates cleaning) is then used to aggregate the cleaning decision vectors of all samples. (17) (18) (19) (20) (twenty one) Where Aggregate represents the global average pooling operation; This scheme achieves sample denoising by constructing a global energy distribution statistic and a dynamic cleaning mechanism. First, the energy value of the current batch is calculated using (13) and (14). mean difference with standard deviation This serves as a global quality benchmark for the dataset within this batch. Based on this, hyperparameters are introduced to control the rigor of the cleaning process. a Construct dynamic thresholds And generate a binary cleaning mask in conjunction with an indicator function. ,in This indicates that samples should be retained; otherwise, cleaning should be performed. To quantify the cleaning effect, a sample discard rate is defined. This represents the ratio of the number of discarded samples to the total number of samples. During the feature aggregation stage, the weighted features after masking are processed using the global average pooling operation (Aggregate). By fusing the data, robust global distribution features can be obtained. This guides the model to focus on highly reliable semantic information in the multi-task joint loss optimization.

[0070] In the second layer, the local reliability and global distribution features of the retained samples are aggregated to obtain robust semantic representation vectors. A curriculum learning regulation mechanism is introduced to enhance the training stability between them. (18) (19) As shown in formula (19), Indicates the training of the first Adjustment vector during the cycle.

[0071] (20) (twenty one) (twenty two) (twenty three) (twenty four) As shown in formulas (20) to (24), This represents the predicted retention probability. This represents the final sample weights after fusion. In the last layer of the filter, adaptive threshold truncation and standard normalization operations are used to obtain the final cleaned dataset. (25) (26) in, This indicates the final cleaning score. This is the final output threshold. The LayerNorm operation for each layer uses residual connections to obtain the final reliability state for all samples.

[0072] Step 4-3: Multi-task joint loss optimization module; In multimodal data inconsistency detection and cleaning tasks, the lack of large-scale, high-quality manual annotations means that treating all samples equally during training significantly reduces method efficiency and generalization performance after cleaning. Therefore, it is necessary to consider the impact of the reliability weight of each sample on the main task. The loss is expressed as: (27) Let each be defined as a reliability-weighted classification operation. Wherein The reliability weights obtained in step 3 are used to focus the model on learning modally consistent "clean" samples. The cross-modal attention distribution for each image-text pair is fed into a logistic alignment layer. If the student model and teacher model differ in their attention regions, the label is inconsistent and further corrected. The logistic distillation loss can be expressed as: (28) in This represents the KL divergence, designed to force student models to mimic the cross-modal reasoning logic of the teacher.

[0073] (29)

[0074] Meanwhile, to prevent the energy score from collapsing or exploding due to lack of constraints in the early stages of training, an energy regularization term as shown in formula (29) is introduced to ensure that only real modal conflict samples will generate high energy.

[0075] (30)

[0076] Finally, the objective function for multi-task joint optimization is calculated using formula (30) to determine the overall loss of inconsistency detection and cleaning.

[0077] An adaptive energy-guided multimodal data inconsistency detection and cleaning system, such as Figure 6 As shown, it includes a multimodal feature extraction and representation module, an adaptive multi-expert energy scoring module, and an energy-based sample reliability quantification and data cleaning module; Multimodal feature extraction and representation module: The original image and text data are modeled using a pre-trained dual-tower architecture. Visual features are extracted using a residual network, and text features are extracted using a distilled version of Transformer. By performing cross-modal feature fusion, a multimodal representation rich in semantic interaction is obtained. Adaptive multi-expert energy scoring module: Constructs four expert signals: prediction uncertainty, cross-modal consistency, prediction entropy, and teacher-student differences, to comprehensively evaluate the degree of inconsistency of samples in terms of semantic alignment, model confidence, and knowledge differences, and weights and synthesizes the multi-dimensional signals into a comprehensive energy score; The energy-based sample reliability quantification and data cleaning module realizes the probabilistic mapping and cleaning of sample reliability based on energy scores. It introduces a Boltzmann-like distribution function to transform the energy scores into normalized reliability weights, which are used as the basis for data screening. By setting a dynamic threshold, high-energy modal inconsistency samples are removed, and a structured dataset is output.

Claims

1. A method for detecting and cleaning multimodal data inconsistencies based on adaptive energy guidance, characterized in that: The steps are as follows: Step 1: Dataset preparation: Select a dataset as the evaluation baseline and perform standardized preprocessing on the data in the dataset. The dataset contains social media tweets with sentiment annotations. Step 2: Extract high-dimensional feature representations of images and text using the multimodal feature extraction and representation module; Step 3: Mine multi-dimensional uncertainty signals through the adaptive multi-expert energy scoring module and construct a comprehensive energy score; Step 4: Use the energy-based sample reliability quantification and data cleaning module to convert energy into reliability weights and execute a dynamic cleaning strategy.

2. The method for detecting and cleaning multimodal data inconsistencies based on adaptive energy guidance according to claim 1, characterized in that, In step 1, the publicly available MVSA-Single dataset is selected as the evaluation baseline.

3. The method for detecting and cleaning multimodal data inconsistencies based on adaptive energy guidance according to claim 1, characterized in that, In step 2, to address the feature heterogeneity between images and text, a pre-trained dual-tower architecture is used to establish a cross-modal semantic alignment foundation. A residual convolutional neural network ResNet-18 is used as a visual encoder to extract visual features, and a distilled bidirectional transformer DistilBERT is used as a text encoder to extract text features. Through deep convolution operations and multi-head self-attention mechanisms, visual spatial structure information and long-dependency semantics of text sequences are captured respectively, thereby obtaining multimodal representations rich in semantic interactions.

4. The method for detecting and cleaning multimodal data inconsistencies based on adaptive energy guidance according to claim 3, characterized in that, In step 2, based on deep convolutional operations and multi-head self-attention mechanisms, a large-scale pre-training and knowledge distillation strategy is employed to learn multimodal contextual information rich in semantic interactions. For the input image-text pair... The characteristic representation of each mode is shown in Equation (1). (1) in, For image The visual feature vectors are extracted by ResNet-18; For text The text feature vector is obtained from the [CLS] tag output of DistilBERT; and These represent the visual encoder and the text encoder, respectively.

5. The method for detecting and cleaning multimodal data inconsistencies based on adaptive energy guidance according to claim 1, characterized in that, In step 3, to uncover multidimensional semantic inconsistency cues, an adaptive multi-expert energy scoring framework is constructed. This framework combines multi-expert game theory to establish a single-modal expert evaluation mechanism based on prediction uncertainty and a bimodal expert evaluation mechanism based on cross-modal alignment. Through a multidimensional uncertainty measurement function, explicit and implicit cues are transformed into sample-level reliability scores, and the expert set is represented as a single-modal expert set. and cross-modal expert ensemble .

6. The method for detecting and cleaning multimodal data inconsistencies based on adaptive energy guidance according to claim 5, characterized in that, Step 3 includes the following steps: Step 3-1: Expert signal calculation and measurement function; Extracting the set of multidimensional uncertainty signals between image-text pairs ,in To determine the number of expert signals, construct a computational sequence for each signal in the signal set. For each computation sequence, the relative entropy and Shannon entropy methods from information theory are introduced to construct a serialized representation for each expert. Expert serialization is represented as ,in To predict the probability distribution of the model, For label or dual mode distribution; The matching pattern is represented by the sequential distribution of predictions from the teacher and student models. To mine the temporal relationship features between modalities, an expert feature sequence table is constructed based on the expert role table, including: Forecasting uncertainty experts: Using teacher models to predict the entropy of distributions to measure the fundamental difficulty of a sample; Cross-modal consistency expert: Calculate the KL divergence between single-modal and multimodal predictions to measure the degree of modal semantic deviation; Prediction Entropy Expert: Calculates the entropy value of the fused prediction distribution to measure the confusion of the final decision; Teacher-student difference expert: Calculating the difference in the predicted distributions of teacher and student models measures the resistance to knowledge transfer; Based on the constructed expert signal calculation sequence and expert feature sequence table, a weighted summation function from the statistical learning method is introduced to construct a comprehensive energy feature matching mode, and the inner product of each expert signal and the learnable weight sequence is calculated; through the obtained inner product, the comprehensive energy feature score of the sample is obtained, and the energy representation containing the richest inconsistency information is selected; through the proposed multi-expert signal metric function method, the optimal inconsistency metric at the sample level is obtained. If there is modal conflict within the sample, its corresponding energy score is significantly increased, as shown in formula (2). (2) in, For the number of expert signal categories, Learnable adaptive weight parameters For the first An uncertainty signal calculated by an expert; Step 3-2: Synthesis of integrated energy fractions and aggregation of heterogeneous signals; Using the expert signal metric function method in step 3-1, we construct sample-level multidimensional uncertainty dependency associations to mine cross-modal inconsistencies, focusing on intra-model predictions and inter-modal semantic alignment associations. Within the cross-modal semantic inconsistency association information, we focus on signal aggregation in the following three dimensions: Model confidence correlation: By constructing this type of correlation information, the ambiguity of model decision-making is resolved, and the global uncertainty characteristics of prediction are captured. This relates to the predicted probability distribution output by the model. Information entropy approaches the theoretical maximum value When the predicted distribution tends to be uniform, we determine that the predicted distribution tends to be uniform. If the predicted distribution tends to be uniform, we let the signal value of that dimension approach the maximum entropy value; otherwise, we let it approach 0. Modal alignment correlation: Modal alignment is usually defined as semantic space consistency. If the sample contains irony or text-image discrepancies, this paper uses the distributional differences between modalities as the criterion. If such semantic deviations exist between modalities, i.e., when the deviation exceeds the standard deviation, the signal value of this type is set... Increase the value if the value is high, otherwise it is low; Knowledge transfer difference association: By constructing an association between the predicted distribution of the student model and the predicted distribution of the teacher model in sequence, information loss or cognitive divergence in the knowledge distillation process is captured. If there is such a prediction difference between the two models, the difference signal value is increased, indicating that the sample is in the knowledge blind spot, otherwise it is 0. By mining single-modal uncertainty and cross-modal inconsistency information, the multi-expert comprehensive energy score of the sample is given by formula (2).

7. The method for detecting and cleaning multimodal data inconsistencies based on adaptive energy guidance according to claim 1, characterized in that, Step 4 includes the following steps: Step 4-1: Sample-level reliability mapping and feature enhancement; Combining the good nonlinear mapping capability of the Boltzmann distribution, a temperature adjustment coefficient is introduced to design a reliability quantification layer, as shown in formula (8): (8) (9) in, This represents the sample obtained in the previous section. Reliability weight, This represents the overall energy score of the sample. Indicates the dynamic temperature coefficient; The calculation formula is as follows: The decay function, The decay rate is used; by introducing a temperature annealing mechanism, the model has a high tolerance for noise in the initial stage, and gradually becomes more stringent in the later stage. The logical boundary of the normalization operation is represented; the weighted sample features are represented as: (10) (11) in, Represents the original cross-modal features. Let represent the weighted feature learning objective. Using the formula above, the local reliability embedding of a sample can be expressed as: (12) in, For feature splicing operations, MLP stands for Multilayer Perceptron; Step 4-2: Global distributed sensing and dynamic cleaning strategy; A three-layer hierarchical filter is constructed to aggregate local and global distribution information. In the first layer, a global statistical aggregation mechanism based on energy distribution is constructed to fully learn the global quality distribution information at the dataset level. By calculating the mean and variance of the energy values ​​of samples within the current processing batch, a reference benchmark representing the global quality distribution of the dataset is constructed. The mean and variance are modeled using a probability density function feature extraction strategy to determine the relative distribution probability of samples in the feature space. Based on the probability distribution, non-hard-truncation sample screening is implemented, and dynamic identification and removal of noisy samples are achieved through statistical smoothing of outliers. When determining whether a sample is "noise," the energy mean and variance of the entire batch are obtained to capture the sample's relative quality position in the global dataset. Simultaneously, to avoid truncation errors caused by a single threshold and reduce sensitivity to outliers, the filter employs a Gaussian distribution assumption strategy to learn global statistical features. The global energy distribution statistics of the dataset can be expressed as: and ,in , Let these represent the mean and standard deviation of the current batch energy, respectively. The dynamic threshold, guided by distribution statistics, is defined as: (13) (14) (15) (16) in, Hyperparameters representing the degree of strictness in controlling the cleaning process. Indicates an indicator function, The generated binary cleaning mask is 1 for keeping and 0 for cleaning. The cleaning decision vectors of all samples are aggregated. (17) (18) (19) (20) (21) Where Aggregate represents the global average pooling operation; Sample denoising is achieved by constructing a global energy distribution statistic and a dynamic cleaning mechanism; firstly, the energy value of the current batch is calculated using (13) and (14). mean difference with standard deviation To characterize the global quality benchmark of the dataset in this batch; based on this, hyperparameters are introduced to control the rigor of the cleaning process. a Construct dynamic thresholds And generate a binary cleaning mask in conjunction with an indicator function. ,in To indicate whether to retain a sample, or to perform cleaning; to quantify the cleaning effect, a sample discard rate is defined. The ratio of the number of eliminated samples to the total number of samples is used in the feature aggregation stage. This is achieved through a global average pooling operation (Aggregate) on the weighted features after masking. By fusing the data, robust global distribution features can be obtained. This guides the model to focus on the semantic information of reliability in the joint loss optimization of multiple tasks; In the second layer, the local reliability and global distribution features of the preserved samples are aggregated to obtain robust semantic representation vectors. (18) (19) As shown in formula (19), Indicates the training of the first Adjustment vector during the cycle. (20) (21) (22) (23) (24) As shown in formulas (20) to (24), This represents the predicted retention probability. This represents the final sample weights after fusion. In the last layer of the filter, adaptive threshold truncation and standard normalization operations are used to obtain the final cleaned dataset. (25) (26) in, This indicates the final cleaning score. To obtain the final reliability state of all samples, the LayerNorm operation of each layer uses residual connections to achieve the final output threshold. Step 4-3: Multi-task joint loss optimization module; In multimodal data inconsistency detection and cleaning tasks, the lack of large-scale, high-quality manual annotations means that treating all samples equally during training significantly reduces method efficiency and generalization performance after cleaning. Therefore, it is necessary to consider the impact of the reliability weight of each sample on the main task, and the loss is expressed as: (27) Let each be defined as a reliability-weighted classification operation, where The reliability weights obtained in step 3 are used to focus the model on learning modally consistent "clean" samples. The cross-modal attention distribution of each image-text pair is fed into a logical alignment layer. If the student model and the teacher model differ in their attention regions, the label is inconsistent and further corrected. The logistic distillation loss is expressed as: (28) in This represents the KL divergence, designed to force student models to mimic the cross-modal reasoning logic of the teacher. (29) Meanwhile, to prevent the energy score from collapsing or exploding due to lack of constraints in the early stage of training, the energy regularization term shown in formula (29) is introduced to ensure that only real modal conflict samples will generate high energy. (30) Finally, the objective function for multi-task joint optimization is calculated using formula (30) to determine the overall loss of inconsistency detection and cleaning.

8. The system used in the adaptive energy-guided multimodal data inconsistency detection and cleaning method according to any one of claims 1-7, characterized in that, It includes a multimodal feature extraction and representation module, an adaptive multi-expert energy scoring module, and an energy-based sample reliability quantification and data cleaning module; Multimodal feature extraction and representation module: The original image and text data are modeled using a pre-trained dual-tower architecture. Visual features are extracted using a residual network, and text features are extracted using a distilled version of Transformer. By performing cross-modal feature fusion, a multimodal representation rich in semantic interaction is obtained. Adaptive multi-expert energy scoring module: Constructs four expert signals: prediction uncertainty, cross-modal consistency, prediction entropy, and teacher-student differences, to comprehensively evaluate the degree of inconsistency of samples in terms of semantic alignment, model confidence, and knowledge differences, and weights and synthesizes the multi-dimensional signals into a comprehensive energy score; The energy-based sample reliability quantification and data cleaning module realizes the probabilistic mapping and cleaning of sample reliability based on energy scores. It introduces a Boltzmann-like distribution function to transform the energy scores into normalized reliability weights, which are used as the basis for data screening. By setting a dynamic threshold, high-energy modal inconsistency samples are removed, and a structured dataset is output.

Images