Adversarial short message dichotomy method based on deep semantic understanding and multi-modal gating

Abstract

The invention discloses an adversarial short message dichotomy method based on deep semantic comprehension and multi-mode gating. The method comprises the following steps: acquiring a short message to be identified; the method comprises the following steps: extracting a semantic embedding feature vector from a short message to be identified through a pre-trained chine-roberta-wwm-ext-large model encoder, and extracting a semantic embedding feature vector from the short message to be identified through a pre-trained chine-roberta-wwm-ext model encoder; extracting pinyin embedding feature vectors from the pinyin sequence of the short message to be recognized through a pre-trained pinyin denoising auto-encoder; extracting a font embedding feature vector from the font structure of the short message to be identified through a pre-trained font denoising auto-encoder; performing weighted fusion on the semantic embedding feature vector, the pinyin embedding feature vector and the font embedding feature vector through a pre-trained gating fusion module to obtain weighted fusion features; and using a pre-trained short message classifier to classify the short messages to be identified according to the weighted fusion features. Through multi-modal fusion and an effective attention mechanism, high-precision and interpretable classification of resistant short messages is realized, and reliable technical support is provided for real-time detection of spam short messages in an actual scene.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, specifically to an adversarial SMS binary classification method based on deep semantic understanding and multimodal gating. Background Technology

[0002] With the widespread adoption of mobile communication technology, SMS, as a peer-to-peer communication medium, not only fulfills normal functions such as verification codes, life notifications, and financial services, but is also widely used for the spread of spam messages, including commercial marketing, fraud, and misleading information. According to industry statistics, the global daily volume of spam SMS messages exceeded 1 billion in 2025. Anti-spam SMS messages, through methods such as homophonic substitution (e.g., replacing "Weixin" with "WeChat"), similar-looking character disguise (e.g., replacing "loan" with "loan"), and inserting interfering words, significantly reduce the performance of traditional detection models based on static keyword matching, posing a serious threat to users' financial security and communication order. Existing SMS classification methods have three major limitations: bag-of-words models (such as TF-IDF+XGBoost) ignore word order and context, making them susceptible to adversarial perturbations; traditional deep learning models (such as CNN / RNN) suffer from insufficient long-distance dependency capture or gradient vanishing problems, and their attention mechanisms are prone to focusing on false features; and general pre-trained models (such as BERT) lack attention guidance for "safe classification" tasks during fine-tuning, have poor transparency in the decision-making process, and are unable to cope with the loss of semantic features caused by information desensitization (such as replacing sensitive information with [MASK]).

[0003] In contrast, this invention proposes a method based on deep semantic understanding and multimodal gating. By introducing feature alignment enhanced by contrastive learning, dynamic gating fusion network, meta-learning fast adaptation mechanism, and multi-loss collaborative training, it achieves adaptive fusion of multimodal features and interpretability of decisions, thereby improving classification accuracy, flexibility, fast iteration, and adversarial robustness. Summary of the Invention

[0004] To address the aforementioned shortcomings of existing technologies, the adversarial SMS binary classification method based on deep semantic understanding and multimodal gating provided by this invention solves the problems of existing technologies being unable to effectively cope with complex adversarial attacks that rely on homophones, similar-looking characters, etc., having long update cycles, high computational costs, and being unable to guarantee high classification accuracy, while also taking into account the interpretability of the model's decisions and the semantic consistency with adversarial samples.

[0005] To achieve the aforementioned objectives, this invention provides an adversarial SMS binary classification method based on deep semantic understanding and multimodal gating, comprising: Get the SMS message to be identified; Semantic embedding feature vectors are extracted from the SMS messages to be identified using a pre-trained chinese-roberta-wwm-ext-large model encoder. Pinyin embedding feature vectors are extracted from the pinyin sequence of the SMS message to be identified using a pre-trained pinyin denoising autoencoder. The glyph embedding feature vector is extracted from the glyph structure of the SMS message to be recognized by a pre-trained glyph denoising autoencoder. The semantic embedding feature vector, the pinyin embedding feature vector, and the character shape embedding feature vector are weighted and fused through a pre-trained gated fusion module to obtain weighted fused features. A pre-trained SMS classifier is used to classify the SMS messages to be identified based on weighted fusion features.

[0006] The beneficial effects of this invention are as follows: 1. By introducing a multimodal feature alignment method based on contrastive learning, the semantic gap between pinyin, character shape, and semantic features is effectively bridged, generating high-quality, highly consistent fused feature representations. Combined with the sample adaptation capability of the dynamic gating fusion network, the model can accurately cope with diverse adversarial attacks such as homophonic substitution and confusion of similar-looking characters. Compared with simplified schemes or static fusion schemes that only use semantic features, this invention exhibits higher classification accuracy and stability when facing complex adversarial scenarios.

[0007] 2. By utilizing rich meta-tasks generated by a large language model for training, model parameters can be quickly adjusted with only a small number of samples when facing novel and unseen adversarial modes, achieving efficient adaptation. This overcomes the bottlenecks of traditional methods, which rely on a large number of real labeled samples, have poor generalization, and are unable to cope with new threats. The cost of sample collection and model update can be reduced by several times.

[0008] 3. By employing a multi-loss collaborative optimization strategy, while optimizing the classification loss, word-level importance weights are explicitly learned through weighted prediction loss, providing a clear basis for the model's classification decisions. This allows users to understand which keywords or features the model bases its judgments on, significantly improving the model's credibility and transparency in high-risk applications such as fraud detection and security risk control, and avoiding the "black box" decision-making problem caused by single-loss optimization.

[0009] 4. By employing a selective parameter freezing and incremental unfreezing strategy, incremental model updates can be performed without retraining from scratch; only a few top-level parameters need to be fine-tuned to incorporate new knowledge. This significantly reduces computational resource consumption and model update time, enabling the system to continuously evolve at a lower cost and maintain its effectiveness against dynamically changing threats over the long term.

[0010] 5. This invention organically integrates data augmentation, feature extraction and alignment, meta-learning training, and dynamic evolution through a complete end-to-end architecture. This design achieves an optimal balance between model classification performance, robustness, interpretability, and operational efficiency, forming a powerful and practical overall solution for adversarial SMS defense. Attached Figure Description

[0011] Figure 1 The flowchart of the adversarial SMS binary classification method based on deep semantic understanding and multimodal gating provided in the embodiment is shown below. Figure 2 The flowcharts for the comparative learning and training of each encoder provided in the embodiments; Figure 3 A flowchart illustrating the weighted fusion of features from various modalities; Figure 4 This is a flowchart illustrating the two-layer optimization of the multimodal SMS classification model using the MAML framework in this embodiment. Detailed Implementation

[0012] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0013] like Figure 1 As shown, in one embodiment of the present invention, an adversarial SMS binary classification method based on deep semantic understanding and multimodal gating (adversarial SMS refers to spam SMS that evades detection through camouflage techniques such as homophones and similar-looking characters) includes the following steps: S1. Obtain the SMS message to be identified.

[0014] S2. Extract semantic embedding feature vectors from the SMS messages to be identified using a pre-trained chinese-roberta-wwm-ext-large model encoder.

[0015] The specific expression is:

[0016] in, For semantic embedding feature vectors, The i-th character of the input text. Let be the feature space vector of dimension d (set to 768 dimensions). This indicates the corresponding processing procedure of the encoder for the chinese-roberta-wwm-ext-large model.

[0017] S3. Extract the pinyin embedding feature vector from the pinyin sequence of the SMS to be identified using a pre-trained pinyin denoising autoencoder.

[0018] First, the SMS text is mapped using a pre-defined 397-dimensional pinyin table, converting the text into a pinyin sequence. Then, an embedding layer converts the pinyin sequence into a 32-dimensional dense vector, which is then input into a subsequent one-way gated recurrent unit (GRU) layer for processing.

[0019] The specific expression is:

[0020]

[0021] in, Let be the pinyin vector of the i-th character in the SMS text. This represents the embedding layer of the Pinyin encoder. For pinyin embedding feature vectors, GRU represents GRU gated recurrent network.

[0022] S4. Extract glyph embedding feature vectors from the glyph structure of the SMS message to be recognized using a pre-trained glyph denoising autoencoder.

[0023] The strokes of characters in the text message are mapped using a vocabulary of size 31, and the structural information of the characters is mapped using a vocabulary of size 9. The strokes and structural information are projected into a 16-dimensional embedding space, generating stroke embedding vectors and structural embedding vectors, respectively. These two 16-dimensional vectors are then concatenated into a 32-dimensional joint glyph representation. This representation is used as input to a one-way gated recurrent unit (GRU, where the hidden state dimension of this GRU layer is also 768) for temporal modeling, and finally, the glyph features of dimension 768 are output.

[0024] The specific expression is:

[0025]

[0026] in, Let be the glyph vector of the i-th character in the SMS text. This is the feature vector embedded for the pinyin. StrokeCount is the stroke encoder (which embeds the pinyin into a feature vector). (Converted to 16-dimensional stroke vectors), StructureCount is the structure encoder (which converts...) (Transformed into a 16-dimensional structure vector).

[0027] This invention trains three encoders for extracting different features through contrastive learning. By constructing positive sample pairs (semantic, pinyin, and glyphic) and negative sample pairs between different samples, it aligns representations in the latent space before feature fusion, forcibly narrowing the distance between multimodal features of the same sample. This significantly improves the quality and robustness of fused features, overcoming the feature noise problem caused by simple splicing or static weighting. Figure 2 As shown, the specific method is as follows: Obtain training samples; The training samples are input into the chinese-roberta-wwm-ext-large model encoder, the pinyin denoising autoencoder, and the character shape denoising autoencoder, respectively, to obtain the corresponding three output features; The triplet loss function is calculated based on the three features of the output. This loss function constructs a (anchor sample, positive sample, negative sample) triplet (where the anchor sample is a random SMS text, the positive sample is an SMS text within the anchor sample with changes to the pinyin / characters but no change to the content, and the negative sample is an SMS text with changed content). The function constrains the representation space learned by the model to satisfy the following conditions: the feature distance between the anchor point and the positive sample should be significantly smaller than the feature distance between the anchor point and the negative sample, while maintaining a predefined safety margin. Its expression is:

[0028] in, The triplet loss value, To anchor the sample, As a positive sample, For negative samples, The distance between two samples is Euclidean distance in the embedding space, and margin represents the minimum distance difference between positive and negative sample pairs. The parameters of the chinese-roberta-wwm-ext-large model encoder, the pinyin denoising autoencoder, and the character denoising autoencoder are updated based on the triplet loss function until the triplet loss function stabilizes.

[0029] S5. The semantic embedding feature vector, the pinyin embedding feature vector, and the character shape embedding feature vector are weighted and fused through the pre-trained gating fusion module to obtain the weighted fused feature.

[0030] like Figure 3 As shown, the specific method is as follows: The three 768-dimensional feature sequences—semantic embedding feature vector, pinyin embedding feature vector, and character shape embedding feature vector—are summed element-wise to obtain preliminary fused features.

[0031] The pre-fused features are subjected to deep context encoding by a two-layer transformer encoder, and global dependency modeling is performed on the pre-fused features. The encoder outputs context-aware feature encoding through its built-in multi-head self-attention mechanism and feedforward neural network (attention head is 12, feedforward neural network dimension is 3072, hidden layer dimension is 768).

[0032] Context-aware feature encodings are input into three identically structured gating networks to extract weights for semantic embedding feature vectors, pinyin embedding feature vectors, and glyph embedding feature vectors, respectively. Each gating network is a two-layer feedforward neural network: the first layer performs a linear projection from 768 dimensions to 384 dimensions using the ReLU activation function; the second layer performs a linear projection from 384 dimensions to 1 dimension using the Sigmoid activation function, ultimately outputting a scalar weight value between 0 and 1, representing the importance of the corresponding feature source. Finally, the original deep semantic features, pinyin features, and glyph features are multiplied by their respective calculated gating weights and then summed in a weighted manner to generate the final adaptive fusion feature.

[0033] Its expression is:

[0034] in, For weighted fusion features, The weights of the semantic embedding feature vectors, For semantic embedding feature vectors, The weights of the pinyin embedding feature vector, For pinyin embedding feature vectors, The weights of the glyph embedding feature vectors, This is the feature vector for character embedding.

[0035] S6. Use a pre-trained SMS classifier to classify the SMS messages to be identified based on weighted fusion features.

[0036] The multimodal SMS classification model consists of a chinese-roberta-wwm-ext-large model encoder, a pinyin denoising autoencoder, a character shape denoising autoencoder, a gating fusion module, and an SMS classifier. This invention uses BERT as the base model, adding a classification head and a weight prediction head to it to construct an SMS classifier for SMS tag classification and word-level weight prediction.

[0037] The training process for the multimodal SMS classification model is as follows: Obtain SMS samples; By leveraging the DeepSeek-V3.1 large language model to augment data from SMS samples to be identified, adversarial examples including metadata are generated. These examples include perturbation variants at three levels: word, phrase, and sentence, simulating diverse attack patterns in real-world adversarial scenarios. The large language model automatically outputs structured data, including adversarial text, word importance weights, and adversarial pattern labels, providing a high-quality multi-task dataset for subsequent meta-learning training.

[0038] The adversarial examples are input into a multimodal SMS classification model for classification, and the classification results are obtained. Based on the classification results, the multimodal SMS classification model is optimized in two layers using the MAML framework to obtain a pre-trained multimodal SMS classification model.

[0039] like Figure 4 As shown, the MAML framework performs two-layer optimization on the multimodal SMS classification model, including outer loop optimization and inner loop optimization. Specifically, the inner loop optimization is as follows: Sample a training batch of tasks, where each task corresponds to a specific adversarial mode (e.g., Pinyin attack only task). The composite loss function corresponding to each sampled task is calculated using the meta-initial parameters of the current multimodal SMS classification model; The parameters of the multimodal SMS classification model are optimized by optimizing the composite loss function until the composite loss function is stable, thus obtaining task-specific parameters suitable for each adversarial mode. The composite loss function includes: a classification loss based on cross-entropy to ensure basic classification accuracy; a weight prediction loss based on mean squared error to supervise the interpretability of word-level importance prediction; and an adversarial consistency loss based on cosine similarity to constrain the semantic consistency between adversarial examples and original samples. A dynamic weighting algorithm (based on adjusting the loss variance ratio) balances the three objectives, achieving a synergistic improvement in model accuracy and robustness. This strategy ensures that the model pursues high classification accuracy while also possessing decision interpretability (through word-level weights) and semantic consistency with adversarial perturbations, avoiding the one-sidedness of model capabilities caused by a single loss objective.

[0040] The expression for the composite loss function is:

[0041] in, This is the composite loss value. The weights for the classification loss, For classifying losses, The weights are used to predict the loss. For weighted prediction loss, Weights to counteract consistency loss, To combat the loss of consistency.

[0042] The optimization method for the composite loss function is as follows: Based on the loss variance adjustment ratio, the weights of each loss function are adjusted using a dynamic weighting algorithm. The specific optimization formula is as follows:

[0043] in, The weights are dynamically adjusted. k The value is c or m or a ;exp represents an exponential function with the natural constant as its base; Indicates the first k The variance of the loss over the most recent training steps, This refers to the hyperparameters used to adjust sensitivity.

[0044] The outer loop is specifically as follows: The composite loss function for each task is calculated using task-specific parameters applicable to each adversarial mode; By optimizing the composite loss function and performing gradient descent optimization on the initial parameters of the SMS classifier, a multimodal SMS classification model with optimized parameters is obtained.

[0045] By rapidly learning specific adversarial task patterns in the inner loop and optimizing the model initialization parameters with strong generalization ability in the outer loop, the system can quickly adapt to adversarial patterns that were not seen during training with only a small number of samples, overcoming the shortcomings of traditional methods that rely on a large number of real adversarial samples and have poor generalization ability.

[0046] During the training phase of the multimodal SMS classification model, a progressive parameter unfreezing strategy is employed for fine-tuning. Specifically, model parameters are unfrozen and trained in stages according to the model's hierarchical structure. The multimodal SMS classification model is continuously trained, and after each training iteration, the value of newly generated adversarial examples is automatically evaluated using the trained model. Samples with value exceeding a set threshold are selected to expand the training set for a new round of training. This not only dynamically ensures the high quality and difficulty of the training set but also forms a self-evolving closed loop with the aforementioned process: improvements in model performance, in turn, guide the generation of more challenging samples in the next round. When new adversarial examples are added, the system only needs to fine-tune the top-level network (gated fusion module and classification head), significantly reducing computational overhead. This mechanism enables the system to continuously evolve and quickly adapt to new adversarial attack patterns.

[0047] In a feasible embodiment of the present invention, assume that the original harmful sample is "Quickly transfer money to this account, otherwise there will be a shortage of 10 million!" For the adversarial test, we first generate three perturbed variants through intelligent data augmentation and meta-learning task generation methods, by carefully setting prompts and inputting them into the existing large language model: word-level perturbation (glyph replacement, such as "汇" → "江", "则" → "侧"), phrase-level perturbation (homophone replacement, such as "汇钱" → "会钱", "账户" → "张户"), and sentence-level perturbation (semantic rewriting, such as "Quickly transfer the funds to the specified account number, otherwise the money will be gone!"), and these perturbations all attempt to avoid detection while retaining the core malicious intent.

[0048] Take the phrase-level perturbed sample "Quickly transfer money to this Zhang account, otherwise there will be a shortage of 10 million!" as an example. The model first performs word segmentation and preliminary importance assessment on it, and assigns higher weights to the core harmful components such as "transfer money", "Zhang account", "otherwise", and "10 million". Subsequently, the model parallelly extracts three features of each character: deep semantics, pinyin, and glyph. The key step lies in the dynamic weight assignment of the gated fusion layer: Since this sample mainly perturbs the pinyin information (homophone replacement), the gated network will intelligently reduce the reliability of the pinyin feature (weight about 0.05), while significantly increasing the weights of the deep semantic feature (weight about 0.55) and the glyph feature (weight about 0.4) that are insensitive to the perturbation, so as to suppress noise and enhance the effective signal in the fused features.

[0049] For other perturbed samples, the gated network also shows adaptability. When facing the sample with glyph perturbation, the weight of the glyph feature will be reduced to an extremely low level (about 0.05), and the model turns to rely on the pinyin and semantic features; while in the sentence-level perturbed sample, since the glyph and pinyin clues have been completely changed, the model almost completely relies on the deep semantic feature (weight about 0.9) for understanding.

[0050] Finally, although the surface forms of the four samples are very different, the feature representations after gated fusion all strengthen their common threatening semantic essence. After the fused features are fed into the classifier, the model can consistently and highly confidently classify all four samples as "harmful", proving its ability to maintain high accuracy and strong robustness under different types of perturbations.

[0051] In summary, the present invention realizes the high-precision and interpretable classification of adversarial text messages through multi-modal fusion and effective attention mechanisms, providing reliable technical support for the real-time detection of spam text messages in practical scenarios.

Claims

1. An adversarial SMS binary classification method based on deep semantic understanding and multimodal gating, characterized in that, include: Get the SMS message to be identified; Semantic embedding feature vectors are extracted from the SMS messages to be identified using a pre-trained chinese-roberta-wwm-ext-large model encoder. Pinyin embedding feature vectors are extracted from the pinyin sequence of the SMS message to be identified using a pre-trained pinyin denoising autoencoder. The glyph embedding feature vector is extracted from the glyph structure of the SMS message to be recognized by a pre-trained glyph denoising autoencoder. The semantic embedding feature vector, the pinyin embedding feature vector, and the character shape embedding feature vector are weighted and fused through a pre-trained gated fusion module to obtain weighted fused features. A pre-trained SMS classifier is used to classify the SMS messages to be identified based on weighted fusion features.

2. The method according to claim 1, characterized in that, The Chinese-roberta-wwm-ext-large model encoder, the Pinyin denoising autoencoder, and the character shape denoising autoencoder were trained through comparative learning. The specific method is as follows: Obtain training samples; The training samples are input into the chinese-roberta-wwm-ext-large model encoder, the pinyin denoising autoencoder, and the character shape denoising autoencoder, respectively, to obtain the corresponding three output features; The triplet loss function is calculated based on the three output features, and its expression is as follows: in, The triplet loss value, To anchor the sample, As a positive sample, For negative samples, The distance between two samples is Euclidean distance in the embedding space, and margin represents the minimum distance difference between positive and negative sample pairs. The parameters of the chinese-roberta-wwm-ext-large model encoder, the pinyin denoising autoencoder, and the character denoising autoencoder are updated based on the triplet loss function until the triplet loss function stabilizes.

3. The method according to claim 1, characterized in that, The specific method for weighted fusion of semantic embedding feature vectors, pinyin embedding feature vectors, and glyph embedding feature vectors through the gating fusion module is as follows: The semantic embedding feature vector, the pinyin embedding feature vector, and the character shape embedding feature vector are concatenated to obtain the preliminary fusion feature; A two-layer transformer encoder is used to perform deep context encoding on the preliminary fused features, and the output is a context-aware feature encoding. The context-aware feature encoding is input into three gated networks with the same structure, and the weights of the semantic embedding feature vector, the pinyin embedding feature vector and the character shape embedding feature vector are extracted respectively. The weighted fusion feature is obtained by weighting and fusing the feature vectors of each modality according to their respective weights. The expression for this weighted fusion feature is as follows: in, For weighted fusion features, The weights of the semantic embedding feature vectors, For semantic embedding feature vectors, The weights of the pinyin embedding feature vector, For pinyin embedding feature vectors, The weights of the glyph embedding feature vectors, This is the feature vector for character embedding.

4. The method according to claim 1, characterized in that, The Chinese-roberta-wwm-ext-large model encoder, the Pinyin denoising autoencoder, the character denoising autoencoder, the gating fusion module, and the SMS classifier constitute a multimodal SMS classification model.

5. The method according to claim 4, characterized in that, The training process for the multimodal SMS classification model is as follows: Obtain SMS samples; By using a large language model to perform data augmentation on SMS samples, adversarial examples including metadata are obtained, including perturbation variants at three levels: word level, phrase level, and sentence level. The adversarial examples are input into a multimodal SMS classification model for classification, and the classification results are obtained. Based on the classification results, the multimodal SMS classification model is optimized in two layers using the MAML framework to obtain a pre-trained multimodal SMS classification model.

6. The method according to claim 5, characterized in that, Based on the classification results, the multimodal SMS classification model is optimized using the MAML framework in two layers, including outer loop optimization and inner loop optimization. Specifically, the inner loop optimization is as follows: Sample a training batch of tasks, where each task corresponds to a specific adversarial mode; The composite loss function corresponding to each sampled task is calculated using the meta-initial parameters of the current multimodal SMS classification model; The parameters of the multimodal SMS classification model are optimized by optimizing the composite loss function until the composite loss function is stable, thus obtaining task-specific parameters suitable for each adversarial mode. The outer loop is specifically: The composite loss function for each task is calculated using task-specific parameters applicable to each adversarial mode; By optimizing the composite loss function and performing gradient descent optimization on the initial parameters of the SMS classifier, a multimodal SMS classification model with optimized parameters is obtained.

7. The method according to claim 6, characterized in that, The composite loss function includes: classification loss based on cross-entropy loss function, weight prediction loss based on mean square error loss function, and adversarial consistency loss based on cosine similarity. The expression for the composite loss function is: in, This is the composite loss value. The weights for the classification loss, For classifying losses, The weights are used to predict the loss. For weighted prediction loss, Weights to counteract consistency loss, To combat the loss of consistency.

8. The method according to claim 7, characterized in that, The optimization method for the composite loss function is as follows: Based on the loss variance adjustment ratio, the weights of each loss function are adjusted using a dynamic weighting algorithm. The specific optimization formula is as follows: in, The weights are dynamically adjusted. k The value is c or m or a ;exp represents an exponential function with the natural constant as its base; Indicates the first k The variance of the loss over the most recent training steps, Hyperparameters for adjusting sensitivity; Indicates the first j The variance of the loss over the most recent training steps.

9. The method according to claim 8, characterized in that, During the training phase of the multimodal SMS classification model, a progressive parameter unfreezing strategy is adopted for fine-tuning. Specifically, the model parameters are unfrozen and trained in stages according to the model hierarchical structure.

10. The method according to claim 9, characterized in that, The multimodal SMS classification model is continuously trained. After each training cycle, the value of newly generated adversarial examples is automatically evaluated using the trained multimodal SMS classification model. Samples with values ​​higher than a set threshold are selected to expand the training set for a new round of training.

Images