Violation short message identification method and system based on deep semantic understanding

Abstract

The invention relates to the technical field of network security and data processing, and discloses a violation short message recognition method and system based on deep semantic understanding, and the method comprises the steps: firstly cleaning an original short message, generating a mixed embedding vector through characters, sub-words and pinyin, and carrying out the recognition of the violation short message; then processing through a double-layer detection engine, wherein the first layer utilizes rules and a lightweight model for rapid preliminary screening; in the second layer, for suspected samples, a double-tower fusion neural network architecture is adopted, local and global features are combined, fusion is carried out through a gating unit, and a large language model is input to carry out deep semantic reasoning. The system executes strategies such as interception or flow limiting according to the risk score, and realizes model iteration through a dynamic knowledge base and incremental learning. According to the method, the resource consumption and the detection precision are balanced through the layered architecture, the antagonistic variants are effectively identified by utilizing multi-dimensional feature fusion, and the method has the adaptive evolution capability for a novel violation mode.

Description

Technical Field

[0001] The present invention relates to the technical fields of network security and data processing, and specifically provides a method and system for identifying illegal text messages based on deep semantic understanding. Background Art

[0002] At present, mobile communication technology has been widely popularized, and text messages, as a basic communication method, carry important information transmission functions. At the same time, the number of illegal contents such as spam messages, telecom fraud, and malicious promotions has increased sharply. Such illegal information not only disrupts the communication order but is also often associated with illegal activities such as financial fraud or privacy theft, threatening the property safety of users. Achieving efficient governance of illegal text messages has become a key requirement in the field of communication security.

[0003] In response to the above situation, the existing technologies mainly adopt feature matching (keyword rule library) or traditional machine learning solutions (shallow machine learning models). Such technologies usually preset keyword libraries, blacklists, or regular expressions on the gateway side to match and filter data streams. Some solutions use Bayesian or support vector machine algorithms to extract shallow features such as text word frequencies for classification and judgment to distinguish normal text messages from illegal text messages.

[0004] The existing technologies have defects. Static rules and shallow models lack deep semantic understanding, have insufficient recognition capabilities when facing variants such as pinyin abbreviations, homophone substitutions, or symbol interference, and cannot recognize variant expressions (such as writing "brush orders" as "shua orders"), metaphors (such as "funding pool" implying illegal fundraising), or context-related risks, easily leading to missed reports. High false positive rate: It is easy to misjudge normal commercial text messages (such as promotional activities), affecting user experience, and the rule library lags behind the iteration of new话术s. Manual maintenance of the keyword library is required, making it difficult to cope with new types of fraud话术s (such as "AI face-swapping fraud", "virtual currency investment"), and difficult to handle unknown risks; Weak multi-language / dialect processing: Poor recognition capabilities for pinyin abbreviations (such as "nmsl") and dialect homophones (such as writing "pig butchering scam" as "杀猪盘"). In addition, directly applying large language models can improve semantic analysis capabilities, but the computational resource consumption is large and the inference latency is high, making it difficult to meet the real-time requirements in high-concurrency scenarios of communication gateways and unable to balance detection accuracy and system throughput. Summary of the Invention

[0005] Aiming at the deficiencies of the existing technologies, the present invention provides a method and system for identifying illegal text messages based on deep semantic understanding, solving the problems that the existing technologies are difficult to accurately identify new variants and adversarial illegal text messages and cannot balance deep semantic detection accuracy and real-time processing efficiency in large-scale concurrent scenarios.

[0006] To achieve the above objectives, the present invention is realized through the following technical solutions: In a first aspect, the present invention provides a method for identifying illegal text messages based on deep semantic understanding, comprising the following steps: S1. Receive the raw SMS data stream and perform cleaning and standardization operations to extract structured features and generate hybrid embedding vectors using hybrid embedding coding units. S2. Input the hybrid embedding vector and the structured features into the dual-layer detection engine. First, the first-layer detection unit performs a preliminary screening based on rule matching and a lightweight pre-trained model. When a suspected risk sample is determined, the suspected risk sample is transmitted to the second-layer detection unit. S3. The second layer detection unit performs deep semantic detection on the suspected risk sample. The second layer detection unit adopts a dual-tower fusion neural network architecture, extracts local feature vectors and global feature vectors respectively, and generates semantic feature vectors through a gated fusion unit, thereby outputting risk scores and classification probabilities. S4. The decision and response module receives the risk score, compares the risk score with a preset set of thresholds, generates an interception instruction, a flow restriction and review instruction or a release instruction based on the comparison result, and sends the instruction to the SMS gateway. S5, the dynamic knowledge base module, collects new samples through external data synchronization interfaces and active learning units, and performs incremental updates or retraining on model parameters.

[0007] By adopting the above technical solution, the system can balance throughput efficiency and recognition accuracy in massive SMS processing scenarios due to its dual-layer cascaded detection architecture and dynamic closed-loop update mechanism. The first layer of detection quickly filters high-frequency known violation patterns through rules and lightweight models, reducing computational load; the second layer of detection utilizes a large model with a dual-tower architecture to deeply mine the deep semantics of the text, effectively identifying subtle violations. Furthermore, by combining structured features with textual semantic features, and through continuous iteration of the dynamic knowledge base, this method solves the problem that traditional static rules are unable to cope with new variant attacks, achieving proactive defense and adaptive evolution against illegal SMS messages.

[0008] Preferably, the process of generating a hybrid embedding vector using a hybrid embedding coding unit in step S1 includes: mapping the standardized text into a character vector sequence through a character-level embedding layer, retaining the features of misspelled words and symbols; mapping the standardized text into a sub-word vector sequence using the vocabulary of a pre-trained model through a sub-word embedding layer, retaining semantic integrity; converting the standardized text into a pinyin sequence and mapping it into a pinyin vector sequence through a pinyin embedding layer, retaining speech features; concatenating the character vector sequence, the sub-word vector sequence, and the pinyin vector sequence along the feature dimension, and mapping the concatenated vector into the hybrid embedding vector through a linear projection layer.

[0009] By adopting the above technical solution and utilizing a multi-dimensional hybrid encoding strategy, it is possible to effectively combat the evasion methods commonly used by black market operators. Character-level features can capture interference patterns using similar-looking characters, character splitting, or the insertion of special symbols; pinyin-level features can identify keyword substitutions using homophones or homonyms; and sub-word-level features maintain semantic coherence. The fusion of these three features gives the model input strong noise resistance and robustness.

[0010] Preferably, the execution process of the first-layer detection unit in step S2 includes: matching the input data with a high-frequency risk rule base; if a match is found, directly outputting a high-risk judgment; if no rule is found, using a small pre-trained model that has undergone knowledge distillation to infer the hybrid embedding vector and calculate a preliminary risk probability; comparing the preliminary risk probability with a preset first judgment threshold; when the preliminary risk probability is greater than or equal to the first judgment threshold, outputting a high-risk judgment; when the preliminary risk probability is less than the first judgment threshold, marking the current sample as the suspected risk sample.

[0011] By adopting the above technical solution, a hierarchical filtering mechanism was constructed. A high-frequency risk rule base was used to achieve millisecond-level interception of known attacks, and a lightweight model was used to quickly identify simple variants. Only suspected samples that are difficult to judge were sent to the computationally intensive deep network, thereby optimizing the overall system's resource allocation and ensuring the real-time requirements of the SMS gateway.

[0012] Preferably, the execution process of the dual-tower fusion neural network architecture in step S3 includes: using the local feature extraction tower to process the hybrid embedding vector through a convolutional neural network structure and outputting the local feature vector; using the global semantic understanding tower to process the hybrid embedding vector through a Transformer structure and outputting the global feature vector; using the gated fusion unit to calculate the fusion coefficient based on the local feature vector and the global feature vector, and using the fusion coefficient to perform weighted synthesis of the local feature vector and the global feature vector to generate the semantic feature vector; and inputting the semantic feature vector into a large language model integrating a low-rank adaptive fine-tuning module for inference.

[0013] By employing the above technical solution, the advantages of convolutional neural networks in extracting local phrase patterns and the advantages of the Transformer architecture in capturing long-range contextual dependencies are combined. The gating fusion unit achieves dynamic weighting of local and global features through learned gating coefficients: the specific calculation logic involves concatenating local and global features, generating a gating value through linear transformation and activation function, and then using this gating value to perform complementary weighted summation of the two types of features. This mechanism enables the model to adaptively focus on the most effective feature dimensions when facing different types of illegal SMS messages.

[0014] Preferably, the low-rank adaptive fine-tuning module is configured as follows: during model inference and training, the pre-trained weight matrix of the large language model is kept frozen, and the injected low-rank decomposition matrix parameters are calculated and updated.

[0015] By adopting the above technical solutions, the memory resources and computation time required for model fine-tuning and iteration are reduced. By updating only the low-rank decomposition matrix, violation feature knowledge from the vertical domain can be quickly injected without compromising the general semantic capabilities of the base model, thereby improving the model's adaptability and update frequency in specific business scenarios.

[0016] Preferably, step S3 further includes inference operations based on prompt word engineering: calling a preset prompt word template library, filling the SMS text into the reserved slots of the template, and constructing an inference instruction sequence; the inference instruction sequence requires the large language model of the second layer detection unit to output the intermediate inference process according to the thought chain pattern; the intermediate inference process covers the speech pattern analysis, inducement behavior analysis and credibility vulnerability analysis, and generates a structured risk assessment result accordingly.

[0017] By employing the aforementioned technical solutions, the large language model is guided to explicitly generate reasoning paths, thereby enhancing its ability to identify complex logical traps and subtle deceptive scenarios. The thought chain model forces the model to perform logical verification before giving a conclusion, reducing misjudgments caused by logical leaps and providing interpretable criteria for subsequent human review.

[0018] Preferably, step S3 further includes a multimodal joint analysis operation: when a Uniform Resource Locator (URL) is detected in the SMS text, a web crawler is triggered to obtain the web page content pointed to by the URL; the text and structural features of the web page content are extracted, and the semantic consistency score between the web page content and the SMS text is calculated; the risk score is adjusted based on the semantic consistency score.

[0019] By adopting the above technical solution, the limitations of detection based on a single text modality are overcome. By verifying the consistency between the content claimed in the SMS message and the actual content of the landing page, phishing links or malicious software download inducements can be accurately identified. Even if the SMS text itself seems normal, risks can be detected through inconsistencies in the context.

[0020] Preferably, the threshold comparison process in step S4 includes a differentiation strategy: reading the profile tags of the receiving user; when the profile tags indicate that the receiving user belongs to a vulnerable group, loading an adjusted dynamic threshold; in the adjusted dynamic threshold, the values ​​of the high-risk determination threshold and the medium-risk determination threshold are lower than the default threshold values.

[0021] By adopting the above technical solutions, user-centric and refined security protection has been achieved. For high-risk groups such as the elderly and students who are easily deceived, the system automatically increases its interception sensitivity, blocking potential threats to the greatest extent possible within a controllable risk range and reducing the probability of these specific groups being scammed.

[0022] Preferably, the incremental update process in step S5 adopts an incremental learning paradigm based on knowledge distillation: the model currently being served online is used as the teacher model, and the new samples and historical samples constitute the incremental training set; the new student model is used to fit the true labels of the new samples, while mimicking the predicted output distribution of the teacher model on the historical samples; the total loss function including hard label loss and soft label loss is calculated, where the hard label loss is the cross-entropy loss between the prediction result of the student model on the new samples and the true labels, and the soft label loss is the divergence between the predicted probability distribution of the student model and the predicted probability distribution of the teacher model on the historical samples.

[0023] By employing the above technical solution, the forgetting problem during model iteration is resolved. Through joint optimization of hard-label loss and soft-label loss, the model can quickly absorb newly emerging violation features while maintaining its ability to discriminate historical samples by mimicking the output distribution of the teacher model. This mechanism ensures that the system's recognition ability steadily increases rather than fluctuates when facing constantly changing attack methods, achieving knowledge accumulation and inheritance.

[0024] Secondly, the present invention provides a system for identifying illegal text messages based on deep semantic understanding, configured to execute a method for identifying illegal text messages based on deep semantic understanding, including: The preprocessing module is configured to receive the raw SMS data stream, perform cleaning and standardization operations, and generate a hybrid embedding vector using the hybrid embedding coding unit; The dual-layer detection engine includes a first-layer detection unit and a second-layer detection unit. The first-layer detection unit is configured to perform initial screening and transmit suspected risk samples to the second-layer detection unit. The second-layer detection unit is configured to perform deep semantic detection on the suspected risk samples using a dual-tower fusion neural network architecture and output a risk score. The decision-making and response module is configured to generate control instructions based on the comparison result between the risk score and a preset threshold, and send the control instructions to the SMS gateway; The dynamic knowledge base module is configured to collect new samples and perform incremental updates to the model parameters.

[0025] By adopting the above technical solution, complex deep learning algorithms are engineered and encapsulated into collaborative modules, constructing a complete chain from data access and multi-level processing to decision response. The decoupled design between the system modules facilitates independent maintenance and upgrades, supports stable operation in high-concurrency environments, and achieves intelligent and automated governance of illegal SMS messages through the combination of hardware and software.

[0026] This invention provides a method and system for identifying illegal text messages based on deep semantic understanding. It has the following beneficial effects: 1. This invention constructs a two-layer cascaded detection engine comprising a lightweight initial screening and a large-model fine-tuning. The first layer quickly filters most known rule samples, while only a small portion of suspected long-tail samples are sent to the second layer for in-depth analysis. This layered and distributed architecture effectively balances detection depth and computational cost, reduces processing latency, and meets the high-concurrency requirements of large-scale SMS services. Furthermore, compared to solutions that rely entirely on large-model inference, this invention reduces the consumption of hardware resources such as GPUs, thereby lowering system deployment and operational costs.

[0027] 2. This invention employs a multi-dimensional hybrid embedding encoding that integrates characters, subwords, and pinyin, combined with a dual-tower fusion neural network architecture for deep semantic representation of text. This mechanism not only captures long-distance contextual dependencies in text but also effectively penetrates adversarial spoofing techniques employed by black market operators, such as using homophones, similar-looking characters, pinyin abbreviations, and the insertion of special symbols. Compared to traditional single-feature models, this solution improves the overall performance of identifying illegal SMS messages and effectively reduces the false positive and false negative rates in complex interference environments.

[0028] 3. This invention introduces a dynamic knowledge base module and an incremental learning mechanism based on knowledge distillation, constructing a closed-loop process from external data synchronization and active learning filtering to model parameter updates. The system can continuously absorb new samples from human review feedback and authoritative external databases, quickly acquiring new knowledge while overcoming forgetting. This enables the model to respond quickly to sudden new fraudulent tactics and variant attacks, shortening the identification cycle and achieving continuous iteration and adaptive evolution of security defense capabilities. Attached Figure Description

[0029] Figure 1 This is an architecture diagram of the illegal SMS identification system based on deep semantic understanding according to the present invention; Figure 2 This is a flowchart of the illegal SMS identification method based on deep semantic understanding of the present invention; Figure 3 This is a flowchart of the dynamic knowledge base update and model iteration process of the present invention. Detailed Implementation

[0030] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] See attached document Figure 1 This invention provides a system for identifying illegal text messages based on deep semantic understanding. The system operates in an electronic device containing a processor, memory, and communication interface. The memory stores computer-readable instructions, which the processor executes to implement the functions of each module. Logically, the system includes a preprocessing module, a two-layer detection engine, a dynamic knowledge base module, and a decision-making and response module. The modules interact via an internal data bus, collaboratively completing the entire process from raw data input to risk decision output.

[0032] The preprocessing module is configured as the system's input, receiving raw SMS data and converting it into a machine-readable vector representation. The preprocessing module includes a cleaning and normalization unit, a structured feature extraction unit, and a hybrid embedding coding unit.

[0033] The cleaning and normalization unit is configured to perform noise reduction and normalization operations on the original SMS text. This unit utilizes a pre-defined regular expression library to remove semantically meaningless symbols from the text. The unit is also equipped with a mapping dictionary to identify non-standard abbreviations and replace them with corresponding standard terms, while simultaneously identifying variant forms of sensitive words and restoring them to standard semantic terms.

[0034] The structured feature extraction unit is configured to extract metadata features related to SMS messages. This unit connects to the operator's database or log server via a communication interface to obtain the historical sending frequency of the sending number, the profile attribute data of the receiving user, and the timestamp information of SMS messages, and encodes the above information into structured feature vectors.

[0035] The hybrid embedding encoding unit is configured to generate text embedding vectors that fuse multi-dimensional features. Internally, this unit constructs three parallel encoding paths: a character-level embedding layer, a sub-word-level embedding layer, and a pinyin-level embedding layer. The character-level embedding layer maps the input text into a sequence of character vectors to preserve misspellings and symbols. The sub-word-level embedding layer maps the input text into a sequence of sub-word vectors based on a pre-trained model's vocabulary to preserve semantic integrity. The pinyin-level embedding layer converts the input text into a pinyin sequence and then maps it into a sequence of pinyin vectors to preserve speech features. The hybrid embedding encoding unit is further configured to concatenate or weightedly fuse the character vector sequence, sub-word vector sequence, and pinyin vector sequence, and generate the final input vector through linear projection.

[0036] The dual-layer detection engine is connected to the output of the preprocessing module and is used to perform cascaded detection on the hybrid embedded vectors and structured feature vectors. The dual-layer detection engine includes a first-layer detection unit and a second-layer detection unit.

[0037] The first-layer detection unit is configured as a lightweight filter. This unit integrates a small pre-trained model that has undergone knowledge distillation and a high-frequency risk rule base. The first-layer detection unit first matches the input data with the high-frequency risk rule base. If a match is found, a high-risk judgment is directly output. If no rule is found, the small pre-trained model is used to calculate a preliminary risk probability. If the preliminary risk probability is higher than a preset first threshold, a high-risk judgment is output; otherwise, the text message is marked as a suspected risk sample and transmitted to the second-layer detection unit.

[0038] The second-layer detection unit is configured as a large-model fine-tuning unit. This unit is based on a large language model and integrates a low-rank adaptive fine-tuning module. The low-rank adaptive fine-tuning module is configured to keep the pre-trained weight matrix of the base model frozen during model inference and training, only calculating and updating the parameters of the injected low-rank decomposition matrix to achieve efficient parameter fine-tuning.

[0039] The second-layer detection unit employs a dual-tower fusion neural network architecture, comprising a local feature extraction tower, a global semantic understanding tower, and a gated fusion unit. The local feature extraction tower uses a convolutional neural network structure to capture local phrase patterns in the text. The global semantic understanding tower uses a Transformer structure to capture long-distance dependencies and contextual semantics. The gated fusion unit connects to the outputs of both the local and global feature vectors, dynamically fusing their features. The gated fusion unit calculates a fusion coefficient based on the local and global feature vectors and uses this coefficient to weight and synthesize the two types of features, generating the final semantic feature vector.

[0040] The second-layer detection unit also includes a multimodal analysis interface. This interface is configured to trigger a web crawler to obtain the webpage content pointed to by the Uniform Resource Locator (URL) when a Uniform Resource Locator is detected in the SMS text, and then perform joint encoding analysis on the webpage text content and the SMS text content.

[0041] The output of the second-layer detection unit is connected to a multi-task classification head, configured to simultaneously output the probability distribution of violation categories and continuous risk scores. During model training, this unit employs the FocalLoss loss function to address the imbalance between positive and negative samples.

[0042] The dynamic knowledge base module connects to the two-layer detection engine. This module includes an external data synchronization interface, a black market keyword database storage unit, and an active learning unit. The external data synchronization interface is configured to periodically connect to an external anti-fraud database to obtain newly added fraud case data. The black market keyword database storage unit is used to store confirmed violation characteristic words.

[0043] The active learning unit is configured to receive user feedback data and manual review results, filter out false positives or missed positives with a confidence level below a preset safety threshold, and add these samples to the training dataset. The dynamic knowledge base module also includes a federated learning interface, configured to aggregate gradient parameters with models from other nodes without transmitting the original SMS data.

[0044] The decision and response module is connected to the two-layer detection engine. This module includes a scoring fusion unit, a threshold determination unit, and an execution unit. The scoring fusion unit receives the risk score and classification label output by the second-layer detection unit. The threshold determination unit stores high-risk, medium-risk, and low-risk thresholds.

[0045] The execution unit is configured to generate control instructions based on a comparison between the risk score and a threshold. When the risk score is greater than the high-risk threshold, an interception instruction is generated; when the risk score is between the medium-risk and high-risk thresholds, a rate-limiting and review instruction is generated; when the risk score is less than the low-risk threshold, a release instruction is generated. The execution unit sends these instructions to the SMS gateway. The decision-making and response module also includes a report generation unit, configured to extract high-weight feature segments from the gating fusion unit and generate a visual report containing the decision criteria.

[0046] See attached document Figure 2 This invention provides a method for identifying illegal text messages based on deep semantic understanding. The method mainly includes the following steps: First, the system receives the raw SMS data stream and performs preprocessing operations. This step includes cleaning the raw SMS text, removing redundant symbols without semantic information according to preset regular expression rules, restoring non-standard abbreviations in the text to standard vocabulary using a mapping dictionary, and identifying variant expressions of sensitive words and standardizing them into general semantic vocabulary. During this process, the system also extracts the historical sending frequency of the sending number, the profile attributes of the receiving user, and the sending time pattern to construct structured features. Simultaneously, the system uses three parallel encoding methods—character-level, sub-word-level, and pinyin-level—to encode the standardized text, generating a hybrid embedding vector containing multi-dimensional linguistic features.

[0047] Next, the system performs a first-layer lightweight detection on the processed data. This step first matches the SMS content against a pre-stored high-frequency risk rule base and a black market keyword base. If a match is successful, it is directly classified as high-risk and subsequent detection is terminated. If no rule is matched, a small pre-trained model that has undergone knowledge distillation is used to infer the hybrid embedding vector and calculate a preliminary risk probability. Based on this preliminary risk probability, the system classifies the SMS into high-risk confirmed samples, low-risk confirmed samples, or suspected risk samples. For confirmed samples, the judgment result is directly output; for suspected risk samples, they are transmitted to the next stage.

[0048] Subsequently, a second layer of deep semantic detection is performed on suspected risk samples. This step inputs the hybrid embedding vector into a dual-tower fusion neural network architecture. This architecture captures local phrase patterns in the text through a local feature extraction branch and long-distance dependencies in the text through a global semantic understanding branch. The system dynamically calculates the weights of the two branches using a gated fusion mechanism to generate a fused semantic feature vector. This feature vector is then input into a large language model that integrates a low-rank adaptive fine-tuning module. The large language model combines prompt word templates to infer the SMS message, outputting a risk score and implicit intent analysis results that include multiple dimensions such as fraud. If the SMS message contains a Uniform Resource Locator (URL), the system automatically extracts the landing page content and performs joint analysis with the text.

[0049] Then, the system performs a comprehensive decision-making and response operation. This step weights and fuses the risk score output by the second-layer detection with the rule matching results of the first-layer detection to obtain the final risk value. The system compares the final risk value with preset high-risk, medium-risk, and low-risk thresholds. When the final risk value is higher than the high-risk threshold, an interception instruction is generated and reported; when the final risk value is between the medium-risk and high-risk thresholds, a flow-limiting instruction is generated and the sample is sent to the manual review queue; when the final risk value is lower than the low-risk threshold, a release instruction is generated and logged.

[0050] Finally, the system performs dynamic knowledge base updates and model iterations. This step involves periodically synchronizing with authoritative external anti-fraud databases and black market keyword databases via external interfaces. Simultaneously, the system collects user reports and manually reviewed data, uses an active learning strategy to filter out high-confidence false positives or false negatives and add them to the training set, and incrementally updates or retrains the model parameters to achieve adaptive identification of new illegal SMS patterns.

[0051] Step S1 primarily performs data preprocessing and feature construction operations. This step aims to convert the unstructured raw SMS text into a high-dimensional tensor representation that can be processed by a computer model. The system first receives the raw SMS data stream through a data interface. This data stream contains the SMS text content and metadata such as sending time, sender identifier, and receiver identifier.

[0052] Upon receiving the raw SMS data, the system immediately initiates a cleaning and standardization process. The system loads a pre-built regular expression library to scan the SMS text. This process aims to filter out irrelevant characters that do not carry specific semantic meaning, including but not limited to consecutive special symbols, invisible characters, and meaningless emoticons. After initial cleaning, the system uses a maintained mapping dictionary to standardize the text. This mapping dictionary stores the correspondence between variant words commonly used in cybercrime and standard semantic words. Through dictionary matching, the system restores the pinyin abbreviations appearing in the text to their corresponding full Chinese names and replaces sensitive words disguised using similar-looking or homophonous characters with standard common words. For example, the system replaces the initials of pinyin representing a specific social media app with the app's official Chinese name and replaces homophonic variations involving gambling with the word "gambling."

[0053] After text standardization is completed, the system enters the hybrid embedding encoding stage. In order to comprehensively capture the semantic information and adversarial sample features in Chinese SMS messages, this invention constructs three parallel encoding paths: character-level embedding, word-level embedding, and pinyin-level embedding.

[0054] In the character-level embedding path, the system maps the normalized text sequence to a character index sequence. This path focuses on capturing glyphic features and fine-grained character combination patterns in the text, playing a crucial role in identifying illegal text messages that use misspellings or uncommon characters to evade detection. The system converts each character index into a corresponding character vector by looking up the character embedding matrix.

[0055] In the sub-word embedding path, the system uses the vocabulary of a pre-trained model to segment the text into sub-word sequences. This path focuses on preserving the complete semantic information of the words. The system converts sub-word indices into sub-word vectors by looking up the sub-word embedding matrix. These sub-word vectors can represent the semantic associations of words in a general context.

[0056] In the pinyin-level embedding path, the system first converts each Chinese character in the text into its corresponding pinyin character, forming a pinyin sequence. This path is specifically designed to address scenarios where homophones or puns are used for circumvention. By searching the pinyin embedding matrix, the system converts the pinyin sequence into pinyin vectors, thereby endowing the model with the ability to perceive speech similarity.

[0057] After obtaining the vector representations of the three different dimensions mentioned above, the system performs a feature fusion operation. The system concatenates the character vectors, sub-word vectors, and pinyin vectors along the feature dimensions, and then maps the concatenated high-dimensional vector to the input dimension required by the model through a linear projection layer. Let the character vector sequence be... The sub-word vector sequence is The pinyin vector sequence is The final input vector after fusion The calculation formula is as follows: ; in, This indicates the character vector sequence Sub-word vector sequence and Pinyin vector sequence The splicing operation performed along the feature channel dimension; This represents the weight matrix of the linear projection layer, used to adjust the dimension of the feature space to fit the input requirements of subsequent neural networks; This represents the bias vector of the linear projection layer. The input vector is calculated using this formula. It also contains the glyph, semantic, and phonetic features of the text, providing a robust data foundation for subsequent deep semantic understanding.

[0058] Step S2 primarily executes a two-layer cascaded detection process, which aims to optimize the allocation of system computing resources while ensuring recognition accuracy through a hierarchical filtering mechanism. The system first inputs the hybrid embedding vector and structured features generated in step S1 into the first-layer lightweight detection unit. This unit is equipped with a high-frequency risk rule base and a lightweight neural network model that has undergone knowledge distillation.

[0059] In the first layer of detection, the system first performs a rule matching operation. The system compares the features of the input data with the feature fingerprints in the high-frequency risk rule base. If the comparison result shows that the input data contains known violation patterns or combinations of black market keywords, the system directly determines that the sample is a high-risk sample, generates an interception instruction, and terminates the subsequent processing of the current sample. If the rule matching fails, the system calls a lightweight neural network model to perform inference calculations on the hybrid embedding vector and outputs a preliminary confidence value representing the probability of violation.

[0060] The system has a preset first judgment threshold to distinguish between confirmed and suspected samples. The system compares the preliminary confidence score with the first judgment threshold. When the preliminary confidence score is greater than or equal to the first judgment threshold, it indicates that the model has a high degree of confidence in determining the violation of the sample, and the system directly outputs a high-risk judgment result and ends the process. When the preliminary confidence score is less than the first judgment threshold, the system marks the sample as a suspected risk sample and transfers the sample's data to the second-layer deep semantic detection unit for refined analysis.

[0061] After entering the second layer of detection, the system activates a large-model fine detector based on a dual-tower fusion architecture. The system simultaneously inputs the hybrid embedding vector into both the local feature extraction tower and the global semantic understanding network. The local feature extraction tower uses a convolutional neural network structure to capture local phrase patterns and keyword co-occurrence features in the text, outputting a local feature vector. The global semantic understanding tower uses a Transformer structure to capture long-distance dependencies and implicit contextual semantics in the text, outputting a global feature vector. To dynamically integrate the two types of features, the system uses a gated fusion unit to calculate the fusion weights and generate the final semantic representation vector. Let the local feature vector be... The global feature vector is The final semantic representation vector is The calculation logic is as follows: ; ; in, This represents the gating coefficient vector, whose values ​​are limited to between 0 and 1; This represents the Sigmoid activation function; This indicates that the local feature vector and the global feature vector are concatenated along the feature dimension; and Let these represent the weight matrix and bias vector of the gated network, respectively; (symbols omitted) This represents the element-wise multiplication operation between vectors. Through this calculation process, the system adaptively adjusts the contribution ratio of local and global features based on the semantic complexity of the input text. Finally, the system inputs this semantic representation vector into the classifier, outputting an accurate risk score and a probability distribution of the violation category.

[0062] Step S3 primarily performs deep model analysis and multimodal joint verification based on prompt word engineering. This step aims to uncover the deep semantic logic of the SMS text and verify the consistency of external links. The system first enters the prompt word automatic assembly stage, which calls a preset prompt word template library. The system fills the SMS text processed in step S1 into the reserved slots of the template, constructing a complete inference instruction sequence. This instruction sequence explicitly requires the large language model not only to output the judgment result, but also to output the intermediate inference process according to the thought chain pattern. The inference dimensions include dialogue pattern analysis, i.e., identifying whether the text conforms to the structure of a specific fraud script; inducement behavior analysis, i.e., judging whether the text contains instructions to request money transfers, ask for verification codes, or induce clicking links; and credibility vulnerability analysis, i.e., detecting whether the text logic has self-contradictions or violates common sense. Based on this instruction sequence, the large language model gradually generates a text stream containing judgment criteria and finally outputs a structured risk assessment result.

[0063] While performing deep semantic analysis, the system concurrently performs multimodal joint analysis. When the system detects a Uniform Resource Locator (URL) in the SMS text, it triggers a security crawler unit. This unit accesses the target webpage pointed to by the URL in a sandbox environment and extracts the webpage's text content and page structure features. The system maps the webpage text content to a vector representation and calculates its semantic consistency score with the SMS text vector. The system checks whether the business identity claimed in the SMS matches the actual domain name and page content of the target webpage. If the SMS claims to be a bank notification but the target webpage is a login interface of a non-official domain, or if the SMS claims to be a promotional activity but the webpage content is blank or contains malicious code, the system will significantly increase the risk score. This cross-modal verification mechanism can effectively identify phishing websites and fake application download links.

[0064] To improve the model's ability to identify long-tail risk samples during training, the system employs a multi-task learning strategy and a specific loss function optimization mechanism. The system's output layer is designed as a multi-task classification head, containing a classifier branch for outputting the probability distribution of the violation category and a regressor branch for outputting continuous risk scores. Addressing the extreme imbalance between the number of violation and normal SMS samples, the system abandons the traditional cross-entropy loss function in the classification task, instead adopting a focus loss function. This loss function reduces the weight of easily classified samples, forcing the model to focus its updates on difficult-to-distinguish samples. Focus Loss Function The calculation formula is as follows: ; in, This represents the model's predicted probability value for the true class, and its value ranges from 0 to 1. This represents the balance factor, used to adjust the proportion of positive and negative samples contributing to the total loss, in order to correct the bias caused by the imbalance of the number of classes. This represents the focusing parameter, used to adjust the weight decay rate based on the difficulty of the samples. When When the value approaches 1, it indicates that the sample is easily classified. Approaching 0, thus reducing the loss value generated by that sample; conversely, when When the loss weight is small, the loss weight of the sample is retained. By jointly optimizing the classification loss and regression loss, the model can provide a discriminative risk confidence score while outputting an accurate class.

[0065] Step S4 primarily executes comprehensive decision-making and tiered response operations. This step aims to convert the quantitative risk scores generated in the preceding steps into specific actionable steps and implement differentiated control strategies based on business scenario requirements. The system first aggregates multi-dimensional risk scores, implicit intent analysis results, and multimodal consistency verification scores from the second-layer large-scale model refiner. The system then uses a weighted fusion algorithm to combine these sub-indicators into a normalized final risk confidence score.

[0066] After obtaining the final risk confidence level, the system retrieves a preset set of decision thresholds, which includes high-risk and medium-risk judgment thresholds. The system compares the final risk confidence level with these two thresholds and, based on the comparison result, classifies the current SMS message into three levels: confirmed violation, suspected violation, or normal communication. Let the final risk confidence level be... The high-risk threshold is The threshold for determining medium risk is .

[0067] When the final risk confidence level is greater than the high-risk determination threshold, that is... The system determines that the SMS message is a confirmed violation sample. In this scenario, the decision module immediately sends an interception command to the SMS gateway, blocking the delivery of the SMS message to the receiving terminal. Simultaneously, the system extracts the sender's number, text content, and judgment basis report of the SMS message, and reports them to the regulatory platform database through a standardized interface. Furthermore, the system extracts the sample's feature fingerprint and stores it in a blacklist cache, enabling direct blocking of repeated transmissions of the same content within a short period, without needing to invoke large model inference again.

[0068] When the final risk confidence level is between the medium-risk assessment threshold and the high-risk assessment threshold, that is... The system determines that the SMS message is a suspected violation sample. In this scenario, the decision-making module does not directly block the SMS message but instead triggers a rate-limiting protection mechanism. The system reduces the sending frequency quota of the source number within a unit of time, slowing down its sending speed. Simultaneously, the system marks the SMS sample and pushes it to the manual review queue. The review terminal will display a risk heatmap and explanatory report generated by the large model to assist reviewers in secondary confirmation. If the manual review confirms a violation, the system will update the source's reputation score; if it is confirmed as a false alarm, the system will mark the sample as a difficult case for subsequent model iterations.

[0069] When the final risk confidence level is less than the threshold for determining medium risk, i.e. The system determines that the SMS message is a normal communication sample. In this scenario, the decision module sends a permission command to the gateway, allowing the SMS message to be sent normally. The system records the log information of this process, including the processing timestamp, model score, and decision result, for system operation and maintenance monitoring and statistical analysis.

[0070] Furthermore, this step also includes differentiated strategy execution logic. When loading the judgment threshold, the system reads the profile tags of the receiving user. For specific sensitive user groups, the system automatically loads adjusted dynamic thresholds. For example, when the recipient is identified as a vulnerable group, the system lowers the values ​​of the high-risk and medium-risk judgment thresholds, thereby improving the sensitivity of intercepting potential fraudulent information and achieving customized security protection for different user groups.

[0071] See attached document Figure 3 Step S5 primarily involves dynamically updating the knowledge base and incrementally iterating the model. This step aims to build an adaptive learning loop, ensuring the system can quickly respond to the emergence of new types of offensive language and continuously optimize its recognition capabilities. The entire update mechanism follows the process of data collection, data cleaning, incremental training, and knowledge base distribution.

[0072] The system's data collection unit continuously gathers potential new knowledge through multi-source information interfaces. This unit regularly synchronizes the latest confirmed fraud case databases from authoritative institutions such as the Public Security Anti-Fraud Center via dedicated lines, while also capturing emerging illegal keywords and variations from internet black market intelligence channels. More importantly, this unit continuously collects user reports and feedback data, as well as high-confidence false positives and false negatives marked during manual review. This feedback data from real-world business scenarios is the most valuable source for model iteration.

[0073] During the data cleaning phase, the system standardizes and labels the collected heterogeneous data. For text from external case libraries, the system automatically performs the same cleaning and normalization operations as in step S1. For samples from internal feedback, the system assigns them the correct category labels based on the conclusions of manual review. The system deduplicates and filters newly added samples after cleaning, removing data that is highly similar to the existing training set to ensure the efficiency and effectiveness of incremental training.

[0074] After data preparation is complete, the system initiates the incremental training process. To learn new patterns without forgetting old knowledge, the system adopts an incremental learning paradigm based on knowledge distillation. The system uses the currently online model as the teacher model, and the new samples and a portion of historical samples together constitute the incremental training set. During training, the new student model not only needs to fit the true labels of the new samples but also needs to mimic the predicted output distribution of the teacher model on historical samples. The loss function for this process consists of two parts. Let the loss function for incremental training be... The calculation formula is as follows: ; in, Representing hard label loss, it is the cross-entropy loss between the student model's predictions on new samples and the true labels, which aims to help the model learn new knowledge; Representing soft label loss, it is the Kullback-Leibler divergence between the predicted probability distribution of the student model and the predicted probability distribution of the teacher model on historical samples. It aims to ensure that the model does not forget the old knowledge it has already mastered while learning new knowledge. It is a balancing coefficient used to adjust the relative importance of the two parts of the loss. This training method allows the model to efficiently absorb new knowledge without starting from scratch and shortening the model's response time to new fraud methods.

[0075] Once incremental training is complete, the fully validated new model will be deployed to the online inference environment, replacing the old teacher model. Simultaneously, novel keywords, high-frequency risk patterns, and variant vocabulary extracted from the new samples will be automatically updated in the high-frequency risk rule base of the first-layer detection unit and the mapping dictionary of the preprocessing module. Through this closed-loop process, the system can continuously learn from external intelligence and internal feedback, achieving synchronous evolution of the knowledge base and model capabilities, thereby effectively combating ever-evolving malicious SMS attacks.

[0076] This embodiment details the actual operation of the proposed method and system for identifying illegal SMS messages based on deep semantic understanding, using a specific application scenario. The scenario is set as a large cloud communication SMS service platform deploying this system. This platform handles tens of millions of SMS messages daily and has recently been frequently attacked by financial fraud using new scripts. Attackers attempt to bypass the platform's existing keyword-matching-based traditional interception system by constantly changing the sender's number, using homophones, inserting special symbols, and using pinyin abbreviations. The illegal SMS message identification system of this invention is deployed in the core processing link of the platform's SMS gateway, after the traditional rule filter. It aims to perform a second, deeper cleansing of suspected traffic that has undergone initial filtering but still poses a risk. The system needs to accurately identify and block highly concealed new types of illegal information while ensuring the timely arrival of normal verification codes and notification SMS messages.

[0077] The entire process of processing a typical investment fraud SMS message is as follows: The system receives a raw SMS data stream to be detected. The text contains elements such as inducing users to add social media accounts, claiming insider channels, promising high returns, and limited-time offers. The preprocessing module first performs a standardized cleaning operation on the text. The noise filtering unit within the module identifies special graphic symbols interspersed in the text as semantically meaningless interference noise and removes them. Simultaneously, the dictionary mapping unit within the module identifies abbreviations or homophones in the pinyin of characters as pronouns referring to a well-known instant messaging software based on contextual semantics and standardizes and maps them to standard common vocabulary. The structured feature extraction unit simultaneously analyzes the sender's metadata, discovering that the sending number is a recently registered and newly active number from another region, and that the user profile of the receiving terminal shows that the recipient belongs to a specific age group that is easily deceived.

[0078] The processed data flows to the first-layer detection engine, the lightweight filter. This filter performs a rapid scan of the text based on the distilled lightweight model. Although the model identifies combinations of high-frequency risky words in the text, given the presence of non-standard symbols and disguised sentence structures in the SMS message, the initial confidence level output by the lightweight model does not reach the high threshold for direct blocking. Therefore, the sample is marked as a suspected risk sample and proceeds to the next layer for refined detection. This step effectively filters out a large amount of obviously normal traffic, reducing the load on the subsequent large model.

[0079] After entering the second-layer detection unit, deep semantic detection is performed using a dual-tower fusion neural network architecture and a large language model. The prompt word generation unit automatically assembles inference instructions based on a preset template. The generated instructions explicitly require the large language model to determine whether the text message constitutes fraud, and analyze it based on the wording pattern, inducement behavior, and credibility vulnerabilities. After receiving the instructions, the large language model performs chain reasoning, analyzing that the text message uses the phrase "internal channel" to establish a false sense of authority, uses the promise of guaranteed profits to violate common sense in financial investment, and uses limited availability to create a sense of scarcity and urgency, which fully conforms to the typical wording characteristics of financial fraud. The model combines the local and global features extracted by the dual-tower architecture to finally output a very high risk score and a definitive classification label.

[0080] The decision-making and response module receives risk scores and executes judgments based on differentiated strategies. Considering that the user profile of the receiving terminal extracted in the preceding steps belongs to a vulnerable group, the system automatically activates differentiated protection strategies and loads an adjusted dynamic threshold set, in which the high-risk judgment threshold value is lower than the default setting. Given that the risk score of the current sample significantly exceeds the judgment threshold, the system determines it as high-risk and immediately sends an interception command to the SMS gateway to block the sending of the SMS. The system simultaneously generates a structured audit report containing the above reasoning and submits it to the regulatory platform database for subsequent verification and evidence collection.

[0081] The adaptive identification process for variant SMS messages occurred the day after the aforementioned case was processed. The system detected a variant SMS message from a different source, whose text content had been distorted. For example, the promise of high returns was changed to other semantically similar expressions, and the social media account prompts were changed to different pinyin combinations. Since the sample successfully intercepted the previous day had triggered the update mechanism of the dynamic knowledge base module, the system had increased the weight of similar inducement pattern features through incremental learning. When processing this variant SMS message, the preprocessing module used a hybrid embedding coding strategy to accurately identify the meaning of the pinyin abbreviations through a pinyin-level embedding layer. At the same time, the global semantic understanding tower of the large model captured the high similarity between the variant words and the original risk words in the semantic space, both pointing to irrational high-return promises. Based on the updated knowledge base and enhanced anti-interference coding capabilities, the system quickly determined that the implicit intent of this variant SMS message was consistent with the illegal sample from the previous day, and the decision module immediately executed the interception operation. This process verified that when facing attackers who change their wording and use pinyin abbreviations, the system can maintain a high accuracy rate in identification through deep semantic understanding and adaptive learning mechanisms, achieving automatic coverage of variant risks without the need for manual addition of new keyword rules.

Claims

1. A method for identifying illegal text messages based on deep semantic understanding, characterized in that, Includes the following steps: S1. Receive the raw SMS data stream and perform cleaning and standardization operations to extract structured features and generate hybrid embedding vectors using hybrid embedding coding units. S2. Input the hybrid embedding vector and the structured features into the dual-layer detection engine. First, the first-layer detection unit performs a preliminary screening based on rule matching and a lightweight pre-trained model. When a suspected risk sample is determined, the suspected risk sample is transmitted to the second-layer detection unit. S3. The second layer detection unit performs deep semantic detection on the suspected risk sample. The second layer detection unit adopts a dual-tower fusion neural network architecture, extracts local feature vectors and global feature vectors respectively, and generates semantic feature vectors through a gated fusion unit, thereby outputting risk scores and classification probabilities. S4. The decision and response module receives the risk score, compares the risk score with a preset set of thresholds, generates an interception instruction, a flow restriction and review instruction or a release instruction based on the comparison result, and sends the instruction to the SMS gateway. S5, the dynamic knowledge base module, collects new samples through external data synchronization interfaces and active learning units, and performs incremental updates or retraining on model parameters.

2. The method for identifying illegal text messages based on deep semantic understanding according to claim 1, characterized in that, The process of generating a hybrid embedding vector using a hybrid embedding coding unit in step S1 includes: The standardized text is mapped to a sequence of character vectors through a character-level embedding layer, preserving the features of misspellings and symbols. The normalized text is mapped into a sequence of sub-word vectors by using the vocabulary of the pre-trained model through the sub-word embedding layer, thus preserving semantic integrity. The standardized text is converted into a pinyin sequence and then mapped to a pinyin vector sequence through a pinyin-level embedding layer, thus preserving speech features. The character vector sequence, the sub-word vector sequence, and the pinyin vector sequence are concatenated along the feature dimension, and the concatenated vector is mapped to the hybrid embedding vector through a linear projection layer.

3. The method for identifying illegal text messages based on deep semantic understanding according to claim 1, characterized in that, The execution process of the first-layer detection unit in step S2 includes: The input data is matched against a high-frequency risk rule base. If a match is found, a high-risk judgment is directly output. If the rule is not hit, the hybrid embedding vector is inferred using a small pre-trained model that has undergone knowledge distillation to calculate the preliminary risk probability; The preliminary risk probability is compared with a preset first judgment threshold. When the preliminary risk probability is greater than or equal to the first judgment threshold, a high-risk judgment is output. When the preliminary risk probability is less than the first judgment threshold, the current sample is marked as the suspected risk sample.

4. The method for identifying illegal text messages based on deep semantic understanding according to claim 1, characterized in that, The execution process of the dual-tower fusion neural network architecture in step S3 includes: The hybrid embedding vector is processed by a convolutional neural network structure using a local feature extraction tower to output the local feature vector. The global semantic understanding tower is used to process the hybrid embedding vector through a Transformer structure, and the global feature vector is output. The gating fusion unit calculates a fusion coefficient based on the local feature vector and the global feature vector, and uses the fusion coefficient to weight and synthesize the local feature vector and the global feature vector to generate the semantic feature vector; The semantic feature vectors are input into a large language model that integrates a low-rank adaptive fine-tuning module for inference.

5. The method for identifying illegal text messages based on deep semantic understanding according to claim 4, characterized in that, The configuration method of the low-rank adaptive fine-tuning module is as follows: During model inference and training, the pre-trained weight matrix of the large language model is kept frozen, and the parameters of the injected low-rank decomposition matrix are calculated and updated.

6. The method for identifying illegal text messages based on deep semantic understanding according to claim 1, characterized in that, The S3 step also includes reasoning operations based on cue word engineering: Call the preset prompt word template library, fill the SMS text into the reserved slots of the template, and construct the inference instruction sequence; The inference instruction sequence requires the large language model of the second-layer detection unit to output the intermediate inference process according to the thought chain pattern. The intermediate reasoning process encompasses verbal pattern analysis, inducement behavior analysis, and credibility vulnerability analysis, thereby generating structured risk assessment results.

7. The method for identifying illegal text messages based on deep semantic understanding according to claim 1, characterized in that, The S3 step also includes a multimodal joint analysis operation: When a Uniform Resource Locator (URL) is detected in the SMS text, a web crawler is triggered to obtain the web page content pointed to by the URL. Extract the text and structural features of the webpage content, and calculate the semantic consistency score between the webpage content and the SMS text; The risk score is adjusted based on the semantic consistency score.

8. The method for identifying illegal text messages based on deep semantic understanding according to claim 1, characterized in that, The threshold comparison process in step S4 includes a differentiation strategy: Read the profile tags of the receiving user; When the profile tag indicates that the receiving user belongs to a vulnerable group, an adjusted dynamic threshold is loaded; In the adjusted dynamic thresholds, the values ​​of the high-risk determination threshold and the medium-risk determination threshold are lower than the default threshold values.

9. The method for identifying illegal text messages based on deep semantic understanding according to claim 1, characterized in that, The incremental update process in step S5 adopts an incremental learning paradigm based on knowledge distillation: Use the model currently being used online as the teacher model, and combine the new samples with historical samples to form an incremental training set; The new student model is used to fit the true labels of the newly added samples, while mimicking the distribution of the predicted output of the teacher model on the historical samples. Calculate the total loss function, which includes hard label loss and soft label loss. The hard label loss is the cross-entropy loss between the student model's prediction result on the new sample and the true label. The soft label loss is the divergence between the student model's predicted probability distribution and the teacher model's predicted probability distribution on the historical sample.

10. A system for identifying illegal text messages based on deep semantic understanding, characterized in that, Configured to perform the method as described in any one of claims 1 to 9, comprising: The preprocessing module is configured to receive the raw SMS data stream, perform cleaning and standardization operations, and generate a hybrid embedding vector using the hybrid embedding coding unit; The dual-layer detection engine includes a first-layer detection unit and a second-layer detection unit. The first-layer detection unit is configured to perform initial screening and transmit suspected risk samples to the second-layer detection unit. The second-layer detection unit is configured to perform deep semantic detection on the suspected risk samples using a dual-tower fusion neural network architecture and output a risk score. The decision-making and response module is configured to generate control instructions based on the comparison result between the risk score and a preset threshold, and send the control instructions to the SMS gateway; The dynamic knowledge base module is configured to collect new samples and perform incremental updates to the model parameters.

Images