Text content auditing method and device based on AC automaton and large model and storage medium
The text content auditing method combining AC automata and large models solves the limitations of existing technologies in recognizing complex semantic text, achieving efficient and accurate text content auditing, reducing computational costs and improving security.
Patent Information
- Application Number
- CN202510997126.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-18
- Publication Date
- 2025-11-11
AI Technical Summary
In existing technologies, regular expression matching and sensitive word matching algorithms have limitations in recognizing complex semantic text, and cannot fully guarantee the accuracy and security of content auditing, especially in preventing the generation of risky content when users input malicious content.
This approach combines AC automata with a large model. By constructing a sensitive word database and a Trie tree, the AC automata is used to identify sensitive words, and the large model is used for risk identification. Semantic risks are sampled and checked, and audit results are output.
It achieves efficient and accurate text content auditing, reduces computational costs, improves the comprehensiveness and security of auditing, and reduces the risk of missed detections.
Smart Images
Figure CN120930639A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of text content auditing technology, specifically to a text content auditing method and apparatus based on AC automata and large model. Background Technology
[0002] With the rapid development of large-scale model technology and the increasing frequency of user use, content security has become a key concern. As generative AI is widely applied, content security is crucial for preventing technology misuse and protecting user rights. Large-scale model content security refers to systematic safeguards that ensure the output content complies with laws, regulations, social ethics, and user needs when using large-scale models to generate content, preventing the generation of harmful, misleading, biased, or illegal information.
[0003] When users interact with large models, the security risks of the content generated by these models largely stem from malicious input and manipulation by users. Therefore, it is necessary to take a user-centric approach and prevent large models from generating risky content by identifying malicious prompts and preventing users from inputting offensive or misleading content.
[0004] When interacting with large models, user input is diverse, ranging from everyday natural language to complex terminology in specialized fields. Regular expression matching is a common text recognition method; however, it has limitations when faced with such diverse user input. Regular expressions rely primarily on pre-defined fixed patterns, which are effective for well-formatted text recognition, but establishing fixed regular expressions is extremely difficult for semantically rich and grammatically flexible text. In the current technological context, sensitive word matching algorithms can quickly retrieve and identify pre-defined sensitive words in text, offering high flexibility. However, these algorithms can only identify sensitive words in the text and cannot perform in-depth violation determination or security classification based on the identified sensitive words. In real-world scenarios, a text may contain sensitive words, but the context may not indicate a violation. In such cases, using sensitive word matching algorithms cannot guarantee comprehensive auditing. Summary of the Invention
[0005] The purpose of this invention is to provide a text content auditing method, apparatus and storage medium based on AC automata and large model, so as to solve the problems existing in the prior art.
[0006] To achieve the above objectives, a text content auditing method based on Aho-Corasick automata and large models includes the following steps:
[0007] S1, Obtain the text content to be audited;
[0008] S2, Construct a sensitive word library for sensitive word identification;
[0009] S3 uses the AC automaton to identify sensitive words and obtain the identification results;
[0010] S4. Based on the results of sensitive word identification, sampling or direct use of large models for risk identification is performed.
[0011] S5 outputs audit results, including the text content to be audited, sensitive word identification results, and large model risk identification results.
[0012] Furthermore, the text content to be audited mentioned in S1 includes the input and output parameters of the large model;
[0013] The input parameters of the large model refer to the text content entered by the user during the interaction with the large model;
[0014] The output parameters of the large model refer to the text content of the large model's response during the interaction process.
[0015] Furthermore, the construction of the sensitive word database mentioned in S2 includes: based on business needs, constructing a database to identify and filter specific words or phrases, and screening the text content to be audited;
[0016] The sensitive word database includes sensitive word IDs, sensitive word categories, and sensitive word names;
[0017] Based on the application scenarios of text auditing, construct a sensitive word database for relevant industries or fields.
[0018] Furthermore, the use of the AC automaton for sensitive word identification as described in S3 includes:
[0019] Construct an AC automaton;
[0020] Sensitive word matching and identification are performed based on the constructed AC automaton.
[0021] Furthermore, the construction of the AC automaton includes:
[0022] Based on the aforementioned sensitive words, a Trie tree is constructed, and all sensitive words are inserted into the prefix tree, with each node representing a character;
[0023] Construct mismatch pointers: For all nodes in the constructed Trie tree, construct mismatch pointers that point to another node. If the current path fails to match, jump to that node to continue matching.
[0024] Starting from the root node of the Trie tree, traverse the text to be audited and match sensitive words based on the Trie tree and mismatch pointers.
[0025] Furthermore, the sensitive word identification result refers to whether the text content to be audited contains sensitive words;
[0026] If a sensitive word is matched based on the AC automaton, the corresponding recognition result is "contains sensitive word";
[0027] If the AC automaton does not match any sensitive word in the sensitive word library, the corresponding recognition result is "does not contain sensitive words".
[0028] Furthermore, the sampling or risk identification based on the sensitive word identification results described in S4 includes:
[0029] If the text contains sensitive words, it is directly input into the large model for risk identification;
[0030] If the text does not contain sensitive words, sampling is performed, and the sampled text is input into the large model for risk identification; unsampled text is output directly, and the output includes the text to be audited and the results of its sensitive word identification.
[0031] Furthermore, the results of the risk identification include the risk status and risk classification of the text to be audited;
[0032] The risk status includes two states: "safe" and "unsafe".
[0033] The risk classification refers to the risk classification given based on the large model if the risk status of the text to be audited is unsafe.
[0034] This invention provides a text content auditing device based on Aho-Corasick automata and a large model, comprising:
[0035] The text acquisition module is used to acquire the text content to be audited;
[0036] The lexicon building module constructs a sensitive word lexicon for sensitive word identification.
[0037] The sensitive word recognition module uses the AC automaton to identify sensitive words and obtain the recognition results.
[0038] The sampling module is used to sample audit text that does not contain sensitive words.
[0039] The large-scale model risk identification module identifies and classifies risks in audit texts based on a large-scale model.
[0040] The output module is used to output the audit results of the text, including the text content to be audited, the results of sensitive word identification, and the results of risk identification by the large model.
[0041] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the text content auditing method based on AC automata and large model as described above.
[0042] By adopting the above technical solution, the present invention has the following beneficial effects:
[0043] This invention provides a text auditing method and apparatus based on Aho-Corasick automata and a large model, which has the following beneficial effects:
[0044] 1. The AC automaton has efficient multi-pattern matching capabilities. Through the structure of Trie trees and mismatch pointers, it can detect all sensitive words simultaneously in O(n) time complexity.
[0045] 2. Combining the AC automaton with a large model for text auditing can improve audit accuracy. The AC automaton can only be used to identify sensitive words, and it cannot effectively identify risks when dealing with complex semantics. Combining it with a large model can determine whether the text is truly in violation of regulations from a semantic perspective.
[0046] 3. The auditing approach combining AC automata with a large model can significantly reduce computational costs. The AC automaton can quickly match sensitive words and complete the initial text filtering, while the large model only analyzes text containing sensitive words, effectively reducing the number of model calls and thus lowering computational costs.
[0047] 4. Although some texts do not contain sensitive words, they may pose a semantic risk. Sampling such texts can effectively reduce the risk of missed detections without sacrificing the overall review quality. Attached Figure Description
[0048] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0049] Figure 1 This is a flowchart illustrating the text content auditing method based on AC automata and large model of the present invention.
[0050] Figure 2 This is another flowchart of the text content auditing method of the present invention;
[0051] Figure 3 The AC automaton matching flowchart provided by this invention;
[0052] Figure 4 The flowchart for large-scale model risk identification provided by this invention;
[0053] Figure 5 This is a schematic diagram of a Trie tree provided by the present invention;
[0054] Figure 6This is a schematic diagram of the mismatch pointer construction provided by the present invention;
[0055] Figure 7 A schematic diagram of the text content auditing device provided by the present invention. Detailed Implementation
[0056] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0057] Combination Figure 1 As shown, a text content auditing method based on AC automata and large model includes the following steps:
[0058] S1, Obtain the text content to be audited;
[0059] S2, Construct a sensitive word library for sensitive word identification;
[0060] S3 uses the AC automaton to identify sensitive words and obtain the identification results;
[0061] S4. Based on the results of sensitive word identification, sampling or direct use of large models for risk identification is performed.
[0062] S5 outputs audit results, including the text content to be audited, sensitive word identification results, and large model risk identification results.
[0063] In one embodiment, the text content to be audited in S1 includes the input parameters and output parameters of the large model;
[0064] The input parameters of the large model refer to the text content entered by the user during the interaction with the large model;
[0065] The output parameters of the large model refer to the text content of the large model's response during the interaction process.
[0066] The text content to be audited is obtained by receiving input and output parameters from the relevant devices of the large model for text auditing.
[0067] Specifically, the construction of the sensitive word database mentioned in S2 includes: based on business needs, building a database to identify and filter specific words or phrases, and screening the text content to be audited;
[0068] Based on the application scenarios of text auditing, construct a sensitive word database for relevant industries or fields;
[0069] The sensitive word database includes sensitive word IDs, sensitive word categories, and sensitive word names, as shown in Table 1.
[0070] Table 1
[0071] ID Sensitive word classification Sensitive word name 1 illegal and violent categories potent anesthetic 2 illegal and violent categories anaesthetization 3 illegal and violent categories Anesthetic 4 illegal and violent categories Anesthetic ether ... ... ...
[0072] In one embodiment, the use of the AC automaton for sensitive word identification in step S3 includes:
[0073] Construct an AC automaton;
[0074] Sensitive word matching and identification are performed based on the constructed AC automaton.
[0075] In one embodiment, see Figure 3 As shown, the construction of the AC automaton includes:
[0076] Based on the aforementioned sensitive words, a Trie tree is constructed, and all sensitive words are inserted into the prefix tree, with each node representing a character;
[0077] Construct mismatch pointers: For all nodes in the constructed Trie tree, construct mismatch pointers that point to another node. If the current path fails to match, jump to that node to continue matching.
[0078] Starting from the root node of the Trie tree, traverse the text to be audited and match sensitive words based on the Trie tree and mismatch pointers.
[0079] AC automata are based on the structure of Trie and combined with the KMP concept. They consist of Trie trees and mismatch information.
[0080] See Figure 3 S401, Construct a Trie tree by inserting all sensitive words from the sensitive word database into the Trie tree. Taking the sensitive words given in Table 1 as an example, form the pattern string {strong anesthetic, anesthesia, anesthetic drug, anesthetic ether}. The Trie tree constructed using this pattern string is as follows: Figure 5 As shown.
[0081] exist Figure 5 In the diagram, node 0 is the root node, and nodes 5, 7, 8, and 10 represent the end nodes of the sensitive words.
[0082] See Figure 3 S402, Constructing the mismatch pointer. The construction of the mismatch pointer adopts breadth-first search, processing the nodes from smallest to largest level. The core idea is to use the mismatch information of the parent node to calculate the mismatch information of the child node.
[0083] The AC automaton uses mismatch pointers to assist in matching multi-pattern strings. Multi-pattern string matching refers to matching one or more sensitive words simultaneously within a text. When a match fails at the current node in the Trie tree, the mismatch pointer indicates which node to jump to and continue trying to match.
[0084] The construction of the mismatch pointer of the Trie tree based on the pattern string {potent anesthetic, anesthesia, anesthetic, anesthetic ether} is shown in Figure 6 , and the construction process is as follows:
[0085] (1) The mismatch pointer of the root node 0 in the first layer points to itself, that is, it points to itself when the input is not equal to {potent, anesthetic};
[0086] (2) The mismatch pointers of the nodes 1 and 6 in the second layer point to the root node 0;
[0087] (3) For the node 2 in the third layer, the mismatch pointer of its parent node 1 points to the node which is the root node 0. When encountering the character "effective", it cannot make a transition, so the mismatch pointer of the node 2 points to the root node 0;
[0088] (3) For the node 7 in the third layer, the mismatch pointer of its parent node 6 points to the node which is the root node 0. When encountering the character "drunk", it cannot make a transition, so the mismatch pointer of the node 7 points to the root node 0;
[0089] (4) For the node 3 in the fourth layer, the mismatch pointer of its parent node 2 points to the node which is the root node 0. When encountering the character "anesthetic", it can make a transition to the node 6, so the mismatch pointer of the node 3 points to the node 6;
[0090] (5) For the node 8 in the fourth layer, the mismatch pointer of its parent node 7 points to the node which is the root node 0. When encountering the character "medicine", it cannot make a transition, so the mismatch pointer of the node 8 points to the root node 0;
[0091] (6) For the node 9 in the fourth layer, the mismatch pointer of its parent node 7 points to the node which is the root node 0. When encountering the character "ethyl", it cannot make a transition, so the mismatch pointer of the node 9 points to the root node 0;
[0092] (7) For the node 4 in the fifth layer, the mismatch pointer of its parent node 3 points to the node which is the node 6. When encountering the character "drunk", it can make a transition to the node 7, so the mismatch pointer of the node 4 points to the node 7;
[0093] (8) For the node 10 in the fifth layer, the mismatch pointer of its parent node 9 points to the node which is the root node 0. When encountering the character "ether", it cannot make a transition, so the mismatch pointer of the node 10 points to the root node 0;
[0094] (9) For the node 5 in the sixth layer, the mismatch pointer of its parent node 4 points to the node which is the node 7. When encountering the character "medicine", it can make a transition to the node 8, so the mismatch pointer of the node 5 points to the node 8.
[0095] Furthermore, in step S403, match sensitive words. Taking the text "The anesthetic power of potent anesthetic" as an example, based on the above pattern string {potent anesthetic, anesthesia, anesthetic, anesthetic ether}, the matching process is as follows:
[0096] (1) Input the character "qiang", and transfer from the root node 0 to node 1;
[0097] (2) Input the character "xiao", and transfer from node 1 to node 2;
[0098] (3) Input the character "ma", and transfer from node 2 to node 3;
[0099] (4) Input the character "zui", and transfer from node 3 to node 4;
[0100] (5) Input the character "yao", and transfer from node 4 to node 5. Node 5 is the end node of the sensitive word, so the sensitive word "qiangxiao mazuiyao" is matched;
[0101] (6) Input the character "de". Node 5 cannot transfer normally, so transfer to node 8 according to the mismatch pointer. Node 8 is the end node of the sensitive word, and the sensitive word "mazuiyao" is matched;
[0102] (7) Node 8 processes the input character "de" and cannot transfer normally. Transfer to the root node 0 according to the mismatch pointer, So the root node 0 does not perform a state transfer;
[0103] (8) Continue to input the character "ma", and transfer from the root node 0 to node 6;
[0104] (9) Input the character "zui", and transfer from node 6 to node 7. Node 7 is the end node of the sensitive word, and the sensitive word "mazui" is matched;
[0105] (10) Input the character "wei", Node 7 cannot transfer normally. Transfer to the root node 0 according to the mismatch pointer, So the root node 0 does not perform a state transfer;
[0106] (11) Input the character "li", Node 7 cannot transfer normally. Transfer to the root node 0 according to the mismatch pointer, So the root node 0 does not perform a state transfer;
[0107] (12) The text input ends, and the sensitive words "qiangxiao mazuiyao", "mazuiyao", and "mazui" are matched.
[0108] Based on the above example, the output of the sensitive word recognition result is as follows:
[0109] {"contains_sensitive_words":"True","sensitive_result":[{"sensitive_category_name":"Illegal and Violent Category","sensitive_word_name":"Powerful Anesthetic"},{"sensitive_category_name":"Illegal and Violent Category","sensitive_word_name":"Anesthetic"},{"sensitive_category_name":"Illegal and Violent Category","sensitive_word_name":"Anesthetic"}]}
[0110] In one embodiment, the sensitive word identification result refers to whether the text content to be audited contains sensitive words;
[0111] If a sensitive word is matched based on the AC automaton, the corresponding recognition result is "contains sensitive word";
[0112] If the AC automaton does not match any sensitive word in the sensitive word library, the corresponding recognition result is "does not contain sensitive words".
[0113] For the acquired text content, the AC automaton is used to identify sensitive words and generate identification results. The identification results include identification tags (contains_sensitive_words) and identification results (sensitive_result).
[0114] The identification flag refers to whether the text matches a sensitive word. If one or more sensitive words are matched, the identification flag (contains_sensitive_words) is True; if no sensitive words are matched, the identification flag (contains_sensitive_words) is False, and the identification result is set to empty.
[0115] The identification results include sensitive words (sensitive_category_name) and sensitive word names (sensitive_word_name).
[0116] In one embodiment, the sampling or risk identification based on the sensitive word identification results described in S4 includes:
[0117] If the text contains sensitive words, it is directly input into the large model for risk identification;
[0118] If the text does not contain sensitive words, sampling is performed, and the extracted samples are input into the large model for risk identification; the unextracted samples are directly output, and the output content includes the text to be audited and the sensitive word identification result.
[0119] Based on the sensitive word identification mark, sampling is performed or the large model is used for risk identification.
[0120] Based on the sensitive word identification mark, the next step is processed. Specifically, see Figure 2 :
[0121] In step S203, based on the text content containing sensitive words, the large model is used for risk identification. If the sensitive word identification mark is True (including sensitive words), the audit text is directly input into the large model for risk identification.
[0122] See Figure 2 As shown, in step S204, random sampling is performed based on the text content without sensitive words.
[0123] Specifically, if the sensitive word identification mark is False (not including sensitive words), the original text is stored in the clean text library. The clean text library is used to store the audit text without sensitive words and its storage time.
[0124] Furthermore, based on the storage time, a fixed time window is established to sample the text in the clean text library. For example, if the fixed time window is defined as 1 hour, starting from 0:00 on the same day, a sampling operation is performed every 1 hour. The overall population for sampling refers to all the clean text within 1 hour, and the sampling method is simple random sampling.
[0125] Furthermore, the extracted random samples are input into the large model for risk identification.
[0126] The process of risk identification by the large model is shown in Figure 4 :
[0127] S501, Input text. The input text refers to the text containing sensitive words described in step S203 and the random samples described in step S204.
[0128] S502, Preprocess the input text. The preprocessing includes word segmentation, adding special marks, mapping to IDs, padding and truncating, and generating tensors.
[0129] Word segmentation means splitting the input text into discrete units (Tokens) that the model can process. For example, after word segmentation of the text "The anesthetic power of a powerful anesthetic", the corresponding Tokens are ['strong', 'effect', 'anesthetic', 'drug', 'of', 'anesthetic', 'power']. The word segmentation result is not unique, and different word segmentation tools may have different results.
[0130] Furthermore, add special markers, <s> "Indicates the start of text,"< / s> "Indicates the end of text," <unk>If "represents an unknown word, then the word segmentation result is [ <s> The anesthetic power of a potent anesthetic drug.< / s> ].
[0131] Furthermore, each token is mapped to an integer ID. The ID corresponds to the vocabulary in the model.
[0132] The mapping results are shown in the following example:
[0133] Tokens: [ <s> The anesthetic power of a potent anesthetic drug.< / s> ]
[0134] Mapping ID: [1,300,185,168,187,177,166,168,187,189,170,2]
[0135] Furthermore, padding and truncation are necessary. To meet the model's input requirements, the sequence length usually needs to be standardized, and truncation (removing some tokens) or padding (adding some special tokens) should be performed based on the model's maximum acceptable length.
[0136] If the model's maximum acceptable length is 15, then the example tokens need to be padded, typically using special markers, such as... <pad>The populated example Tokens and mapping IDs are as follows:
[0137] Tokens: [ <s> The anesthetic power of a potent anesthetic drug.< / s> , <pad> , <pad> , <pad>]
[0138] Mapping ID: [1,300,185,168,187,177,166,168,187,189,170,2,0,0,0]
[0139] Further, the list of tokenIDs is converted into a tensor.
[0140] The tensor sequence generated from the input text typically consists of two main parts: a token ID sequence (input_ids) and an attention mask (attention_mask).
[0141] The attention mask is used to identify which parts of the input are real tokens (value 1) and which parts are padding tokens (value 0).
[0142] The generated tensor dictionary for the example Tokens and mapping IDs above is as follows:
[0143]
[0144] Both input_ids and attention_mask are two-dimensional tensors of shape batch_size * sequence_length, where batch_size represents the number of texts in a batch and sequence_length represents the length of the input sequence, consistent with the maximum acceptable length of the model.
[0145] In the example above, the number of input texts is 1, and the maximum acceptable length of the model is 15. Therefore, the shapes of input_ids and attention_mask are 1*15.
[0146] S503, Model Inference.
[0147] Tensor sequences are input into the model for inference. The large model refers to a multi-label classification evaluation model, which can simultaneously detect whether the input text involves multiple independent risk categories. Each category independently determines whether there is a risk. Categories are not mutually exclusive, and the input text can contain multiple risks at the same time.
[0148] The output consists of logits for multiple categories, where logits refer to the normalized scores output by the model. In multi-label classification tasks, each category has a corresponding logit value, representing the model's confidence in that category.
[0149] If the model supports 5 risk categories (unsafe_categories): violent crime, pornography and vulgarity, privacy violation, discrimination and hate speech, and intellectual property.
[0150] Table 2 shows examples of the model's output logits after inputting text.
[0151] Table 2
[0152] Risk Classification logit value Violent crime 1.8 Pornography and Vulgarity -0.5 Privacy infringement 3.2 Discrimination and hate speech 0.1 Intellectual Property -2.0
[0153] S504, multi-head classification output.
[0154] Based on the logit value of the model classification, a risk score for each category is output after transformation by the Sigmoid function.
[0155] The Sigmod function formula is as follows: p = 1 / (1 + e) -logit )
[0156] Where p represents the risk score, and logit is the normalized score output by the multi-label classification.
[0157] Table 3 shows examples of the risk scores for the texts in various categories after Sigmoid transformation.
[0158] Table 3
[0159] Risk Classification logit value Risk Score Violent crime 1.8 0.86 Pornography and Vulgarity -0.5 0.38 Privacy infringement 3.2 0.96 Discrimination and hate speech 0.1 0.52 Intellectual Property -2.0 0.12
[0160] S505, Risk Assessment.
[0161] Based on the thresholds set by the model, it is determined whether the input text triggers a risk in each category. If the risk scores for all categories are below the threshold, the text is considered safe; otherwise, it is considered unsafe, and the categories exceeding the threshold are the risk categories to which the input text belongs.
[0162] Assuming the model sets a threshold of 0.8, based on the risk scores in the above examples, since the risk scores for "violent crime" and "privacy infringement" both exceed 0.8, they are judged as unsafe.
[0163] S506 outputs structured results.
[0164] The output includes a security identifier (security_id) and risk categories (unsafe_categories). The security identifier includes "safe" and "unsafe"; the risk categories are those corresponding to risk scores exceeding the set threshold.
[0165] Based on the above example, the output structured result is as follows:
[0166] {"security_id":"unsafe","unsafe_categories":["violent crime","privacy violation"]}
[0167] In one embodiment, the result of the risk identification includes the risk status and risk classification of the text to be audited;
[0168] The risk status includes two states: "safe" and "unsafe".
[0169] The risk classification refers to the risk classification given based on the large model if the risk status of the text to be audited is unsafe, such as "violent crime" or "defamation".
[0170] The audit results are output, including the original text content, sensitive word identification results, and large model risk identification results.
[0171] The original text content is the text content to be audited.
[0172] In yet another embodiment, see Figure 7 As shown, the present invention also provides a text content auditing device based on Aho-Corasick automata and a large model, comprising:
[0173] The text acquisition module is used to acquire the text content to be audited;
[0174] The lexicon building module constructs a sensitive word lexicon for sensitive word identification.
[0175] The sensitive word recognition module uses the AC automaton to identify sensitive words and obtain the recognition results.
[0176] The sampling module is used to sample audit text that does not contain sensitive words.
[0177] Large-scale model risk identification module: Based on the large-scale model, risk identification and classification are performed on audit texts;
[0178] The output module is used to output the audit results of the text, including the text content to be audited, the results of sensitive word identification, and the results of risk identification by the large model.
[0179] See Figure 7 As shown, the device mainly includes a text acquisition module 31, a lexicon construction module 32, a sensitive word recognition module 33 (Trie tree construction module 331, mismatch pointer construction module 332, and matching module 333), a sampling module 34, a large model risk recognition module 35 (data preprocessing module 351, model inference module 352, and risk judgment module 353), and an output module 36.
[0180] The sensitive word identification module 33 includes a Trie tree construction module 331, which is used to construct a prefix tree based on all sensitive words in the sensitive word library; a mismatch pointer construction module 332, which is used to construct mismatch pointers for all nodes on the Trie tree to assist in sensitive word matching; and a matching module 333, which is used to match all sensitive words in the text to be audited.
[0181] The sampling module 34 is used to sample text that does not contain sensitive words in the text to be audited, in order to reduce the false negative rate.
[0182] The large model risk identification module 35 includes a data preprocessing module 351, which converts the input text into a mathematical representation (usually a numerical vector or tensor) that can be effectively processed and calculated within the model; a model inference module 352, which inputs the preprocessed input data into the trained model, performs calculations, and outputs multi-class logit values; and a risk judgment module 353, which calculates the risk score and, based on a set risk threshold, obtains the risk judgment result of the text.
[0183] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the text content auditing method based on AC automata and large model as described above.
[0184] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.< / pad> < / pad> < / pad> < / pad> < / unk>
Claims
1. A text content auditing method based on Aho-Corasick automata and large model, characterized in that, Includes the following steps: S1, Obtain the text content to be audited; S2, Construct a sensitive word library for sensitive word identification; S3 uses the AC automaton to identify sensitive words and obtain the identification results; S4. Based on the results of sensitive word identification, sampling or direct use of large models for risk identification is performed. S5 outputs audit results, including the text content to be audited, sensitive word identification results, and large model risk identification results.
2. The method according to claim 1, characterized in that, The text content to be audited as described in S1 includes the input and output parameters of the large model; The input parameters of the large model refer to the text content entered by the user during the interaction with the large model; The output parameters of the large model refer to the text content of the large model's response during the interaction process.
3. The method according to claim 2, characterized in that, The construction of the sensitive word database mentioned in S2 includes: based on business needs, building a database to identify and filter specific words or phrases to screen the text content to be audited; The sensitive word database includes sensitive word IDs, sensitive word categories, and sensitive word names; Based on the application scenarios of text auditing, construct a sensitive word database for relevant industries or fields.
4. The method according to claim 1, characterized in that, S3 describes the use of the AC automaton for sensitive word identification, which includes: Construct an AC automaton; Sensitive word matching and identification are performed based on the constructed AC automaton.
5. The method according to claim 1, characterized in that, The construction of the AC automaton includes: Based on the aforementioned sensitive words, a Trie tree is constructed, and all sensitive words are inserted into the prefix tree, with each node representing a character; Construct mismatch pointers: For all nodes in the constructed Trie tree, construct mismatch pointers that point to another node. If the current path fails to match, jump to that node to continue matching. Starting from the root node of the Trie tree, traverse the text to be audited and match sensitive words based on the Trie tree and mismatch pointers.
6. The method according to claim 5, characterized in that, The sensitive word identification result refers to whether the text content to be audited contains sensitive words; If a sensitive word is matched based on the AC automaton, the corresponding recognition result is "contains sensitive word"; If the AC automaton does not match any sensitive word in the sensitive word library, the corresponding recognition result is "does not contain sensitive words".
7. The method according to claim 6, characterized in that, S4 describes sampling or risk identification based on sensitive word recognition results, which includes: If the text contains sensitive words, it is directly input into the large model for risk identification; If the text does not contain sensitive words, sampling is performed, and the sampled text is input into the large model for risk identification; unsampled text is output directly, and the output includes the text to be audited and the results of its sensitive word identification.
8. The method according to claim 6, characterized in that, The results of the risk identification include the risk status and risk classification of the text to be audited; The risk status includes two states: "safe" and "unsafe". The risk classification refers to the risk classification given based on the large model if the risk status of the text to be audited is unsafe.
9. A text content auditing device based on Aho-Corasick automata and a large model, characterized in that, include: The text acquisition module is used to acquire the text content to be audited; The lexicon building module constructs a sensitive word lexicon for sensitive word identification. The sensitive word recognition module uses the AC automaton to identify sensitive words and obtain the recognition results. The sampling module is used to sample audit text that does not contain sensitive words. The large-scale model risk identification module identifies and classifies risks in audit texts based on a large-scale model. The output module is used to output the audit results of the text, including the text content to be audited, the results of sensitive word identification, and the results of risk identification by the large model.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the text content auditing method based on AC automata and large model as described in any one of claims 1-8.