Large model domain security enhancement method and system based on hybrid strategy

Abstract

The invention discloses a large model domain security enhancement method based on a hybrid strategy, and discloses a system with the large model domain security enhancement method based on the hybrid strategy. According to the large model domain security enhancement method based on the hybrid strategy, a security defense strategy which does not need large model training and can be optimized for a domain specific security scene is provided. A user does not need to have a large language model fine tuning training technology basis and a data set construction basis, and does not need to have large-scale computing power. Security vulnerabilities found after online application can be blocked in time, and meanwhile, the security defense strategy provided by the invention can be quickly migrated to multiple field applications.

Description

Technical Field

[0001] The present invention relates to the technical field of dynamic uncertainty measurement, and in particular to a fusion method and system for dynamic measurement uncertainty evaluation. Background Art

[0002] With their powerful learning and reasoning capabilities, large models have achieved remarkable results in various fields. These models have been applied to multiple scenarios such as intelligent question answering, content generation, and text translation. As the application scenarios gradually expand, large models process a large amount of sensitive data and personal information in many scenarios. Therefore, the security of large models (also known as the value alignment problem) has become one of the main issues faced. The current industry faces the following challenges: (1) Harmful content generation: large models may generate inappropriate, malicious, or harmful content. (2) Data privacy leakage: large models process a large amount of sensitive data during training, and there is a risk of data privacy leakage. The model may inadvertently generate information containing personal privacy, threatening user data security. (3) Bias and discrimination: due to the bias of training data, large models may learn and amplify these biases, resulting in discrimination when generating content or making decisions.

[0003] The industry usually uses supervised fine-tuning (SFT) training and reinforcement learning from human feedback (RLHF) to enhance the security alignment capability of large models. This type of method mainly works in the model training stage. By training and adjusting the model weights, the model response content is aligned with human values, reducing the generation of harmful content, and can effectively alleviate the security issues of large models in general scenarios. However, there are still the following problems: (1) For the application of large models in specific vertical fields, this type of method has high requirements for training resources and training technology, and it is difficult to timely add field-specific security risk issues in the application of specific industries; (2) Even after training with a large amount of data, it is still difficult for large models to completely avoid generating harmful content. Therefore, this type of method cannot timely plug the security vulnerabilities found during the application process; (3) When the large model needs to be applied to multiple products or multiple fields at the same time, this type of method needs to retrain the model, so it is impossible to quickly migrate the model security capabilities. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a large model field security enhancement method based on a hybrid strategy. The user does not need to have a large language model fine-tuning training technology foundation and a data set construction foundation, and does not need to have large-scale computing power. Security vulnerabilities discovered after the online application can be blocked in time, and the security defense strategy proposed by the present invention can be quickly migrated to multiple fields of application.

[0005] The present invention also proposes a system with a large model domain security enhancement method based on a hybrid strategy.

[0006] The large model domain security enhancement method based on a hybrid strategy according to the first aspect of the present invention is characterized by comprising the following steps:

[0007] Based on the pre-defined sensitive word library and rule library, the generated dialogue text is matched with hierarchical sensitive words to detect whether the information input by the user contains sensitive words or illegal content;

[0008] The semantic recognition model based on the pre-trained language model performs semantic recognition on the information input by the user to determine whether it contains implicit security risks;

[0009] Use a semantic recognition model based on a pre-trained language model to conduct real-time review of the streaming output content of large models, and cut off the streaming output in time if sensitive content occurs.

[0010] The large model domain security enhancement method based on a hybrid strategy according to an embodiment of the present invention has at least the following beneficial effects: the security enhancement method provided by the present invention introduces the sensitive word review algorithm based on the prefix tree and the semantic review model based on the RoBERTa model into the application of large models in specific fields, and can effectively enhance the domain security capabilities without fine-tuning training of large models, thereby reducing the application costs of large models in the industry. At the same time, it can promptly respond to and block security vulnerabilities discovered during online applications, making it easier for operation and maintenance personnel to perform system maintenance. This method can be quickly migrated to multiple fields of application and is highly practical.

[0011] According to some embodiments of the present invention, the semantic recognition model based on the pre-trained language model performs semantic recognition on the information input by the user to determine whether it contains implicit security risks, specifically including:

[0012] The multiple sensitive word matching algorithm of the prefix tree algorithm is used to determine whether the information input by the user contains first-level, second-level, third-level banned words and regular expressions;

[0013] When the algorithm determines that it is a first-level banned word, this method directly outputs that the content is banned;

[0014] When the algorithm determines that it is a second-level banned word, the probability value of the text content is determined through the semantic recognition model. When the preset second-level banned probability threshold is reached, it is determined to be banned content;

[0015] When the algorithm determines that the text contains level 3 banned words, the probability value of the text is determined through the semantic recognition model. When the preset level 3 banned probability threshold is reached, it is determined to be banned content.

[0016] According to some embodiments of the present invention, the step of determining whether the information input by the user contains primary, secondary, and tertiary banned words and regular expressions by using a multiple sensitive word matching algorithm of a prefix tree algorithm specifically includes:

[0017] Classify existing deformation-sensitive words;

[0018] Preprocess the information input by the user according to the classification of the deformed sensitive words to obtain the preprocessed sensitive words;

[0019] Based on the preprocessed sensitive words, a prefix tree algorithm is used to determine whether the preprocessed sensitive words contain first-level, second-level, third-level banned words and regular expressions.

[0020] According to some embodiments of the present invention, the prefix tree is constructed based on pinyin, and 24 child nodes are created under the root node, wherein nodes 0 to 22 respectively store Chinese sensitive words with their first letters of pinyin, and node 23 stores sensitive words starting with numbers or other non-Chinese and English characters, and marks whether the node is a terminal node while storing the Chinese characters and their pinyin; wherein the root node and the first-layer child nodes are fixed.

[0021] According to some embodiments of the present invention, the existing sensitive words generally include three categories: the first category is sensitive words with special characters added; the second category is sensitive words deformed by using pinyin, homophones, homophones, etc.; the third category is sensitive words that have been split and traditionalized.

[0022] According to some embodiments of the present invention, the step of performing corresponding preprocessing on the information input by the user according to the classification of the deformed sensitive words to obtain the preprocessed sensitive words specifically includes:

[0023] For the first category, if multiple special characters are inserted into a sensitive word, the special characters are replaced with "*";

[0024] For the second category, when encountering a continuous pinyin string, the sensitive word filtering algorithm uses a regular expression to segment the pinyin string based on the maximum matching principle. When matching the pinyin, the node array is used to store the matched nodes, and the pre array is used to store all the nodes corresponding to the pinyin. If there is a successfully matched node in the previous round, it will be stored in the node. The current round of matching will start from the child nodes in the pre array until the terminal node is matched;

[0025] The third category of sensitive words will be directly separated: for English strings, there are 409 common Chinese pinyin combinations; if the English string is a pinyin string, it will be separated into single pinyin; if the English string cannot be separated into pinyin, it will be separated into single English characters.

[0026] According to some embodiments of the present invention, the pre-trained language model is a RoBERTa model, which adopts a Transformer structure and is composed of multiple encoders, each of which is composed of two sub-modules: a multi-head self-attention mechanism module and a forward feedback neural network module.

[0027] According to some embodiments of the present invention, the pre-trained language model sets different thresholds for the semantic recognition model according to the level of keywords hit by user input; when it is judged to be a second-level banned word, and the semantic recognition model judges that the probability of danger of the text content exceeds 0.9, it is judged as banned content; when it is judged that the text has third-level banned words, and the semantic recognition model judges that the probability of danger of the text content exceeds 0.95, it is judged as banned content; even if no banned words are hit, when the semantic recognition model judges that the probability of danger of the text content exceeds 0.99, it will also be judged as banned content.

[0028] According to some embodiments of the present invention, the step of using a semantic recognition model based on a pre-trained language model to conduct real-time auditing of the streaming output content of the large model and promptly cutting off the streaming output if sensitive content occurs specifically includes:

[0029] The model performs a content review every time a complete sentence is generated. Once the text content is determined to be sensitive, the model stops streaming output, responds with a safety-preserving script, and creates a new conversation.

[0030] The large model domain security enhancement system based on a hybrid strategy according to the second aspect of the present invention is deployed at the interaction end of the large model, and specifically includes:

[0031] The sensitive word definition module can perform hierarchical sensitive word matching on the generated dialogue text based on the pre-defined sensitive word library and rule library to detect whether the information input by the user contains sensitive words or illegal content;

[0032] The user language processing module can perform semantic recognition on the information input by the user based on the semantic recognition model of the pre-trained language model to determine whether it contains implicit security risks;

[0033] The real-time audit module can use a semantic recognition model based on a pre-trained language model to perform real-time audit on the streaming output content of a large model, and cut off the streaming output in time if sensitive content occurs.

[0034] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0036] Figure 1 A schematic diagram of the steps of a large model domain security enhancement method based on a hybrid strategy according to an embodiment of the present invention;

[0037] Figure 2 for Figure 1 The structural block diagram of the large model domain security enhancement system based on hybrid strategy according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0038] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0039] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0040] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0041] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0042] Large models are limited by existing technologies and it is difficult to build complete moral values ​​and input tendencies, which may generate harmful content. Based on the above reasons, it is necessary to build a security defense strategy for large models. This application provides a large model field security enhancement method based on a hybrid strategy, which adopts a hybrid strategy of multiple sensitive word matching + semantic recognition + real-time verification of model output content to enhance the security capabilities of large models.

[0043] Reference Figure 1 The large model domain security enhancement method based on hybrid strategy provided in the embodiment of the present application includes at least the following steps.

[0044] Step S100: multiple sensitive words matching.

[0045] Based on the pre-defined sensitive word library and rule library, the generated dialogue text is matched with hierarchical sensitive words to detect whether the information input by the user contains sensitive words or illegal content.

[0046] Preferably, before classifying sensitive words, in order to reduce the problem of ambiguity of similar characters, conventional text processing methods such as converting traditional Chinese to simplified Chinese, converting uppercase and lowercase letters, removing special symbols, and dividing text by periods (the original text will also be retained for detection) are also required.

[0047] Step S200: semantic recognition.

[0048] The semantic recognition model based on the pre-trained language model performs semantic recognition on the information input by the user to determine whether it contains implicit security risks.

[0049] Step S300: real-time verification.

[0050] Use a semantic recognition model based on a pre-trained language model to conduct real-time review of the streaming output content of large models, and cut off the streaming output in time if sensitive content occurs.

[0051] The present invention uses a prefix tree's multiple sensitive word matching technology combined with a RoBERTa model to determine whether the text content is dangerous. The specific process is: when a piece of text content is input, the model first uses a prefix tree-based multiple sensitive word matching algorithm to determine whether the text content has first-level, second-level, and third-level banned words and regular expressions. When the algorithm determines that it is a first-level banned word, this method directly outputs that the content is banned; when the algorithm determines that it is a second-level banned word, the probability value of the text content is determined by the semantic recognition model, and when a certain probability value (the threshold is set to 0.9) is reached, it is determined to be banned content; when the algorithm determines that the text has a third-level banned word, the probability value of the text is determined by the semantic recognition model, and when a certain probability value (the threshold is set to 0.95) is reached, it is determined to be banned content. After the above multiple defense mechanisms are passed, the large model is input for reasoning, and the reply content of the large model's streaming output is periodically identified, and the streaming output is cut off in time once sensitive content occurs.

[0052] Prefix tree, also known as dictionary tree, is an ordered tree used to store associative arrays, in which the keys are usually strings. Due to its unique structure, the prefix tree algorithm can be applied to string retrieval, text prediction, automatic completion, spelling check, word frequency statistics, sorting, the longest common prefix of strings, prefix matching of string searches, and other data structures and algorithm auxiliary structures. The present invention performs sensitive word review based on the prefix tree algorithm.

[0053] The prefix tree algorithm can be used for classification of deformed sensitive words. The above steps can be specifically divided into the following processes.

[0054] Step S100: multiple sensitive words matching. It includes:

[0055] Step S110: Preprocessing of sensitive word filtering algorithm.

[0056] After studying and analyzing deformation sensitive words, the existing deformation sensitive words are classified and summarized, and deformation sensitive words can be roughly divided into the following three categories:

[0057] (1) Add special characters to sensitive words, add non-Chinese symbols between sensitive words, or use symbols to replace a character in a sensitive word. For example, use "*" to replace a character in a sensitive word to form a modified sensitive word.

[0058] (2) Use pinyin, homophones, or homophones to deform sensitive words.

[0059] (3) Sensitive words that have been split and converted to traditional Chinese characters.

[0060] Step S120: Text classification preprocessing.

[0061] Classifying text is the first step in filtering sensitive words. Different discrete processing methods are used for the three types of sensitive words introduced in 1.

[0062] For the first category, if multiple special characters are inserted into a sensitive word, only one "*" will be used to replace it.

[0063] The situation of pinyin separation in the second category is more complicated. The English string may be pinyin or English words, so further processing is required when separating the English string.

[0064] The third category of sensitive words will be directly separated. For English strings, there are 409 common Chinese pinyin combinations. This article uses regular matching as the core to identify pinyin strings. If it is a pinyin string, it will be correctly separated into single pinyin; if it cannot be separated into pinyin, it will be separated into single English characters.

[0065] Preferably, the classification of sensitive words needs to be implemented through a prefix tree. The nodes of the prefix tree are generally composed of English characters, so a node of the prefix tree generally has 26 child nodes. Chinese is different. There are nearly 7,000 common Chinese characters. If the prefix tree of Chinese sensitive words is constructed according to the prefix tree of English, the difficulty of searching will be greatly increased. Therefore, this method improves the structure of the English prefix tree to adapt to the construction of the prefix tree of Chinese sensitive words.

[0066] The algorithm is based on the idea of ​​decision tree and the composition of Chinese Pinyin. It constructs an improved prefix tree for Chinese sensitive words. The tree can support the search of multiple variant sensitive words and the storage of special characters. Since the first letter of Chinese Pinyin is composed of 23 letters (excluding "u", "v", and "i"), 24 child nodes are created under the root node of the tree, of which nodes 0 to 22 store Chinese sensitive words with the first letter Pinyin respectively, and node 23 stores sensitive words starting with numbers or other non-Chinese and English characters, and marks whether the node is a terminal node while storing Chinese characters and their Pinyin. The root node and the first-level child nodes are fixed.

[0067] Specifically, in the above step S120, the specific implementation method is as follows.

[0068] For texts containing common sensitive words, they can be directly filtered in the word library after text classification. In the process of matching modified sensitive words, for the third type of modified sensitive words (sensitive words that have been split and converted to traditional Chinese), the sensitive word filtering algorithm has added such modified sensitive words as much as possible when building the sensitive word library, and can be directly matched using the prefix tree. For the first and second types of modified words, special matching algorithms are required for filtering.

[0069] (1) Maximum matching algorithm for the first type of sensitive words

[0070] For the first type of sensitive words, the special symbol may only replace one of the sensitive word characters, or it may just be a space between the Chinese characters to play an interference role. Therefore, fuzzy matching is performed in two cases during matching.

[0071] For example, the sensitive word "AB*D" (ABCD is stored in the sensitive word library), its matching process is as follows: after successfully matching the two characters "AB", the next character to be matched is "*". If "*" does not replace any character in the sensitive word, at this time, the first-layer node is the first letter of the word A, the second-layer node is the Chinese character "A", and the third-layer node is the Chinese character "B", then the next character "D" is directly compared with the character of the lower-layer node, that is, the fourth-layer node, and it is found that there is no match. Then "D" is compared with the character of the next layer, that is, the fifth-layer node, and it is found that it is exactly matched. This matching node is both a leaf node and a terminal node, so this round of matching ends. Therefore, it is considered that "AB*D" in the test text matches "ABCD" in the sensitive word library, and the sensitive word is considered to be hit.

[0072] (2) Maximum matching algorithm for the second type of sensitive words

[0073] For the second type of inflected words, when encountering a continuous pinyin string, the sensitive word filtering algorithm uses a regular expression to segment the pinyin string based on the maximum matching principle. When matching the pinyin, due to the complexity of the matching situation, additional space is required to store the nodes, and the node array is used to store the matched nodes. Since a pinyin may correspond to multiple Chinese characters, when detecting the pinyin string, the pre array stores all the nodes corresponding to the pinyin. If there is a successfully matched node in the previous round, it will be stored in the node. This round of matching will start from the child node in the pre array until the terminal node is matched.

[0074] Step S200: semantic recognition.

[0075] The embodiment of the present application adopts the RoBERTa model, uses the Transformer structure, and is composed of multiple encoders, each of which is composed of two submodules: a multi-head self-attention mechanism module and a forward feedback neural network module. The core idea of ​​the semantic recognition model based on the RoBERTa model is to calculate the relationship between each word in a sentence and all the words in the sentence, and then use these relationships to adjust the weight of each word to obtain a new expression for each word. This expression not only contains the semantics of the word itself, but also contains its relationship with other words. Therefore, the vector obtained by this method has a more global expression than the traditional word vector. During the training process of this model, domain-specific security issues can be introduced, such as gray operation problems in the finance and taxation industry, gray area problems in the legal industry, etc., and potential domain-specific security risks can be identified with the help of the semantic recognition ability of the RoBERTa model.

[0076] During operation, different thresholds are set for the semantic recognition model according to the keyword level hit by the user input. When it is judged as a second-level banned word, and the semantic recognition model judges that the dangerous probability of the text content exceeds 0.9, it is judged as banned content; when it is judged that the text contains third-level banned words, and the semantic recognition model judges that the dangerous probability of the text content exceeds 0.95, it is judged as banned content. Even if no banned words are hit, when the semantic recognition model judges that the dangerous probability of the text content exceeds 0.99, it will be judged as banned content.

[0077] Step S300: real-time verification.

[0078] In order to deal with various emergencies and uncontrollable situations, it is also necessary to add a real-time model output review mechanism.

[0079] This embodiment adopts a sentence slicing review mechanism, that is, the output answer of the model is reviewed once a complete sentence is generated. Once the text content is determined to be sensitive, the model stops streaming output, replies with a safety-preserving phrase, and creates a new conversation.

[0080] Another embodiment of the present application provides a large model domain security enhancement system based on a hybrid strategy, which is deployed at the interaction end of the large model. The system 20 includes:

[0081] The sensitive word definition module 201 can perform hierarchical sensitive word matching on the generated dialogue text based on the pre-defined sensitive word library and rule library to detect whether the information input by the user contains sensitive words or illegal content;

[0082] The semantic recognition module 202 can perform semantic recognition on the information input by the user based on the semantic recognition model of the pre-trained language model to determine whether it contains implicit security risks;

[0083] The real-time verification module 203 can use a semantic recognition model based on a pre-trained language model to conduct real-time review of the streaming output content of the large model, and cut off the streaming output in time if sensitive content occurs.

[0084] The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0085] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0086] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions without violating the spirit of the present application. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A large model domain security enhancement method based on a hybrid strategy, characterized in that: The following steps are involved: Based on the pre-defined sensitive word library and rule library, the generated dialogue text is matched with hierarchical sensitive words to detect whether the information input by the user contains sensitive words or illegal content; The semantic recognition model based on the pre-trained language model performs semantic recognition on the information input by the user to determine whether it contains implicit security risks; Use a semantic recognition model based on a pre-trained language model to conduct real-time review of the streaming output content of large models, and cut off the streaming output in time if sensitive content occurs.

2. The method according to claim 1, characterized in that: The semantic recognition model based on the pre-trained language model performs semantic recognition on the information input by the user to determine whether it contains implicit security risks, specifically including: The multiple sensitive word matching algorithm of the prefix tree algorithm is used to determine whether the information input by the user contains first-level, second-level, third-level banned words and regular expressions; When the algorithm determines that it is a first-level banned word, this method directly outputs that the content is banned; When the algorithm determines that it is a second-level banned word, the probability value of the text content is determined through the semantic recognition model. When the preset second-level banned probability threshold is reached, it is determined to be banned content; When the algorithm determines that the text contains level 3 banned words, the probability value of the text is determined through the semantic recognition model. When the preset level 3 banned probability threshold is reached, it is determined to be banned content.

3. The method according to claim 2, characterized in that The step of determining whether the information input by the user contains first-level, second-level, third-level prohibited words and regular expressions by using the multiple sensitive word matching algorithm of the prefix tree algorithm specifically includes: Classify existing deformation-sensitive words; Preprocess the information input by the user according to the classification of the deformed sensitive words to obtain the preprocessed sensitive words; Based on the preprocessed sensitive words, a prefix tree algorithm is used to determine whether the preprocessed sensitive words contain first-level, second-level, third-level banned words and regular expressions.

4. The method according to claim 2, characterized in that: The prefix tree is constructed based on pinyin, and 24 child nodes are created under the root node, among which nodes 0 to 22 respectively store Chinese sensitive words with their first letter pinyin, and node 23 stores sensitive words starting with numbers or other non-Chinese and English characters, and marks whether the node is a terminal node while storing the Chinese characters and their pinyin; the root node and the first-level child nodes are fixed.

5. The method according to claim 3, characterized in that: The existing sensitive words generally include three categories: the first category is sensitive words with special characters added; the second category is sensitive words deformed by using pinyin, homophones, homophones, etc.; the third category is sensitive words that have been split and converted to traditional Chinese.

6. The method according to claim 5, characterized in that The step of preprocessing the information input by the user according to the classification of the deformed sensitive words to obtain the preprocessed sensitive words specifically includes: For the first category, if multiple special characters are inserted into a sensitive word, the special characters are replaced with "*"; For the second category, when encountering a continuous pinyin string, the sensitive word filtering algorithm uses a regular expression to segment the pinyin string based on the maximum matching principle. When matching the pinyin, the node array is used to store the matched nodes, and the pre array is used to store all the nodes corresponding to the pinyin. If there is a successfully matched node in the previous round, it will be stored in the node. The current round of matching will start from the child nodes in the pre array until the terminal node is matched; The third category of sensitive words will be directly separated: for English strings, there are 409 common Chinese pinyin combinations; if the English string is a pinyin string, it will be separated into single pinyin; if the English string cannot be separated into pinyin, it will be separated into single English characters.

7. The method according to claim 1, characterized in that The pre-trained language model is a RoBERTa model, which adopts a Transformer structure and consists of multiple encoders. Each encoder consists of two sub-modules: a multi-head self-attention mechanism module and a forward feedback neural network module.

8. The method according to claim 7 and claim 3, characterized in that: The pre-trained language model sets different thresholds for the semantic recognition model according to the level of keywords hit by user input; when it is judged to be a second-level banned word, and the semantic recognition model judges that the dangerous probability of the text content exceeds 0.9, it is judged as banned content; when it is judged that the text has third-level banned words, and the semantic recognition model judges that the dangerous probability of the text content exceeds 0.95, it is judged as banned content; even if no banned words are hit, when the semantic recognition model judges that the dangerous probability of the text content exceeds 0.99, it will also be judged as banned content.

9. The method according to claim 1, characterized in that: The step of using a semantic recognition model based on a pre-trained language model to conduct real-time auditing of the streaming output content of the large model and promptly cutting off the streaming output if sensitive content occurs specifically includes: The model performs a content review every time a complete sentence is generated. Once the text content is determined to be sensitive, the model stops streaming output, responds with a safety-preserving script, and creates a new conversation.

10. A large model domain security enhancement system based on a hybrid strategy, deployed at the interaction end of the large model, specifically includes: The sensitive word definition module can perform hierarchical sensitive word matching on the generated dialogue text based on the pre-defined sensitive word library and rule library to detect whether the information input by the user contains sensitive words or illegal content; The semantic recognition module can perform semantic recognition on the information input by the user based on the semantic recognition model of the pre-trained language model to determine whether it contains implicit security risks; The real-time verification module can use a semantic recognition model based on a pre-trained language model to conduct real-time review of the streaming output content of the large model, and cut off the streaming output in time if sensitive content occurs.

Images