Large language model input content compliance detection method, device and equipment

Abstract

The invention discloses a large language model input content compliance detection method, device and equipment. The method comprises the steps of processing text content based on a BERT model, preliminarily judging whether sensitive words exist in the text content in combination with context semantics, calling a GPT model to further extract the sensitive words in the text content when the sensitive words exist in the text content, returning prompt information of rejecting generation of the content to a user when the sensitive words are extracted, and returning prompt information of rejecting generation of the content to the user when the sensitive words are not extracted. The text content is input into the large language model to be processed, the content corresponding to the text content is generated and returned to the user, and the sensitive word detection efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to natural language processing technology, and in particular to a method, device and equipment for detecting compliance of input content of a large language model. Background Art

[0002] The booming development of generative artificial intelligence (AIGC) has significantly boosted overall societal productivity. Large language models (LLMs) are a leading example of AIGC applications. Currently, LLMs possess considerable capabilities across multiple modalities, including text, images, and audio. They are well-suited to identifying and meeting the common needs of human users, effectively reducing manual production and service costs.

[0003] Large language models typically perform inference based on user input and generate corresponding content to return to the user. However, if the user input contains sensitive words, this may contaminate the large language model and cause it to output illegal content.

[0004] Currently, most methods use keyword matching to intercept sensitive words in user input. However, this method requires the establishment of a huge sensitive word library, which has low processing efficiency. In addition, the keyword matching method does not consider the contextual semantics and is prone to false detection or missed detection. Summary of the Invention

[0005] The present invention provides a method, device and equipment for detecting compliance of input content of a large language model to improve the efficiency and accuracy of sensitive word detection.

[0006] In a first aspect, the present invention provides a method for detecting compliance of input content of a large language model, comprising:

[0007] Get the text content entered by the user;

[0008] Process the text content based on the BERT model and determine whether sensitive words exist in the text content in combination with contextual semantics;

[0009] When sensitive words exist in the text content, sensitive words are extracted from the text content based on the GPT model;

[0010] When sensitive words are extracted, a prompt message is returned to the user to refuse to generate content;

[0011] When no sensitive words are extracted, the text content is input into a large language model for processing, and content corresponding to the text content is generated and returned to the user.

[0012] Optionally, the large language model input content compliance detection method also includes:

[0013] When there are no sensitive words in the text content, the text content is input into the large language model for processing, and content corresponding to the text content is generated and returned to the user.

[0014] Optionally, processing the text content based on a BERT model and determining whether sensitive words are present in the text content in combination with contextual semantics may include:

[0015] Convert the word segments in the text content into vector expressions to obtain a first word embedding vector;

[0016] Converting the position of a word in the text content into a vector expression to obtain a first position embedding vector;

[0017] Fusing the first word embedding vector and the first position embedding vector to obtain a first fused vector;

[0018] Inputting the first fused vector into an encoding layer for processing to obtain a first encoding vector, wherein the encoding layer includes multiple stacked multi-head attention units, and the output of the previous multi-head attention unit is used as the input of the next multi-head attention unit;

[0019] The first encoding vector is mapped to a result label of whether the text content contains sensitive words.

[0020] Optionally, before inputting the first fused vector into the coding layer for processing, the method further includes:

[0021] Using a keyword matching algorithm to match sensitive words with literal meanings from the text content;

[0022] Calculate the sum of the weights of all sensitive words to obtain the total weight of the text content, wherein the weights of the sensitive words are obtained by pre-training the BERT model;

[0023] The first fusion vector is weighted by using the total weight, and the weighted first fusion vector is used as an input of the coding layer.

[0024] Optionally, before inputting the first fused vector into the coding layer for processing, the method further includes:

[0025] Determining the input frequency of the text content based on historical input text;

[0026] Calculating the inverse of the input frequency as an attention coefficient for the text content;

[0027] The first fusion vector is adjusted using the attention coefficient, and the adjusted first fusion vector is used as an input of the coding layer.

[0028] Optionally, before inputting the first fused vector into the coding layer for processing, the method further includes:

[0029] Using a keyword matching algorithm to match literal sensitive words from the text content;

[0030] Calculate the sum of the weights of all sensitive words to obtain the total weight of the text content, wherein the weights of the sensitive words are obtained by pre-training the BERT model;

[0031] weighting the first fusion vector using the total weight;

[0032] Determining the input frequency of the text content based on historical input text;

[0033] Calculating the inverse of the input frequency as an attention coefficient for the text content;

[0034] The weighted first fusion vector is adjusted using the attention coefficient, and the adjusted first fusion vector is used as the input of the encoding layer.

[0035] Optionally, before obtaining the text content entered by the user, it also includes:

[0036] Obtain a batch of training text samples from a training text sample set, where the batch of training text samples includes multiple training text samples;

[0037] For each of the training text samples, converting the word segments in the training text sample into vector expressions to obtain a second word embedding vector;

[0038] Converting the position of a word in the training text sample into a vector expression to obtain a second position embedding vector;

[0039] Fusing the second word embedding vector and the second position embedding vector to obtain a second fused vector;

[0040] Using a keyword matching algorithm to match literal sensitive words from the training text sample;

[0041] Assigning initial values ​​to the weights of sensitive words, calculating the sum of the weights of all sensitive words, and obtaining the total weight of the training text sample;

[0042] weighting the second fusion vector using the total weight of the training text samples;

[0043] Determining an input frequency of the training text sample based on a training round of the training text sample;

[0044] Calculating the inverse of the input frequency of the training text sample as the attention coefficient of the training text sample;

[0045] Adjusting the weighted second fusion vector using the attention coefficient of the training text sample;

[0046] Inputting the adjusted second fusion vector into the encoding layer for processing to obtain an encoding vector;

[0047] Mapping the encoding vector of the training text sample to a result label indicating the presence of sensitive words in the training text sample;

[0048] Calculate the prediction accuracy of the BERT model for this batch of training text samples;

[0049] When the accuracy rate is less than a preset value, the weight of the sensitive word is adjusted along the gradient descent direction, and the step of obtaining a batch of training text samples from the training text sample set is returned to be executed until the accuracy rate is greater than or equal to the preset value, and the weight of the sensitive word is determined.

[0050] In some embodiments of the present invention, the large language model input content compliance detection method further includes:

[0051] When the text content input by the user is input for the first round of dialogue, the basic BERT model is called to process the text content and determine the professional field to which the text content belongs;

[0052] When the text content input by the user is input for the second round or later of dialogue, the BERT model of the professional field is called to process the text content.

[0053] In a second aspect, the present invention further provides a large language model input content compliance detection device, comprising:

[0054] The text content acquisition module is used to obtain the text content input by the user;

[0055] A sensitive word determination module is used to process the text content based on the BERT model and determine whether sensitive words exist in the text content in combination with contextual semantics;

[0056] A sensitive word extraction module is used to extract sensitive words from the text content based on the GPT model when sensitive words exist in the text content;

[0057] The prompt information return module is used to return a prompt information to the user to refuse to generate content when sensitive words are extracted;

[0058] The text content input module is used to input the text content into the large language model for processing when no sensitive words are extracted, generate content corresponding to the text content, and return it to the user.

[0059] In a third aspect, the present invention further provides an electronic device, comprising:

[0060] one or more processors;

[0061] a storage device for storing one or more programs;

[0062] When the one or more programs are executed by the one or more processors, the one or more processors implement the large language model input content compliance detection method provided in the first aspect of the present invention.

[0063] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large language model input content compliance detection method provided in the first aspect of the present invention.

[0064] The large language model input content compliance detection method provided by the present invention processes text content based on the BERT model, and preliminarily determines whether there are sensitive words in the text content in combination with contextual semantics. When sensitive words exist in the text content, the GPT model is called to further extract sensitive words from the text content. When sensitive words are extracted, a prompt message of refusing to generate content is returned to the user. When no sensitive words are extracted, the text content is input into the large language model for processing, content corresponding to the text content is generated, and returned to the user, thereby improving the efficiency and accuracy of sensitive word detection.

[0065] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0067] Figure 1 This is a flow chart of a method for detecting compliance of input content of a large language model provided by the present invention;

[0068] Figure 2 A schematic diagram of the coding layer structure of a BERT model provided by the present invention;

[0069] Figure 3 This is a schematic diagram of the structure of a large language model input content compliance detection device provided by the present invention;

[0070] Figure 4 A schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0071] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0072] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0073] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0074] The acquisition, storage, use, and processing of data in this application's technical solution comply with relevant national laws and regulations.

[0075] Figure 1 This is a flow chart of a method for compliance detection of input content of a large language model provided by the present invention. This embodiment can be applied to compliance detection of input content of a large language model to improve detection efficiency and accuracy. The method can be executed by the large language model input content compliance detection device provided by the present invention. The device can be implemented by software and / or hardware and is usually configured in an electronic device, such as Figure 1 As shown, the large language model input content compliance detection method includes the following steps:

[0076] S101: Obtain text content input by the user.

[0077] In an embodiment of the present invention, the text content can be directly input by the user, or it can be converted and extracted from other content input by the user. For example, the user inputs voice, pictures, etc., and text content can be extracted from the voice and pictures. The present invention does not limit this.

[0078] S102: Process the text content based on the BERT model and determine whether sensitive words exist in the text content in combination with the contextual semantics.

[0079] In this embodiment of the present invention, the BERT (Bidirectional Encoder Representations from Transformers) model is used to process text content. This model, combined with the contextual semantics of the text content, determines the presence of sensitive words at a semantic level. Traditional language models process text sequentially, from left to right or right to left. This approach limits the model's perception to the immediate context preceding the target word. BERT uses a bidirectional approach, considering both the left and right contexts of a word in a sentence. Instead of analyzing text sequentially, BERT examines all words in a sentence simultaneously.

[0080] In some embodiments of the present invention, processing text content based on the BERT model and determining whether sensitive words exist in the text content in combination with contextual semantics includes the following sub-steps:

[0081] S1021. Convert the word segments in the text content into vector expressions to obtain a first word embedding vector.

[0082] In this embodiment of the present invention, the BERT model's input layer segments the text content and performs a token embedding operation on the tokens, converting the tokens in the text content into vector representations to obtain a first token embedding vector. Token embedding involves converting each word into a fixed-dimensional token embedding vector using a dictionary encoding.

[0083] S1022: Convert the position of the word in the text content into a vector expression to obtain a first position embedding vector.

[0084] In this embodiment of the present invention, the BERT model's input layer performs a position embedding operation on the words in the text content, converting the word's position within the text content into a vector representation to obtain a first position embedding vector. Position embedding involves numbering each word in the text content and assigning each number to a vector. By combining the position vector with the word vector, each word is given a certain amount of position information.

[0085] S1023: Fuse the first word embedding vector and the first position embedding vector to obtain a first fused vector.

[0086] In an embodiment of the present invention, the first word embedding vectors of all word segments are spliced ​​according to the order of the word segments in the text content, and the first position embedding vectors of all word segments are spliced ​​according to the order of the word segments in the text content, and then the two spliced ​​vectors are aligned and added at corresponding positions to obtain a first fusion vector.

[0087] In some embodiments of the present invention, before executing step S1024 and inputting the first fused vector into the coding layer for processing, the method further includes:

[0088] 1. Use keyword matching algorithm to match sensitive words with literal meaning from text content.

[0089] For example, an existing keyword matching algorithm is used to match literal sensitive words from the text content. Since the keyword matching algorithm does not consider the contextual semantic information of the word segmentation, the matched sensitive words are literal sensitive words.

[0090] 2. Calculate the sum of the weights of all sensitive words to obtain the total weight of the text content. The weights of sensitive words are obtained by pre-training the BERT model.

[0091] Exemplarily, the sum of the weights of all matched sensitive words is calculated to obtain the total weight of the text content, where the weight of the sensitive word is obtained by pre-training the BERT model. The higher the weight, the higher the probability that the sensitive word belongs to a sensitive word at the semantic level.

[0092] 3. Use the total weight to weight the first fusion vector, and use the weighted first fusion vector as the input of the coding layer.

[0093] Exemplarily, the total weight is multiplied by the first fusion vector for weighting, and the weighted first fusion vector is used as the input of the encoding layer.

[0094] In an embodiment of the present invention, a keyword matching algorithm is used to match sensitive words with literal meanings from the text content, and the sum of the weights of all matched sensitive words is calculated to obtain the total weight of the text content, wherein the weight of the sensitive words is obtained by pre-training the BERT model, and then the total weight is multiplied by the first fusion vector for weighting, and the weighted first fusion vector is used as the input of the encoding layer. By weighting the first fusion vector, the prediction accuracy of the subsequent BERT model can be improved.

[0095] In some embodiments of the present invention, before executing step S1024 and inputting the first fused vector into the coding layer for processing, the method further includes:

[0096] 1. Determine the input frequency of text content based on historical input text.

[0097] In the embodiment of the present invention, a fuzzy semantic matching algorithm can be used, and two text contents with different literal meanings but the same semantics can be regarded as the same text content. In the embodiment of the present invention, the input frequency of the current text content is determined based on the historical input text.

[0098] 2. Calculate the inverse of the input frequency as the attention coefficient for the text content.

[0099] For example, the inverse of the input frequency is calculated as the attention coefficient for the text content. Assuming that most of the system is used normally, with only a few users mistakenly entering sensitive content, the input frequency of the first tagged content is counted (if it has not been counted before, it can be assumed to be 1 or another default value). The inverse of the input frequency is used as the attention coefficient for the text content. The lower the input frequency, the larger the attention coefficient, emphasizing the need to pay special attention to sensitive content with unusually low input frequency.

[0100] 3. Use the attention coefficient to adjust the first fusion vector, and use the adjusted first fusion vector as the input of the encoding layer.

[0101] Exemplarily, the attention coefficient is multiplied by the first fusion vector, the first fusion vector is adjusted, and the adjusted first fusion vector is used as the input of the encoding layer.

[0102] In an embodiment of the present invention, the inverse of the input frequency of the text content is calculated as the attention coefficient for the text content, and the attention coefficient is used to adjust the first fusion vector, and the adjusted first fusion vector is used as the input of the encoding layer, emphasizing the need to pay special attention to sensitive content with abnormally low input frequency, which can improve the accuracy of sensitive word detection in text content with low input frequency.

[0103] In some embodiments of the present invention, before executing step S1024 and inputting the first fused vector into the coding layer for processing, the method further includes:

[0104] 1. Use keyword matching algorithm to match sensitive words with literal meaning from text content.

[0105] For example, an existing keyword matching algorithm is used to match literal sensitive words from the text content. Since the keyword matching algorithm does not consider the contextual semantic information of the word segmentation, the matched sensitive words are literal sensitive words.

[0106] 2. Calculate the sum of the weights of all sensitive words to obtain the total weight of the text content. The weights of sensitive words are obtained by pre-training the BERT model.

[0107] Exemplarily, the sum of the weights of all matched sensitive words is calculated to obtain the total weight of the text content, where the weight of the sensitive word is obtained by pre-training the BERT model. The higher the weight, the higher the probability that the sensitive word belongs to a sensitive word at the semantic level.

[0108] 3. Use the total weight to weight the first fusion vector.

[0109] Exemplarily, weighting is performed by multiplying the total weight by the first fusion vector.

[0110] 4. Determine the input frequency of the text content based on historical input text.

[0111] In the embodiment of the present invention, a fuzzy semantic matching algorithm can be used, and two text contents with different literal meanings but the same semantics can be regarded as the same text content. In the embodiment of the present invention, the input frequency of the current text content is determined based on the historical input text.

[0112] 5. Calculate the inverse of the input frequency as the attention coefficient for the text content.

[0113] For example, the inverse of the input frequency is calculated as the attention coefficient for the text content. The lower the input frequency, the greater the attention coefficient, emphasizing that sensitive content with abnormally low input frequency needs to be paid special attention.

[0114] 6. Use the attention coefficient to adjust the weighted first fusion vector, and use the adjusted first fusion vector as the input of the encoding layer.

[0115] Exemplarily, the weighted first fusion vector is multiplied by the attention coefficient, the weighted first fusion vector is adjusted, and the adjusted first fusion vector is used as the input of the encoding layer.

[0116] In an embodiment of the present invention, a keyword matching algorithm is used to match literal sensitive words from the text content, and the sum of the weights of all matched sensitive words is calculated to obtain the total weight of the text content, wherein the weight of the sensitive words is obtained by pre-training the BERT model, and then the total weight is multiplied by the first fusion vector for weighting, and the inverse of the input frequency of the text content is calculated as the attention coefficient for the text content, and the attention coefficient is used to adjust the weighted first fusion vector, and the adjusted first fusion vector is used as the input of the encoding layer. By weighting the first fusion vector, the prediction accuracy of the subsequent BERT model can be improved. At the same time, it is emphasized that attention should be paid to sensitive content with abnormally low input frequency, which can improve the accuracy of sensitive word detection in text content with low input frequency.

[0117] In an embodiment of the present invention, the BERT model can be trained in advance to determine the weights of sensitive words. For example, the training process is as follows:

[0118] 1. Obtain a batch of training text samples from a training text sample set, where a batch of training text samples includes multiple training text samples.

[0119] In an embodiment of the present invention, a training text sample set is prepared in advance. The training text sample set includes multiple batches of training text samples, and each batch of training text samples includes multiple training text samples. The training text samples are annotated with the presence of sensitive words. During training, a batch of training text samples is obtained from the training text sample set each time.

[0120] 2. For each training text sample, convert the word segmentation in the training text sample into a vector expression to obtain the second word embedding vector.

[0121] For each training text sample, the word segments in the training text sample are converted into vector expressions to obtain a second word embedding vector. The specific process can refer to the process of converting the word segments in the text content into vector expressions in the aforementioned embodiment, and the present invention will not repeat it here.

[0122] 3. Convert the position of the word in the training text sample into a vector expression to obtain the second position embedding vector.

[0123] For each training text sample, the position of the word segment in the training text sample in the text content is converted into a vector representation to obtain a first position embedding vector. The specific process can be referred to the process of converting the position of the word segment in the text content into a vector representation in the previous embodiment, and the present invention will not repeat it here.

[0124] 4. Fuse the second word embedding vector and the second position embedding vector to obtain the second fused vector.

[0125] In an embodiment of the present invention, the second word embedding vectors of all word segmentations are spliced ​​according to the order of the word segmentations in the training text sample, and the second position embedding vectors of all word segmentations are spliced ​​according to the order of the word segmentations in the training text sample, and then the two spliced ​​vectors are aligned and added at corresponding positions to obtain a second fusion vector.

[0126] 5. Use keyword matching algorithm to match sensitive words with literal meaning from training text samples.

[0127] For example, an existing keyword matching algorithm is used to match literal sensitive words from training text samples.

[0128] 6. Assign initial values ​​to the weights of sensitive words, calculate the sum of the weights of all sensitive words, and obtain the total weight of the training text sample.

[0129] In an embodiment of the present invention, an initial value is assigned to the weight of the sensitive word, and the sum of the weights of all sensitive words is calculated to obtain the total weight of the training text sample.

[0130] 7. Use the total weight of the training text samples to weight the second fusion vector.

[0131] Exemplarily, weighting is performed by multiplying the total weight of the training text sample by the second fusion vector.

[0132] 8. Determine the input frequency of the training text sample based on the training rounds of the training text sample.

[0133] In an embodiment of the present invention, the input frequency of the training text sample is determined based on the training rounds of the training text sample.

[0134] 9. Calculate the inverse of the input frequency of the training text sample as the attention coefficient for the training text sample.

[0135] For example, the inverse of the input frequency of the training text sample is calculated as the attention coefficient for the training text sample. The lower the input frequency, the larger the attention coefficient, emphasizing that sensitive content with abnormally low input frequency needs to be paid special attention.

[0136] 10. Use the attention coefficient of the training text sample to adjust the weighted second fusion vector.

[0137] Exemplarily, the weighted second fusion vector is multiplied by the attention coefficient, the weighted second fusion vector is adjusted, and the adjusted second fusion vector is used as the input of the encoding layer.

[0138] 11. Input the adjusted second fusion vector into the coding layer for processing to obtain a coding vector.

[0139] Exemplarily, the adjusted second fusion vector is input to the coding layer for processing to obtain a coding vector. Specifically, the coding layer processing process can refer to the coding layer processing process of the first fusion vector in subsequent embodiments, and the present invention will not be repeated here.

[0140] 12. Map the encoding vector of the training text sample to the result label of the sensitive word in the training text sample.

[0141] In an embodiment of the present invention, the second encoding vector is input into a classifier for classification, and the second encoding vector is mapped to a result label indicating whether the training text sample contains a sensitive word. For example, if a sensitive word is determined to be present, the classifier outputs a result label of "1", and if a sensitive word is determined not to be present, the classifier outputs a result label of "0".

[0142] 13. Calculate the prediction accuracy of the BERT model for this batch of training text samples.

[0143] For each batch of training text samples, the prediction accuracy of the BERT model for the batch of training text samples is counted.

[0144] 14. When the accuracy is less than the preset value, adjust the weight of the sensitive word along the gradient descent direction, and return to the step of obtaining a batch of training text samples from the training text sample set until the accuracy is greater than or equal to the preset value, and determine the weight of the sensitive word at this time.

[0145] S1024. Input the first fusion vector into the encoding layer for processing to obtain a first encoding vector. The encoding layer includes multiple stacked multi-head attention units, and the output of the previous multi-head attention unit serves as the input of the next multi-head attention unit.

[0146] In an embodiment of the present invention, the first fusion vector is input into the encoding layer of the BERT model for processing. The encoding layer of the BERT model encodes the first fusion vector based on a multi-head self-attention mechanism to obtain a first encoded vector.

[0147] Figure 2 A schematic diagram of the coding layer of a BERT model provided by the present invention is shown in FIG. Figure 2 As shown in the figure, the encoding layer of the BERT model includes N layers of multi-head attention units stacked in sequence, where N is a positive integer greater than or equal to 2. The output of the previous multi-head attention unit is used as the input of the next multi-head attention unit, and the output of the last multi-head attention unit is used as the first encoding vector.

[0148] For example, Figure 2As shown in the figure, the multi-head attention unit includes a multi-head attention layer (Multi-HeadAttention Layer), a normalization layer (Norm Layer), a fully connected layer (FFN Layer) and a normalization layer (NormLayer). The multi-head attention layer processes the input vector based on the multi-head attention mechanism to obtain an attention vector. The normalization layer normalizes the attention vector to obtain a first normalized vector, and then adds the first normalized vector to the corresponding elements of the input vector of the multi-head attention layer to obtain a second fusion vector. The second fusion vector is sent to the fully connected layer for full connection mapping to obtain a fully connected vector. The normalization layer normalizes the fully connected vector to obtain a second normalized vector, and then adds the second normalized vector to the corresponding elements of the second fusion vector to obtain the output vector of the multi-head attention unit.

[0149] S1025. Map the first encoding vector to a result label of whether the text content contains sensitive words.

[0150] In this embodiment of the present invention, the first encoding vector is input into a pre-trained classifier for classification, and the first encoding vector is mapped to a result label indicating whether the text content contains sensitive words. For example, if a sensitive word is determined to be present, the classifier outputs a result label of "1", and if a sensitive word is determined not to be present, the classifier outputs a result label of "0". If a sensitive word is determined to be present, step S103 is executed; if a sensitive word is determined not to be present, step S105 is executed.

[0151] S103. Extract sensitive words from the text content based on the GPT model.

[0152] In step S102, if sensitive words are determined to be present in the text content, sensitive word extraction is performed based on the GPT (Generative Pre-trained Transformer) model. The GPT model, pre-trained on a large-scale corpus, learns the statistical laws of language. This allows it to not only distinguish the categories of user questions but also extract sensitive words from them. However, the GPT model places significant demands on backend server computing power, resulting in long response times and prolonged user wait times. Therefore, in this embodiment of the present invention, the GPT model is only invoked to further detect and extract sensitive words if the BERT model determines that sensitive words are present in the input text content. If a sensitive word is extracted, step S104 is executed; if not, step S105 is executed.

[0153] S104: Return a prompt message to the user indicating that the generated content has been rejected.

[0154] When the GPT model extracts sensitive words, a prompt message is returned to the user to refuse to generate content.

[0155] S105: Input the text content into the large language model for processing, generate content corresponding to the text content, and return it to the user.

[0156] When the GPT model does not extract sensitive words, the text content is input into the large language model for processing, and the content corresponding to the text content is generated and returned to the user.

[0157] The large language model input content compliance detection method provided by the present invention processes text content based on the BERT model, and preliminarily determines whether there are sensitive words in the text content in combination with contextual semantics. When sensitive words exist in the text content, the GPT model is called to further extract sensitive words from the text content. When sensitive words are extracted, a prompt message of refusing to generate content is returned to the user. When no sensitive words are extracted, the text content is input into the large language model for processing, content corresponding to the text content is generated, and returned to the user, thereby improving the efficiency and accuracy of sensitive word detection.

[0158] In some embodiments of the present invention, a basic BERT model can be set up. The basic BERT model is a general model that does not involve application fields. A corresponding BERT model can also be trained for each specific field, for example, a BERT model for government services and a BERT model for party building.

[0159] The interactive dialogue between the user and the large language model is usually in the form of multiple rounds of dialogue. When the text content input by the user is the input for the first round of dialogue, the basic BERT model is called to process the text content and determine the professional field to which the text content belongs. Specifically, the professional field to which the text content belongs can be determined by keyword matching. When the text content input by the user is the input for the second round and subsequent dialogues, the BERT model of the professional field is called to process the text content. The present invention calls the basic BERT model in the first round of dialogue to improve the model response speed and reduce the user waiting time. After determining the professional field to which the dialogue belongs, the BERT model of the professional field is called to process the text content, which can improve the prediction accuracy of the BERT model.

[0160] In some embodiments of the present invention, after sensitive words are extracted, they are stored in a sensitive word library as a data set for training a BERT model, thereby achieving self-iteration of the BERT model.

[0161] Figure 3 This is a structural diagram of a large language model input content compliance detection device provided by the present invention, such as Figure 3 As shown, the large language model input content compliance detection device includes:

[0162] The text content acquisition module 101 is used to acquire the text content input by the user;

[0163] Sensitive word determination module 102, configured to process the text content based on the BERT model and determine whether sensitive words exist in the text content in combination with contextual semantics;

[0164] The sensitive word extraction module 103 is used to extract sensitive words from the text content based on the GPT model when sensitive words exist in the text content;

[0165] The prompt information return module 104 is used to return a prompt information to the user to refuse to generate content when sensitive words are extracted;

[0166] The text content input module 105 is used to input the text content into the large language model for processing when no sensitive words are extracted, generate content corresponding to the text content, and return it to the user.

[0167] In some embodiments of the present invention, the text content input module 105 is further configured to:

[0168] When there are no sensitive words in the text content, the text content is input into the large language model for processing, and content corresponding to the text content is generated and returned to the user.

[0169] In some embodiments of the present invention, the sensitive word determination module 102 includes:

[0170] A first word embedding submodule, configured to convert the word segments in the text content into vector expressions to obtain a first word embedding vector;

[0171] A first position embedding submodule, configured to convert the position of a word in the text content into a vector expression to obtain a first position embedding vector;

[0172] A first fusion submodule, configured to fuse the first word embedding vector and the first position embedding vector to obtain a first fused vector;

[0173] an encoding processing submodule, configured to input the first fused vector into an encoding layer for processing to obtain a first encoding vector, wherein the encoding layer includes a plurality of stacked multi-head attention units, where the output of the previous multi-head attention unit serves as the input of the next multi-head attention unit;

[0174] The label mapping submodule is used to map the first encoding vector to a result label of whether the text content contains sensitive words.

[0175] In some embodiments of the present invention, the sensitive word determination module 102 further includes:

[0176] A first keyword matching submodule is configured to use a keyword matching algorithm to match literal sensitive words from the text content before inputting the first fusion vector into the encoding layer for processing;

[0177] A first weight calculation submodule is configured to calculate the sum of the weights of all sensitive words to obtain a total weight of the text content, wherein the weights of the sensitive words are obtained by pre-training the BERT model;

[0178] The first weighting submodule is configured to weight the first fusion vector using the total weight, and use the weighted first fusion vector as input of the coding layer.

[0179] In some embodiments of the present invention, the sensitive word determination module 102 further includes:

[0180] A first input frequency determination submodule, configured to determine the input frequency of the text content based on historical input texts before inputting the first fusion vector into the encoding layer for processing;

[0181] A first attention coefficient calculation submodule, configured to calculate the inverse of the input frequency as an attention coefficient for the text content;

[0182] The first adjustment submodule is configured to adjust the first fusion vector by using the attention coefficient, and use the adjusted first fusion vector as the input of the coding layer.

[0183] In some embodiments of the present invention, the sensitive word determination module 102 further includes:

[0184] A second keyword matching submodule is configured to use a keyword matching algorithm to match literal sensitive words from the text content before inputting the first fusion vector into the encoding layer for processing;

[0185] A second weight calculation submodule is configured to calculate the sum of the weights of all sensitive words to obtain a total weight of the text content, wherein the weights of the sensitive words are obtained by pre-training the BERT model;

[0186] a second weighting submodule, configured to weight the first fusion vector using the total weight;

[0187] A second input frequency determination submodule, configured to determine the input frequency of the text content based on historical input text;

[0188] A second attention coefficient calculation submodule, configured to calculate the inverse of the input frequency as an attention coefficient for the text content;

[0189] The second adjustment submodule is configured to adjust the weighted first fusion vector by using the attention coefficient, and use the adjusted first fusion vector as the input of the coding layer.

[0190] In some embodiments of the present invention, the apparatus for detecting compliance of input content of a large language model further includes:

[0191] A sample acquisition module is used to acquire a batch of training text samples from a training text sample set before acquiring the text content input by the user, wherein the batch of training text samples includes multiple training text samples;

[0192] A word embedding module is used to convert the word segments in each training text sample into vector expressions to obtain a second word embedding vector;

[0193] A position embedding module, configured to convert the position of a word in the training text sample into a vector expression to obtain a second position embedding vector;

[0194] a fusion module, configured to fuse the second word embedding vector and the second position embedding vector to obtain a second fused vector;

[0195] A keyword matching module, configured to use a keyword matching algorithm to match literal sensitive words from the training text sample;

[0196] A weight calculation module is used to assign initial values ​​to the weights of sensitive words, calculate the sum of the weights of all sensitive words, and obtain the total weight of the training text sample;

[0197] a first weighting module, configured to weight the second fusion vector using the total weight of the training text sample;

[0198] A frequency determination module, configured to determine an input frequency of the training text sample based on a training round of the training text sample;

[0199] an attention coefficient calculation module, configured to calculate the inverse of the input frequency of the training text sample as the attention coefficient for the training text sample;

[0200] An adjustment module, configured to adjust the weighted second fusion vector using the attention coefficient of the training text sample;

[0201] An encoding module, configured to input the adjusted second fusion vector into an encoding layer for processing to obtain an encoding vector;

[0202] A label mapping module, configured to map the encoding vector of the training text sample to a result label of the presence of sensitive words in the training text sample;

[0203] The statistics module is used to calculate the prediction accuracy of the BERT model for the batch of training text samples;

[0204] The weight determination module is used to adjust the weight of the sensitive word along the gradient descent direction when the accuracy rate is less than a preset value, and return to execute the step of obtaining a batch of training text samples from the training text sample set until the accuracy rate is greater than or equal to the preset value, and determine the weight of the sensitive word.

[0205] In some embodiments of the present invention, the apparatus for detecting compliance of input content of a large language model further includes:

[0206] A first calling module is configured to, when the text content input by the user is input for the first round of dialogue, call a basic BERT model to process the text content and determine the professional field to which the text content belongs;

[0207] The second calling module is used to call the BERT model in the professional field to process the text content when the text content input by the user is the second round and subsequent dialogue input.

[0208] The above-mentioned large language model input content compliance detection device can execute the large language model input content compliance detection method provided by the aforementioned embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the large language model input content compliance detection method.

[0209] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided for example only and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0210] like Figure 4As shown, the electronic device includes at least one processor 11 and memory, such as a read-only memory (ROM) 12 and a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer programs stored in the ROM 12 or loaded from the storage unit 18 into the RAM 13. RAM 13 can also store various programs and data required for the operation of the electronic device. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0211] Multiple components in the electronic device are connected to the I / O interface 15, including an input unit 16, such as a keyboard, mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, optical disk, etc.; and a communication unit 19, such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0212] Processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any other suitable processor, controller, microcontroller, etc. Processor 11 executes the various methods and processes described above, such as the large language model input content compliance detection method.

[0213] In some embodiments, the large language model input content compliance detection method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the large language model input content compliance detection method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the large language model input content compliance detection method in any other appropriate manner (for example, by means of firmware).

[0214] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0215] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0216] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer-readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0217] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device that has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0218] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0219] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0220] An embodiment of the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the large language model input content compliance detection method provided in any embodiment of the present application.

[0221] During implementation, the computer program product may be written in one or more programming languages ​​or a combination thereof to perform the operations of the present invention. The programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0222] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0223] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for detecting compliance of input content of a large language model, characterized in that: include: Get the text content entered by the user; Process the text content based on the BERT model and determine whether sensitive words exist in the text content in combination with contextual semantics; When sensitive words exist in the text content, sensitive words are extracted from the text content based on the GPT model; When sensitive words are extracted, a prompt message is returned to the user to refuse to generate content; When no sensitive words are extracted, the text content is input into a large language model for processing, and content corresponding to the text content is generated and returned to the user.

2. The large language model input content compliance detection method according to claim 1 is characterized in that: Also includes: When there are no sensitive words in the text content, the text content is input into the large language model for processing, and content corresponding to the text content is generated and returned to the user.

3. The large language model input content compliance detection method according to claim 1 or 2, characterized in that: The text content is processed based on the BERT model, and the contextual semantics are combined to determine whether sensitive words are present in the text content, including: Convert the word segments in the text content into vector expressions to obtain a first word embedding vector; Converting the position of a word in the text content into a vector expression to obtain a first position embedding vector; Fusing the first word embedding vector and the first position embedding vector to obtain a first fused vector; Inputting the first fused vector into an encoding layer for processing to obtain a first encoding vector, wherein the encoding layer includes multiple stacked multi-head attention units, and the output of the previous multi-head attention unit is used as the input of the next multi-head attention unit; The first encoding vector is mapped to a result label of whether the text content contains sensitive words.

4. The large language model input content compliance detection method according to claim 3 is characterized in that: Before inputting the first fused vector into the coding layer for processing, the method further includes: Using a keyword matching algorithm to match sensitive words with literal meanings from the text content; Calculate the sum of the weights of all sensitive words to obtain the total weight of the text content, wherein the weights of the sensitive words are obtained by pre-training the BERT model; The first fusion vector is weighted by using the total weight, and the weighted first fusion vector is used as an input of the coding layer.

5. The large language model input content compliance detection method according to claim 3 is characterized in that: Before inputting the first fused vector into the coding layer for processing, the method further includes: Determining the input frequency of the text content based on historical input text; Calculating the inverse of the input frequency as an attention coefficient for the text content; The first fusion vector is adjusted using the attention coefficient, and the adjusted first fusion vector is used as an input of the coding layer.

6. The large language model input content compliance detection method according to claim 3 is characterized in that: Before inputting the first fused vector into the coding layer for processing, the method further includes: Using a keyword matching algorithm to match sensitive words with literal meanings from the text content; Calculate the sum of the weights of all sensitive words to obtain the total weight of the text content, wherein the weights of the sensitive words are obtained by pre-training the BERT model; weighting the first fusion vector using the total weight; Determining the input frequency of the text content based on historical input text; Calculating the inverse of the input frequency as an attention coefficient for the text content; The weighted first fusion vector is adjusted using the attention coefficient, and the adjusted first fusion vector is used as the input of the encoding layer.

7. The method for detecting compliance of input content of a large language model according to claim 6, characterized in that: Before getting the text content entered by the user, it also includes: Obtain a batch of training text samples from a training text sample set, where the batch of training text samples includes multiple training text samples; For each of the training text samples, converting the word segments in the training text sample into vector expressions to obtain a second word embedding vector; Converting the position of a word in the training text sample into a vector expression to obtain a second position embedding vector; Fusing the second word embedding vector and the second position embedding vector to obtain a second fused vector; Using a keyword matching algorithm to match literal sensitive words from the training text sample; Assigning initial values ​​to the weights of sensitive words, calculating the sum of the weights of all sensitive words, and obtaining the total weight of the training text sample; weighting the second fusion vector using the total weight of the training text samples; Determining an input frequency of the training text sample based on a training round of the training text sample; Calculating the inverse of the input frequency of the training text sample as the attention coefficient of the training text sample; Adjusting the weighted second fusion vector using the attention coefficient of the training text sample; Inputting the adjusted second fusion vector into the encoding layer for processing to obtain an encoding vector; Mapping the encoding vector of the training text sample to a result label indicating the presence of sensitive words in the training text sample; Calculate the prediction accuracy of the BERT model for this batch of training text samples; When the accuracy rate is less than a preset value, the weight of the sensitive word is adjusted along the gradient descent direction, and the step of obtaining a batch of training text samples from the training text sample set is returned to be executed until the accuracy rate is greater than or equal to the preset value, and the weight of the sensitive word is determined.

8. The method for detecting compliance of input content of a large language model according to claim 1, characterized in that: Also includes: When the text content input by the user is input for the first round of dialogue, the basic BERT model is called to process the text content and determine the professional field to which the text content belongs; When the text content input by the user is input for the second round or later of dialogue, the BERT model of the professional field is called to process the text content.

9. A large language model input content compliance detection device, characterized in that: include: The text content acquisition module is used to obtain the text content input by the user; A sensitive word determination module is used to process the text content based on the BERT model and determine whether sensitive words exist in the text content in combination with contextual semantics; A sensitive word extraction module is used to extract sensitive words from the text content based on the GPT model when sensitive words exist in the text content; The prompt information return module is used to return a prompt information to the user to refuse to generate content when sensitive words are extracted; The text content input module is used to input the text content into the large language model for processing when no sensitive words are extracted, generate content corresponding to the text content, and return it to the user.

10. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the large language model input content compliance detection method as described in any one of claims 1-8.

Images