Large language model input content compliance detection method, device and equipment

Abstract

The application discloses a large language model input content compliance detection method, device and equipment. The text content is processed based on a BERT model, and whether sensitive words exist in the text content is preliminarily judged in combination with context semantics. When the text content contains sensitive words, a GPT model is called to further extract sensitive words from the text content. When the sensitive words are extracted, prompt information for refusing to generate content is returned to the user. When no sensitive words are extracted, the text content is input into a large language model for processing, content corresponding to the text content is generated, and the content is returned to the user, thereby improving the sensitive word detection efficiency and accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to natural language processing technology, and in particular to a large language model input content compliance detection method, device and equipment. BACKGROUND

[0002] With the vigorous development of Artificial Intelligence Generated Content (AIGC), the overall social productivity has been effectively improved. Among them, the Large Language Model (LLM) is the leader in many AIGC applications. At present, the large language model has considerable ability in text, image, audio and other modal scenes. The large language model can well recognize and meet the general needs of human users, and can effectively reduce the cost of artificial production and service.

[0003] The large language model usually makes inferences based on the content input by the user and generates corresponding content to return to the user. However, if the content input by the user contains sensitive words, it may pollute the large language model, resulting in the output of illegal content by the large language model.

[0004] At present, the way of keyword matching is mostly used to intercept sensitive words in user input content, but this way needs to establish a large sensitive word library, and the processing efficiency is low, and the keyword matching method does not consider the context semantics, which is easy to cause false detection or missed detection. SUMMARY

[0005] The present application provides a large language model input content compliance detection method, device and equipment to improve the sensitivity of the detection efficiency and accuracy.

[0006] In a first aspect, the present application provides a large language model input content compliance detection method, comprising:

[0007] obtaining text content input by a user;

[0008] processing the text content based on a BERT model, and determining whether there is a sensitive word in the text content in combination with the context semantics;

[0009] when the text content contains a sensitive word, extracting the sensitive word from the text content based on a GPT model;

[0010] when the sensitive word is extracted, returning a prompt information of refusing to generate content to the user;

[0011] when the sensitive word is not extracted, inputting the text content into a large language model for processing, generating content corresponding to the text content, and returning the content to the user.

[0012] Optionally, the large language model input content compliance detection method further comprises:

[0013] When the text content does not contain sensitive words, inputting the text content into a large language model for processing to generate content corresponding to the text content and returning the content to the user.

[0014] Optionally, the text content is processed based on a BERT model to determine whether the text content contains sensitive words in combination with context semantics, comprising:

[0015] Converting word segmentation in the text content into vector representation to obtain a first word embedding vector;

[0016] Converting the position of the word segmentation in the text content in the text content into vector representation to obtain a first position embedding vector;

[0017] Fusing the first word embedding vector and the first position embedding vector to obtain a first fusion vector;

[0018] Inputting the first fusion vector into an encoding layer for processing to obtain a first encoding vector, the encoding layer comprising a plurality of stacked multi-head attention units, the output of a previous multi-head attention unit serving as the input of a subsequent multi-head attention unit;

[0019] Mapping the first encoding vector into a result label of whether the text content contains sensitive words.

[0020] Optionally, before inputting the first fusion vector into the encoding layer for processing, further comprising:

[0021] Using a keyword matching algorithm to match sensitive words of literal meaning from the text content;

[0022] Calculating the sum of the weights of all sensitive words to obtain the total weight of the text content, wherein the weight of the sensitive word is obtained by pre-training the BERT model;

[0023] Using the total weight to weight the first fusion vector, and using the weighted first fusion vector as the input of the encoding layer.

[0024] Optionally, before inputting the first fusion vector into the encoding layer for processing, further comprising:

[0025] Determining the input frequency of the text content based on historical input text;

[0026] Calculating the reciprocal of the input frequency as the attention coefficient of the text content;

[0027] The first fusion vector is adjusted using the attention coefficient, and the adjusted first fusion vector is taken as the input of the encoding layer.

[0028] Optionally, before the first fusion vector is input into the encoding layer for processing, the method further includes:

[0029] Sensitive words of literal meaning are matched from the text content using a keyword matching algorithm.

[0030] The sum of the weights of all sensitive words is calculated to obtain the total weight of the text content, wherein the weight of the sensitive word is obtained by pre-training the BERT model.

[0031] The first fusion vector is weighted using the total weight.

[0032] The input frequency of the text content is determined based on historical input text.

[0033] The reciprocal of the input frequency is calculated as the attention coefficient for the text content.

[0034] The weighted first fusion vector is adjusted using the attention coefficient, and the adjusted first fusion vector is taken as the input of the encoding layer.

[0035] Optionally, before the text content input by the user is obtained, the method further includes:

[0036] A batch of training text samples are obtained from a training text sample set, and the batch of training text samples includes a plurality of training text samples.

[0037] For each training text sample, the word segmentation in the training text sample is converted into a vector representation to obtain a second word embedding vector.

[0038] The position of the word segmentation in the training text sample in the training text sample is converted into a vector representation to obtain a second position embedding vector.

[0039] The second word embedding vector and the second position embedding vector are fused to obtain a second fusion vector.

[0040] Sensitive words of literal meaning are matched from the training text sample using a keyword matching algorithm.

[0041] An initial value is assigned to the weight of the sensitive word, and the sum of the weights of all sensitive words is calculated to obtain the total weight of the training text sample.

[0042] The second fusion vector is weighted using the total weight of the training text sample.

[0043] determining an input frequency of the training text sample based on a training round of the training text sample;

[0044] calculating a reciprocal of the input frequency of the training text sample as an attention coefficient of the training text sample;

[0045] adjusting the weighted second fusion vector using the attention coefficient of the training text sample;

[0046] inputting the adjusted second fusion vector into an encoding layer for processing to obtain an encoding vector;

[0047] mapping the encoding vector of the training text sample to a result label of the training text sample existing a sensitive word;

[0048] statistically determining a prediction accuracy of the BERT model for the batch of training text samples;

[0049] when the accuracy is less than a preset value, adjusting the weight of the sensitive word in a gradient descent direction, and returning to execute the step of obtaining a batch of training text samples from the training text sample set until the accuracy is greater than or equal to the preset value, and determining the weight of the sensitive word.

[0050] In some embodiments of the present application, the large language model input content compliance detection method further comprises:

[0051] when the text content input by the user is a first round of dialogue input, calling a basic BERT model to process the text content and determining a professional field to which the text content belongs;

[0052] when the text content input by the user is a second round and subsequent dialogue input, calling the BERT model of the professional field to process the text content.

[0053] In a second aspect, the present application further provides a large language model input content compliance detection device, comprising:

[0054] a text content acquisition module for acquiring text content input by a user;

[0055] a sensitive word determination module for processing the text content based on a BERT model and determining whether a sensitive word exists in the text content in combination with context semantics;

[0056] a sensitive word extraction module for extracting a sensitive word from the text content based on a GPT model when the text content contains a sensitive word;

[0057] a prompt information return module for returning prompt information of refusing to generate content to the user when a sensitive word is extracted;

[0058] The text content input module is configured to input the text content into the large language model for processing when no sensitive word is extracted, to generate content corresponding to the text content, and to return the content to the user.

[0059] In a third aspect, the present application further provides an electronic device, comprising:

[0060] one or more processors;

[0061] a storage device configured to store one or more programs;

[0062] When the one or more programs are executed by the one or more processors, the one or more processors implement the large language model input content compliance detection method provided in the first aspect of the present application.

[0063] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large language model input content compliance detection method provided in the first aspect of the present application.

[0064] The large language model input content compliance detection method provided by the present application processes the text content based on the BERT model, preliminarily judges whether there is a sensitive word in the text content in combination with the context semantics, calls the GPT model to further extract the sensitive word from the text content when the text content contains a sensitive word, returns a prompt information of refusing to generate content to the user when the sensitive word is extracted, inputs the text content into the large language model for processing when no sensitive word is extracted, generates the content corresponding to the text content, and returns the content to the user, thereby improving the sensitive word detection efficiency and accuracy.

[0065] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0067] Figure 1 A flowchart of a large language model input content compliance detection method provided by the present application;

[0068] Figure 2 A structural diagram of an encoding layer of a BERT model provided by the present application;

[0069] Figure 3 A structural schematic diagram of a large language model input content compliance detection device provided by the present application is provided.

[0070] Figure 4 A structural schematic diagram of an electronic device provided by an embodiment of the present application is provided.

[0071] The specific embodiments have been shown and described in the above drawings, and will be described in more detail hereinafter. These drawings and the written description are not intended to restrict the scope of the present application in any way, but to illustrate the present application by reference to specific embodiments. DETAILED DESCRIPTION

[0072] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0073] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0074] The acquisition, storage, use, processing and the like of data in the technical solutions of the present application all comply with the relevant provisions of national laws and regulations.

[0075] Figure 1 A flowchart of a large language model input content compliance detection method provided by the present application, the present embodiment can be applicable to compliance detection of input content of a large language model, improve detection efficiency and accuracy, the method can be executed by a large language model input content compliance detection device provided by the present application, the device can be realized by software and / or hardware, and is usually configured in an electronic device, such as Figure 1 As shown, the large language model input content compliance detection method comprises the following steps:

[0076] S101, acquire text content input by a user.

[0077] In the embodiments of the present application, the text content can be directly input by the user, or can be extracted from other content input by the user, for example, the user inputs voice, pictures, etc., and the text content can be extracted from the voice and pictures, which is not limited in the present application.

[0078] S102, process the text content based on a BERT model, and determine whether there is a sensitive word in the text content in combination with context semantics.

[0079] In the embodiments of the present application, the BERT (Bidirectional Encoder Representations from Transformers) model is used to process the text content, and whether there is a sensitive word in the text content is determined from the semantic level in combination with the context semantics of the text content. The traditional language model processes the text in sequence from left to right or from right to left, and this method limits the perception of the model to the direct context before the target word. BERT uses a bidirectional method to consider the left and right context of the word segmentation in the sentence, rather than analyzing the text in sequence, and BERT simultaneously views all word segmentations in the sentence.

[0080] In some embodiments of the present application, the text content is processed based on the BERT model, and whether there is a sensitive word in the text content is determined in combination with the context semantics, including the following sub-steps:

[0081] S1021, convert the word segmentation in the text content into vector expression to obtain a first word embedding vector.

[0082] In the embodiments of the present application, the input layer of the BERT model performs word segmentation on the text content, and performs Token Embedding operation on the word segmentation, converts the word segmentation in the text content into vector expression, and obtains a first word embedding vector. Wherein, Token Embedding refers to representing each word with the encoding in the dictionary library, and converting it into a fixed-dimension word embedding vector.

[0083] S1022, convert the position of the word segmentation in the text content into vector expression to obtain a first position embedding vector.

[0084] In the embodiment of the present application, the input layer of the BERT model performs a position embedding operation on the word segmentation in the text content, converts the position of the word segmentation in the text content into a vector expression, and obtains a first position embedding vector. Position embedding refers to numbering each word segmentation in the text content, and then each number corresponds to a vector. By combining the position vector and the word vector, a certain position information is introduced to each word segmentation.

[0085] S1023, fuse the first word embedding vector and the first position embedding vector to obtain a first fusion vector.

[0086] In the embodiment of the present application, the first word embedding vectors of all word segmentations are spliced according to the order of the word segmentations in the text content, and the first position embedding vectors of all word segmentations are spliced according to the order of the word segmentations in the text content. Then, the two spliced vectors are aligned and added at the corresponding positions to obtain the first fusion vector.

[0087] In some embodiments of the present application, before performing step S1024, inputting the first fusion vector into the encoding layer for processing, it further includes:

[0088] 1. Using a keyword matching algorithm to match sensitive words of literal meaning from the text content.

[0089] For example, using an existing keyword matching algorithm to match sensitive words of literal meaning from the text content. Since the keyword matching algorithm does not consider the context semantic information of the word segmentation, the matched sensitive words belong to the sensitive words of literal meaning.

[0090] 2. Calculate the sum of the weights of all sensitive words to obtain the total weight of the text content, wherein the weight of the sensitive word is obtained by pre-training the BERT model.

[0091] For example, calculate the sum of the weights of all matched sensitive words to obtain the total weight of the text content, wherein the weight of the sensitive word is obtained by pre-training the BERT model, and the higher the weight, the higher the probability that the sensitive word belongs to the sensitive word of the semantic level.

[0092] 3. Weight the first fusion vector using the total weight, and input the weighted first fusion vector as the input of the encoding layer.

[0093] For example, multiply the first fusion vector by the total weight to weight, and input the weighted first fusion vector as the input of the encoding layer.

[0094] In the embodiment of the present application, the keyword matching algorithm is used to match the sensitive words with literal meaning from the text content, and the sum of the weights of all the matched sensitive words is calculated to obtain the total weight of the text content, wherein the weight of the sensitive word is obtained by pre-training the BERT model, then the total weight is multiplied by the first fusion vector for weighting, and the weighted first fusion vector is used as the input of the encoding layer, and the prediction accuracy of the subsequent BERT model can be improved by weighting the first fusion vector.

[0095] In some embodiments of the present application, before performing step S1024, inputting the first fusion vector into the encoding layer for processing, it further includes:

[0096] 1. Determine the input frequency of the text content based on the historical input text.

[0097] In the embodiment of the present application, the fuzzy semantic matching algorithm can be used, and the two text contents with different literal meanings but the same semantics can be regarded as the same text content. In the embodiment of the present application, the input frequency of the text content is determined based on the historical input text.

[0098] 2. Calculate the reciprocal of the input frequency as the attention coefficient of the text content.

[0099] For example, the reciprocal of the input frequency is calculated as the attention coefficient of the text content. Assuming that most of the system is used normally, only a small number of users input sensitive content incorrectly, at this time, the input frequency of the first label content is counted (if it has not been counted before, it can be assumed to be 1 or other default value), and the reciprocal of the input frequency is used as the attention coefficient of the text content. The smaller the input frequency, the greater the attention coefficient, and the sensitive content with abnormally small input frequency needs to be paid more attention to.

[0100] 3. Adjust the first fusion vector using the attention coefficient, and input the adjusted first fusion vector into the encoding layer.

[0101] For example, the first fusion vector is multiplied by the attention coefficient to adjust the first fusion vector, and the adjusted first fusion vector is input into the encoding layer.

[0102] In the embodiment of the present application, the reciprocal of the input frequency of the text content is calculated as the attention coefficient of the text content, the first fusion vector is adjusted using the attention coefficient, and the adjusted first fusion vector is input into the encoding layer, which emphasizes that the sensitive content with abnormally small input frequency needs to be paid more attention to, and the sensitive word detection accuracy of the text content with small input frequency can be improved.

[0103] In some embodiments of the present application, before performing step S1024, inputting the first fusion vector into the encoding layer for processing, it further includes:

[0104] 1. Matching the sensitive words in literal meaning from the text content by using a keyword matching algorithm.

[0105] For example, the sensitive words in literal meaning are matched from the text content by using an existing keyword matching algorithm. Since the keyword matching algorithm does not consider the context semantic information of the word segmentation, the matched sensitive words belong to the sensitive words in literal meaning.

[0106] 2. Calculate the sum of the weights of all sensitive words to obtain the total weight of the text content, wherein the weight of the sensitive word is obtained by pre-training the BERT model.

[0107] For example, the sum of the weights of all matched sensitive words is calculated to obtain the total weight of the text content, wherein the weight of the sensitive word is obtained by pre-training the BERT model, and the higher the weight, the higher the probability that the sensitive word belongs to the sensitive word in semantic level.

[0108] 3. Weighting the first fusion vector by using the total weight.

[0109] For example, the first fusion vector is weighted by using the total weight.

[0110] 4. Determining the input frequency of the text content based on the historical input text.

[0111] In the embodiment of the present application, the fuzzy semantic matching algorithm can be used, and two text contents with different literal meanings but the same semantics can be regarded as the same text content. In the embodiment of the present application, the input frequency of the text content is determined based on the historical input text.

[0112] 5. Calculating the reciprocal of the input frequency as the attention coefficient of the text content.

[0113] For example, the reciprocal of the input frequency is calculated as the attention coefficient of the text content, the input frequency is less, the attention coefficient is larger, and the sensitive content with abnormally low input frequency needs to be paid more attention to.

[0114] 6. Adjusting the weighted first fusion vector by using the attention coefficient, and taking the adjusted first fusion vector as the input of the encoding layer.

[0115] For example, the weighted first fusion vector is adjusted by using the attention coefficient, and the adjusted first fusion vector is taken as the input of the encoding layer.

[0116] In the embodiment of the present application, the keyword matching algorithm is used to match the sensitive words with literal meaning from the text content, and the sum of the weights of all the matched sensitive words is calculated to obtain the total weight of the text content, wherein the weight of the sensitive word is obtained by pre-training the BERT model, then the total weight is multiplied by the first fusion vector for weighting, the reciprocal of the input frequency of the text content is calculated as the attention coefficient of the text content, and the first fusion vector after weighting is adjusted by using the attention coefficient, and the adjusted first fusion vector is used as the input of the encoding layer, so that the prediction accuracy of the subsequent BERT model can be improved, and at the same time, the sensitive content with abnormally low input frequency needs to be paid attention to, so that the sensitive word detection accuracy of the text content with low input frequency can be improved.

[0117] In the embodiment of the present application, the BERT model can be pre-trained to determine the weight of the sensitive word. For example, the training process is as follows:

[0118] 1. A batch of training text samples is obtained from the training text sample set, and the batch of training text samples includes a plurality of training text samples.

[0119] In the embodiment of the present application, a training text sample set is prepared in advance, the training text sample set includes a plurality of batches of training text samples, and each batch of training text samples includes a plurality of training text samples. The training text sample is labeled with whether there is a sensitive word. During training, a batch of training text samples is obtained from the training text sample set each time.

[0120] 2. For each training text sample, the word segmentation in the training text sample is converted into vector expression to obtain a second word embedding vector.

[0121] For each training text sample, the word segmentation in the training text sample is converted into vector expression to obtain a second word embedding vector. The specific process can refer to the process of converting the word segmentation in the text content into vector expression in the foregoing embodiment, which will not be repeated here.

[0122] 3. The position of the word segmentation in the training text sample in the training text sample is converted into vector expression to obtain a second position embedding vector.

[0123] For each training text sample, the position of the word segmentation in the training text sample in the text content is converted into vector expression to obtain a first position embedding vector. The specific process can refer to the process of converting the position of the word segmentation in the text content in the text content into vector expression in the foregoing embodiment, which will not be repeated here.

[0124] 4. The second word embedding vector and the second position embedding vector are fused to obtain a second fusion vector.

[0125] In the embodiment of the present application, the second word embedding vectors of all the segmented words are spliced according to the order of the segmented words in the training text sample, and the second position embedding vectors of all the segmented words are spliced according to the order of the segmented words in the training text sample, then the two spliced vectors are aligned and the corresponding positions are added, to obtain the second fusion vector.

[0126] 5. The literal meaning sensitive words are matched from the training text sample by using a keyword matching algorithm.

[0127] For example, the literal meaning sensitive words are matched from the training text sample by using an existing keyword matching algorithm.

[0128] 6. An initial value is given to the weight of the sensitive word, the sum of the weights of all sensitive words is calculated, and the total weight of the training text sample is obtained.

[0129] In the embodiment of the present application, an initial value is given to the weight of the sensitive word, the sum of the weights of all sensitive words is calculated, and the total weight of the training text sample is obtained.

[0130] 7. The second fusion vector is weighted by using the total weight of the training text sample.

[0131] For example, the second fusion vector is weighted by using the total weight of the training text sample.

[0132] 8. The input frequency of the training text sample is determined based on the training round of the training text sample.

[0133] In the embodiment of the present application, the input frequency of the training text sample is determined based on the training round of the training text sample.

[0134] 9. The reciprocal of the input frequency of the training text sample is calculated as the attention coefficient of the training text sample.

[0135] For example, the reciprocal of the input frequency of the training text sample is calculated as the attention coefficient of the training text sample, the input frequency is less, the attention coefficient is greater, and the sensitive content with an abnormal small input frequency is emphasized and needs to be paid attention to.

[0136] 10. The weighted second fusion vector is adjusted by using the attention coefficient of the training text sample.

[0137] For example, the weighted second fusion vector is adjusted by using the attention coefficient, and the adjusted second fusion vector is used as the input of the encoding layer.

[0138] 11. The adjusted second fusion vector is input into the encoding layer for processing to obtain an encoding vector.

[0139] Exemplarily, the adjusted second fusion vector is input into an encoding layer for processing to obtain an encoding vector. Specifically, the processing procedure of the encoding layer can refer to the processing procedure of the first fusion vector by the encoding layer in subsequent embodiments, which will not be described herein again.

[0140] 12. mapping the encoding vector of the training text sample to a result label of the existence of the sensitive word in the training text sample.

[0141] In the embodiment of the present application, the second encoding vector is input into a classifier for classification, and the second encoding vector is mapped to a result label of the existence of the sensitive word in the training text sample. Exemplarily, if it is determined that the sensitive word exists, the classifier outputs the result label "1", and if it is determined that the sensitive word does not exist, the classifier outputs the result label "0".

[0142] 13. counting the prediction accuracy of the BERT model for the batch of training text samples.

[0143] For each batch of training text samples, the prediction accuracy of the BERT model for the batch of training text samples is counted.

[0144] 14. when the accuracy is less than the preset value, adjusting the weight of the sensitive word in the gradient descent direction, and returning to execute the step of obtaining a batch of training text samples from the training text sample set until the accuracy is greater than or equal to the preset value, and determining the weight of the sensitive word at this time.

[0145] S1024, input the first fusion vector into an encoding layer for processing to obtain a first encoding vector, the encoding layer comprising a plurality of stacked multi-head attention units, the output of a previous multi-head attention unit being input into a subsequent multi-head attention unit.

[0146] In the embodiment of the present application, the first fusion vector is input into the encoding layer of the BERT model for processing, and the encoding layer of the BERT model encodes the first fusion vector based on the multi-head self-attention mechanism to obtain the first encoding vector.

[0147] Figure 2 A structural diagram of the encoding layer of the BERT model provided by the present application is shown in FIG. 1. Figure 2 As shown in FIG. 1, the encoding layer of the BERT model comprises N layers of multi-head attention units stacked in sequence, N being a positive integer greater than or equal to 2. The output of a previous multi-head attention unit is input into a subsequent multi-head attention unit, and the output of the last multi-head attention unit is the first encoding vector.

[0148] Exemplarily, as shown in FIG. 2, the encoding layer of the BERT model comprises a first multi-head attention unit 201, a second multi-head attention unit 202, a third multi-head attention unit 203, and a fourth multi-head attention unit 204 stacked in sequence. Figure 2As shown, the multi-head attention unit includes a multi-head attention layer, a normalization layer, a full connection layer (FFN Layer), and a normalization layer. The multi-head attention layer processes the input vector based on the multi-head attention mechanism to obtain an attention vector. The normalization layer normalizes the attention vector to obtain a first normalized vector. Then, the first normalized vector and the input vector of the multi-head attention layer are added to obtain a second fusion vector. The second fusion vector is sent to the full connection layer for full connection mapping to obtain a full connection vector. The normalization layer normalizes the full connection vector to obtain a second normalized vector. Then, the second normalized vector and the second fusion vector are added to obtain an output vector of the multi-head attention unit.

[0149] S1025, mapping the first encoding vector into a result label of whether the sensitive word exists in the text content.

[0150] In the embodiment of the present application, the first encoding vector is input into the pre-trained classifier for classification, and the first encoding vector is mapped into a result label of whether the sensitive word exists in the text content. For example, if it is determined that the sensitive word exists, the classifier outputs the result label "1", and if it is determined that the sensitive word does not exist, the classifier outputs the result label "0". If it is determined that the sensitive word exists, step S103 is performed, and if it is determined that the sensitive word does not exist, step S105 is performed.

[0151] S103, extracting the sensitive word from the text content based on the GPT model.

[0152] In step S102, if it is determined that the sensitive word exists in the text content, the sensitive word is extracted from the text content based on the GPT (Generative Pre-trained Transformer) model. The GPT model learns the statistical rules of language through pre-training of a large corpus, and can not only distinguish the category of the user's question, but also extract the sensitive word in the user's question. However, the GPT model has a great requirement for the computing power of the background server, and the response time is relatively long, which leads to a long waiting time for the user. Therefore, in the embodiment of the present application, only when the BERT model determines that the input text content contains the sensitive word, the GPT model is called to further detect and extract the sensitive word. When the sensitive word is extracted, step S104 is performed, and when the sensitive word is not extracted, step S105 is performed.

[0153] S104, returning prompt information for refusing to generate content to the user.

[0154] When the GPT model extracts the sensitive word, prompt information for refusing to generate content is returned to the user.

[0155] S105, input the text content into the large language model for processing to generate the content corresponding to the text content and return to the user.

[0156] When the GPT model does not extract the sensitive word, input the text content into the large language model for processing to generate the content corresponding to the text content and return to the user.

[0157] The large language model input content compliance detection method provided by the application processes the text content based on the BERT model, preliminarily judges whether there is a sensitive word in the text content in combination with the context semantics, calls the GPT model to further extract the sensitive word from the text content when the text content contains the sensitive word, returns the prompt information of refusing to generate the content to the user when the sensitive word is extracted, inputs the text content into the large language model for processing to generate the content corresponding to the text content and returns to the user when the sensitive word is not extracted, and improves the sensitive word detection efficiency and accuracy.

[0158] In some embodiments of the application, a basic BERT model can be set, and the basic BERT model is a general model that does not involve application fields. A corresponding BERT model can also be trained for each specific field, for example, a BERT model for government services and a BERT model for party building.

[0159] The user interacts with the large language model in the form of multi-turn dialogue. When the text content input by the user is the first turn of dialogue, the basic BERT model is called to process the text content and determine the professional field to which the text content belongs. Specifically, the professional field to which the text content belongs can be determined by keyword matching. When the text content input by the user is the second turn and subsequent turns of dialogue, the BERT model of the professional field is called to process the text content. The application calls the basic BERT model in the first turn of dialogue to improve the model response speed and reduce the user waiting time. After determining the professional field of the dialogue, the BERT model of the professional field is called to process the text content, which can improve the prediction accuracy of the BERT model.

[0160] In some embodiments of the application, after the sensitive word is extracted, the sensitive word is stored in the sensitive word library as a data set for training the BERT model to realize the self-iteration of the BERT model.

[0161] Figure 3 The structure diagram of the large language model input content compliance detection device provided by the application is shown in Figure 3 The large language model input content compliance detection device comprises:

[0162] The text content acquisition module 101 is configured to acquire the text content input by the user.

[0163] The sensitive word determination module 102 is configured to determine whether a sensitive word exists in the text content based on processing of the text content by a BERT model in combination with context semantics.

[0164] The sensitive word extraction module 103 is configured to extract a sensitive word from the text content based on a GPT model when the text content contains a sensitive word.

[0165] The prompt information returning module 104 is configured to return prompt information for refusing to generate content to a user when a sensitive word is extracted.

[0166] The text content input module 105 is configured to input the text content into a large language model for processing when no sensitive word is extracted, to generate content corresponding to the text content, and to return the content to the user.

[0167] In some embodiments of the present application, the text content input module 105 is further configured to:

[0168] input the text content into a large language model for processing when the text content contains no sensitive word, to generate content corresponding to the text content, and to return the content to the user.

[0169] In some embodiments of the present application, the sensitive word determination module 102 includes:

[0170] The first word embedding submodule is configured to convert word segmentation in the text content into vector representation to obtain a first word embedding vector.

[0171] The first position embedding submodule is configured to convert the position of the word segmentation in the text content in the text content into vector representation to obtain a first position embedding vector.

[0172] The first fusion submodule is configured to fuse the first word embedding vector and the first position embedding vector to obtain a first fusion vector.

[0173] The encoding processing submodule is configured to input the first fusion vector into an encoding layer for processing to obtain a first encoding vector, the encoding layer including a plurality of stacked multi-head attention units, the output of a previous multi-head attention unit serving as the input of a subsequent multi-head attention unit.

[0174] The label mapping submodule is configured to map the first encoding vector into a result label of whether a sensitive word exists in the text content.

[0175] In some embodiments of the present application, the sensitive word determination module 102 further includes:

[0176] The first keyword matching submodule is configured to match sensitive words with literal meanings from the text content by using a keyword matching algorithm before the first fusion vector is input into the encoding layer for processing.

[0177] The first weight calculation submodule is configured to calculate the sum of the weights of all sensitive words to obtain a total weight of the text content, wherein the weight of the sensitive word is obtained by pre-training the BERT model.

[0178] The first weighting submodule is configured to weight the first fusion vector by using the total weight, and input the weighted first fusion vector into the encoding layer.

[0179] In some embodiments of the present application, the sensitive word determination module 102 further comprises:

[0180] The first input frequency determination submodule is configured to determine the input frequency of the text content based on historical input texts before the first fusion vector is input into the encoding layer for processing.

[0181] The first attention coefficient calculation submodule is configured to calculate the reciprocal of the input frequency as an attention coefficient for the text content.

[0182] The first adjustment submodule is configured to adjust the first fusion vector by using the attention coefficient, and input the adjusted first fusion vector into the encoding layer.

[0183] In some embodiments of the present application, the sensitive word determination module 102 further comprises:

[0184] The second keyword matching submodule is configured to match sensitive words with literal meanings from the text content by using a keyword matching algorithm before the first fusion vector is input into the encoding layer for processing.

[0185] The second weight calculation submodule is configured to calculate the sum of the weights of all sensitive words to obtain a total weight of the text content, wherein the weight of the sensitive word is obtained by pre-training the BERT model.

[0186] The second weighting submodule is configured to weight the first fusion vector by using the total weight.

[0187] The second input frequency determination submodule is configured to determine the input frequency of the text content based on historical input texts.

[0188] The second attention coefficient calculation submodule is configured to calculate the reciprocal of the input frequency as an attention coefficient for the text content.

[0189] A second adjusting sub-module is configured to adjust the weighted first fusion vector by using the attention coefficient, and take the adjusted first fusion vector as the input of the encoding layer.

[0190] In some embodiments of the present application, the large language model input content compliance detection device further comprises:

[0191] A sample obtaining module is configured to obtain a batch of training text samples from a training text sample set before obtaining the text content input by the user, and the batch of training text samples comprises a plurality of training text samples.

[0192] A word embedding module is configured to convert the word segmentation in each training text sample into a vector representation to obtain a second word embedding vector.

[0193] A position embedding module is configured to convert the position of the word segmentation in the training text sample in the training text sample into a vector representation to obtain a second position embedding vector.

[0194] A fusion module is configured to fuse the second word embedding vector and the second position embedding vector to obtain a second fusion vector.

[0195] A keyword matching module is configured to match sensitive words of literal meaning from the training text sample by using a keyword matching algorithm.

[0196] A weight calculation module is configured to assign an initial value to the weight of the sensitive word, calculate the sum of the weights of all sensitive words, and obtain the total weight of the training text sample.

[0197] A first weighting module is configured to weight the second fusion vector by using the total weight of the training text sample.

[0198] A frequency determination module is configured to determine the input frequency of the training text sample based on the training round of the training text sample.

[0199] An attention coefficient calculation module is configured to calculate the reciprocal of the input frequency of the training text sample as the attention coefficient of the training text sample.

[0200] An adjusting module is configured to adjust the weighted second fusion vector by using the attention coefficient of the training text sample.

[0201] An encoding module is configured to input the adjusted second fusion vector into an encoding layer for processing to obtain an encoding vector.

[0202] A label mapping module is configured to map the encoding vector of the training text sample into a result label indicating the presence of a sensitive word in the training text sample.

[0203] A statistical module is configured to count a prediction accuracy of the batch of training text samples on the BERT model.

[0204] A weight determination module is configured to adjust the weight of the sensitive word in a gradient descent direction when the accuracy is less than the preset value, and return to the step of obtaining a batch of training text samples from the training text sample set until the accuracy is greater than or equal to the preset value, and determine the weight of the sensitive word.

[0205] In some embodiments of the present application, the large language model input content compliance detection device further comprises:

[0206] A first calling module is configured to call the basic BERT model to process the text content input by the user when the text content input by the user is the first round of dialogue input, and determine the professional field to which the text content belongs.

[0207] A second calling module is configured to call the BERT model of the professional field to process the text content input by the user when the text content input by the user is the second round and subsequent dialogue input.

[0208] The large language model input content compliance detection device described above can execute the large language model input content compliance detection method provided by the foregoing embodiments of the present application, and has the corresponding functional modules and beneficial effects of executing the large language model input content compliance detection method.

[0209] Figure 4 A structural schematic diagram of an electronic device is provided for embodiments of the present application. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (such as headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0210] As Figure 4As shown, the electronic device includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., connected to the at least one processor 11 in communication. The memory stores a computer program executable by the at least one processor 11, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for operation of the electronic device can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0211] A plurality of components in the electronic device are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, a speaker, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0212] The processor 11 can be various general and / or special-purpose processing components having processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the large language model input content compliance detection method.

[0213] In some embodiments, the large language model input content compliance detection method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the large language model input content compliance detection method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the large language model input content compliance detection method by any other appropriate means, such as by means of firmware.

[0214] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a load programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0215] Computer programs used to implement the processes of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program

[0216] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. Computer-readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0217] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0218] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0219] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0220] The embodiments of the present application also provide a computer program product, comprising a computer program which, when executed by a processor, implements the large language model input content compliance detection method provided by any of the embodiments of the present application.

[0221] The computer program code can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce the computer implemented process such that the

[0222] It should be understood that the various forms of flow shown in the figures are illustrative examples of implementing the steps of the application. Several steps have been described as being performed by a single device. It will be understood that these steps can be performed by a single device or multiple devices, and that the steps can be performed in an order different from that shown in the figures. For example, the steps described in the figures can be performed in parallel or in a different order, as long as the desired results of the application are achieved. The application is not limited in this regard.

[0223] The specific embodiments have been shown and described for the purposes of illustrating the physiological principles of the application and its practical application. It is therefore to be understood that various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the application. The scope of the application is not to be limited by specific illustrative embodiments. The application is to cover any and all modifications and the same is therefore intended to be within the scope of the application.

Claims

1. A method for large language model input content compliance detection, characterized in that, The method comprises the following steps: obtaining text content input by a user; processing the text content based on a BERT model, and determining whether there is a sensitive word in the text content in combination with context semantics; when the text content contains a sensitive word, extracting the sensitive word from the text content based on a GPT model; when a sensitive word is extracted, returning prompt information for refusing to generate content to the user; when no sensitive word is extracted, inputting the text content into a large language model for processing, generating content corresponding to the text content, and returning the content to the user; the BERT model comprises an encoding layer, and before the encoding layer receives input, the BERT model further comprises: adopting a keyword matching algorithm to match sensitive words of literal meaning from the text content; calculating the sum of the weights of all sensitive words to obtain the total weight of the text content, wherein the weight of the sensitive word is obtained by pre-training the BERT model; adopting the total weight to weight the input of the encoding layer, and adopting the weighted input as the input of the encoding layer; or determining the input frequency of the text content based on historical input text; calculating the reciprocal of the input frequency as the attention coefficient of the text content; adopting the attention coefficient to adjust the input of the encoding layer, and adopting the adjusted input as the input of the encoding layer; or adopting a keyword matching algorithm to match sensitive words of literal meaning from the text content; calculating the sum of the weights of all sensitive words to obtain the total weight of the text content, wherein the weight of the sensitive word is obtained by pre-training the BERT model; adopting the total weight to weight the input of the encoding layer; determining the input frequency of the text content based on historical input text; calculating the reciprocal of the input frequency as the attention coefficient of the text content; adopting the attention coefficient to adjust the weighted input, and adopting the adjusted input as the input of the encoding layer.

2. The method of claim 1, wherein the method further comprises: Further comprising: when the text content does not contain a sensitive word, inputting the text content into a large language model for processing, generating content corresponding to the text content, and returning the content to the user.

3. The method of claim 1 or 2, wherein the method further comprises: processing the text content based on a BERT model, and determining whether there is a sensitive word in the text content in combination with context semantics, comprising: converting the word segmentation in the text content into vector expression to obtain a first word embedding vector; converting the position of the word segmentation in the text content in the text content into vector expression to obtain a first position embedding vector; fusing the first word embedding vector and the first position embedding vector to obtain a first fusion vector; inputting the first fusion vector into an encoding layer for processing to obtain a first encoding vector, the encoding layer comprising a plurality of stacked multi-head attention units, the output of a previous multi-head attention unit being used as the input of a subsequent multi-head attention unit; mapping the first encoding vector into a result label of whether the text content contains a sensitive word.

4. The method of claim 1, wherein the method further comprises: Before obtaining the text content input by the user, the method further comprises the following steps: obtaining a batch of training text samples from a training text sample set, the batch of training text samples comprising a plurality of training text samples; For each of the training text samples, convert the word segmentation in the training text sample into a vector representation to obtain a second word embedding vector; convert the position of the word segmentation in the training text sample into a vector representation to obtain a second position embedding vector; fuse the second word embedding vector and the second position embedding vector to obtain a second fused vector; adopt a keyword matching algorithm to match sensitive words of literal meaning from the training text sample; assign an initial value to the weight of the sensitive word, calculate the sum of the weights of all sensitive words to obtain the total weight of the training text sample; weight the second fused vector using the total weight of the training text sample; determine the input frequency of the training text sample based on the training round of the training text sample; calculate the reciprocal of the input frequency of the training text sample as the attention coefficient of the training text sample; adjust the weighted second fused vector using the attention coefficient of the training text sample; input the adjusted second fused vector into the encoding layer for processing to obtain an encoding vector; map the encoding vector of the training text sample to the result label of the existence of sensitive words in the training text sample; statistic the prediction accuracy of the BERT model for this batch of training text samples; when the accuracy is less than the preset value, adjust the weight of the sensitive word in the gradient descent direction, and return to execute the step of obtaining a batch of training text samples from the training text sample set until the accuracy is greater than or equal to the preset value, and the weight of the sensitive word is determined.

5. The method of claim 1, wherein the method further comprises: Also includes: when the text content input by the user is the first round of dialogue input, call the basic BERT model to process the text content and determine the professional field to which the text content belongs; when the text content input by the user is the second round and subsequent dialogue input, call the BERT model of the professional field to process the text content.

6. A large language model input content compliance detection apparatus, characterized by, The method for detecting compliance of input content of large language model according to any one of claims 1-5 comprises: a text content acquisition module for acquiring text content input by a user; a sensitive word determination module for processing the text content based on a BERT model and determining whether there is a sensitive word in the text content in combination with context semantics; a sensitive word extraction module for extracting a sensitive word from the text content based on a GPT model when the text content contains a sensitive word; a prompt information return module for returning prompt information for refusing to generate content to the user when a sensitive word is extracted; a text content input module for inputting the text content into a large language model for processing when no sensitive word is extracted, generating content corresponding to the text content, and returning the content to the user.

7. An electronic device, comprising: comprises: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method for detecting compliance of input content of large language model according to any one of claims 1-5.

Images