Large model streaming output dialogue risk control method and device, equipment and medium

Abstract

According to the technical scheme, the method comprises the steps that input risk control detection is conducted on input questions of a user, if risks exist, a default risk control answer template is inquired, and all characters of the risk control answer template are sequentially returned in a streaming mode; if the risk does not exist, the problem is transmitted to a large model to generate streaming output; generating a to-be-detected text based on streaming output, performing content security detection on the to-be-detected text, determining a content security detection result, and if the detection result is no risk, continuing to generate and output a large model; and if the detection result is that the risk exists, stopping the generation of the large model, replacing the currently buffered content with the preset prompt information, and then outputting the preset prompt information. According to the scheme of the invention, through risk control detection of input and output stages in a large model dialogue, the detection efficiency is improved, and the problems of long user waiting time and poor experience in full-buffer detection during large model streaming output, frequent detection interface calling and low generation speed in streaming output real-time detection are solved.

Description

Technical Field

[0001] The present invention relates to the field of large model technology, and in particular to a large model streaming output dialogue risk control method, device, equipment and medium. Background Art

[0002] In a dialogue system based on a large model, there may be text involving sensitive topics in the user's prompt words and the output results of the large language model, so risk control is required for input and output to ensure the security of the content generated by the large model. At present, in a dialogue system based on a large model of the Transformer decoder, the decoder uses an autoregressive mechanism to generate target language sentences word by word during the inference phase, resulting in a slow response of the dialogue system using synchronous output, long user waiting time, and poor user experience. Therefore, streaming output is usually used to improve the user experience of dialogue systems based on large models.

[0003] However, for the streaming output scenario, if you wait for the large model to generate all the answers before performing risk control detection, the above-mentioned synchronous output problem will occur. If the large model performs risk control detection every time it generates a token, it will lead to frequent risk control detection and affect the generation speed.

[0004] Therefore, a method is needed to solve the problem of long waiting time for users to perform full buffer detection during streaming output of large models, so as to improve the efficiency of streaming output of large models. Summary of the invention

[0005] The embodiments of the present invention provide a large-model streaming output dialogue risk control method, device, equipment and medium, which can solve the problem of long user waiting time for full buffer detection during large-model streaming output, and improve the efficiency of large-model streaming output.

[0006] According to one aspect of the present invention, a large model streaming output dialogue risk control method is provided, comprising:

[0007] Perform input risk control detection on the user's input questions. If there is a risk, query the default risk control answer template and return each character of the risk control answer template in sequence in a streaming manner; if there is no risk, pass the question to the big model to generate streaming output;

[0008] Generate a text to be detected based on the streaming output and perform content security detection on the text to be detected, wherein the content security detection is performed through a third-party API or based on a pre-trained content security detection multi-classification model, and each classification corresponds to a different risk type;

[0009] Determine the content security test result. If the test result is risk-free, continue to generate and output the large model; if the test result is risky, stop generating the large model and replace the current buffered content with the preset prompt information and then output it.

[0010] Optionally, the input risk control detection of the user's input problem includes:

[0011] Vectorizing the user input question to obtain a user input question vector;

[0012] Take the user input question vector as input and retrieve the vector with the highest similarity in the pre-built risk control vector library;

[0013] Determine whether the similarity of the matching vector exceeds the set similarity threshold. If it does not exceed the threshold, the current input question is determined to be risk-free. If it exceeds the threshold and the vector type in the current matching vector metadata is a question, the user input question is determined to be risky. Otherwise, obtain the original keyword corresponding to the matching vector from the matching vector metadata and determine whether the keyword is included in the user input question. If it is included, it means that the current input question is risky, otherwise, there is no risk.

[0014] Optionally, the risk control vector library is constructed in the following manner:

[0015] Obtain the vectorized results of risk questions and keywords and the metadata information of the vector content, where the metadata information includes the type and content of the vector. The type includes two types: questions and keywords, and the content includes the original question or keyword corresponding to the vector.

[0016] Initialize an empty vector library and load the vectorized model. Load the risk question and keyword files in sequence. Each line of the file represents a risk record. Traverse each line of the file and input the reading results into the vectorized model.

[0017] The output result of the vectorization model, the type of the current vectorized content, and the current vectorized content are added to the vector library as an interface for inputting the vector library to obtain the risk control vector library.

[0018] Optionally, generating the text to be detected based on the streaming output includes:

[0019] Create a streaming output cache token counter and a streaming output cache token string. Every time the large model generates a token, it will add the currently generated token to the end of the streaming output cache token string and increment the streaming output cache token counter by 1.

[0020] It is determined whether the value of the streaming output cache token counter is greater than a preset value. If it is greater than the preset value, the current streaming output cache token and the streaming output token that has been detected in the past are concatenated to obtain the text to be detected.

[0021] Optionally, if the detection result is no risk, continuing to generate and output the large model includes:

[0022] Continue to generate the large model and traverse the current buffered character streaming output in sequence, initialize the streaming cache token counter to 0, and set the streaming output cache token string to empty.

[0023] Optionally, the content security detection third-party API calls an API of a specified third-party platform;

[0024] The pre-trained content security detection multi-classification model, in which each classification corresponds to a different risk type, can be obtained by constructing a large number of questions with risk classification labels and label pairs for training.

[0025] Optionally, the content security detection multi-classification model is trained in the following way:

[0026] Read the training data file. Each line of the training data file represents a piece of data. Each piece of data contains two columns. The first column is the content, and the second column is the risk label corresponding to the content.

[0027] Shuffle the order of the data; divide the shuffled data into training set, validation set and test set according to a specific ratio;

[0028] Load a specific model and tokenizer;

[0029] Compilation of the model is completed through a specific method;

[0030] The training set, validation set, and test set are segmented by the tokenizer respectively, and then the model training method is called to implement the model training;

[0031] After the model is trained, it is evaluated. If the model accuracy on the test set meets the requirements, the model is saved locally.

[0032] According to another aspect of the present invention, a large model streaming output dialogue risk control device is provided, comprising:

[0033] The input risk control detection unit is used to perform input risk control detection on the user's input questions. If there is a risk, the default risk control answer template is queried, and each character of the risk control answer template is returned in sequence in a streaming manner; if there is no risk, the question is passed to the large model to generate a streaming output;

[0034] A content security detection unit, configured to generate a text to be detected based on the streaming output and perform content security detection on the text to be detected, wherein the content security detection is performed through a third-party API or based on a pre-trained content security detection multi-classification model, and each of the classifications corresponds to a different risk type;

[0035] The output unit is used to determine the content security detection result. If the detection result is risk-free, the generation of the large model will continue and output; if the detection result is risky, the generation of the large model will be stopped and the current buffered content will be replaced with the preset prompt information and then output.

[0036] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0037] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the large model streaming output dialogue risk control method described in any embodiment of the present invention.

[0038] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the large model streaming output dialogue risk control method described in any embodiment of the present invention when executed.

[0039] The technical solution of the embodiment of the present invention includes: performing input risk control detection on the user's input question, if there is a risk, querying the default risk control answer template, and returning each character of the risk control answer template in sequence in a streaming manner; if there is no risk, passing the question to the large model to generate a streaming output; generating a text to be detected based on the streaming output and performing content security detection on the text to be detected, wherein the content security detection is performed through a third-party API or a pre-trained content security detection multi-classification model, and each classification corresponds to a different risk type; determining the content security detection result, if the detection result is no risk, continuing to generate and output the large model; if the detection result is risky, stopping the generation of the large model and replacing the current buffered content with a preset prompt message and then outputting it. The solution of the embodiment of the present invention improves the efficiency of detection by performing risk control detection on the input and output stages in the large model dialogue, on the one hand, and solves the problem of long waiting time and poor experience of users in full buffer detection when the large model is streamed, and the problem of frequent calls to the detection interface and slow generation speed of real-time detection of streaming output.

[0040] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 This is a flow chart of a large model streaming output dialogue risk control method provided by an embodiment of the present invention;

[0043] Figure 2 This is an input risk control detection flow chart provided by an embodiment of the present invention;

[0044] Figure 3 This is a flow chart of flow output risk control detection provided by an embodiment of the present invention;

[0045] Figure 4 It is a structural schematic diagram of a large model streaming output dialogue risk control device provided by an embodiment of the present invention;

[0046] Figure 5 It is a structural schematic diagram of an electronic device for implementing the large model streaming output dialogue risk control device method of an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0048] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0049] like Figure 1 As shown, an embodiment of the present invention provides a large model streaming output dialogue risk control method, which may include the following steps:

[0050] S110. Perform input risk control detection on the user's input question. If there is a risk, query the default risk control answer template and return each character of the risk control answer template in sequence in a streaming manner; if there is no risk, pass the question to the big model to generate streaming output.

[0051] Receive user input questions and then perform input risk control detection. If there is a risk, query the default risk control answer template, where the risk control answer template supports configuration. Then traverse each character of the risk control answer template and return each character in turn in a streaming manner. The streaming is implemented using SSE. Through SSE, the server can push data to the client in real time, instead of the client polling the server to obtain updates.

[0052] SSE messages consist of data, id, event, etc., where data is mandatory and represents the actual message data; id is an optional unique identifier for the message; event is optional and defines the event type, which is used by the client to distinguish the type of message. The following is an example, data: I'm sorry, I can't provide the information you need, let's change the topic. For Python servers, you can easily generate SSE response streams by installing sse-starlette. Specifically, import EventSourceResponse from the sse_starlette.sse package, then use the output iterator of the large model as the input parameter of EventSourceResponse, and return an EventSourceResponse object. The EventSourceResponse object obtains data from the iterator each time, and then assembles the data into the standard format of sse and returns it to the client.

[0053] In an embodiment of the present invention, input risk control detection is performed on user input problems, including:

[0054] Vectorizing the user input question to obtain a user input question vector;

[0055] Take the user input question vector as input and retrieve the vector with the highest similarity in the pre-built risk control vector library;

[0056] Determine whether the similarity of the matching vector exceeds the set similarity threshold. If it does not exceed the threshold, the current input question is determined to be risk-free. If it exceeds the threshold and the vector type in the current matching vector metadata is a question, the user input question is determined to be risky. Otherwise, obtain the original keyword corresponding to the matching vector from the matching vector metadata and determine whether the keyword is included in the user input question. If it is included, it means that the current input question is risky, otherwise, there is no risk.

[0057] Figure 2 is an input risk control detection flow chart provided by an embodiment of the present invention, such as Figure 2 As shown: Load the vectorization model, which is the same as the vectorization model used to build the risk control vector library. Input the user input question into the vectorization model to obtain the vectorized representation of the user input question. Then use the input question vector as input to retrieve the most similar Top1 vector in the risk control vector library to determine whether the similarity of the matching vector exceeds the set similarity threshold. If it does not exceed the threshold, the current input question is considered risk-free. If it exceeds the threshold and the vector type in the current matching vector metadata is a question, the current input question is considered risky. Otherwise, obtain the original keyword corresponding to the matching vector from the matching vector metadata and determine whether the keyword is included in the current user input question. If it is included, it means that the current input question is risky. Otherwise, there is no risk.

[0058] In the embodiment of the present invention, the risk control vector library is constructed in the following manner:

[0059] Obtain the vectorized results of risk questions and keywords and the metadata information of the vector content, where the metadata information includes the type and content of the vector. The type includes two types: questions and keywords, and the content includes the original question or keyword corresponding to the vector.

[0060] Initialize an empty vector library and load the vectorized model. Load the risk question and keyword files in sequence. Each line of the file represents a risk record. Traverse each line of the file and input the reading results into the vectorized model.

[0061] The output result of the vectorization model, the type of the current vectorized content, and the current vectorized content are added to the vector library as an interface for inputting the vector library to obtain the risk control vector library.

[0062] The embodiment of the present invention uses a pre-built risk control vector library, which stores the vectorization results of some risky questions and keywords and the metadata information of the vector content, wherein the metadata information includes the type and content of the vector, the type includes two types: questions and keywords, and the content includes the original question or keyword corresponding to the vector. The risk vector library can use Milvus, Chroma, etc. The vectorization of risk questions and keywords can use vectorization models such as Bge or M3E. The risk control vector library construction process is as follows: initialize an empty vector library and load the vectorization model, load the risk question and keyword files in turn, wherein each line of the file represents a risk record, traverse each line of the file and input the read result into the vectorization model, and then add the output result of the vectorization model, the type of the current vectorized content, and the current vectorized content as the input to call the vector library interface to the vector library.

[0063] S120. Generate a text to be detected based on the streaming output and perform content security detection on the text to be detected, wherein the content security detection is performed through a third-party API or based on a pre-trained content security detection multi-classification model, and each of the classifications corresponds to a different risk type.

[0064] In an embodiment of the present invention, generating a text to be detected based on the streaming output includes:

[0065] Create a streaming output cache token counter and a streaming output cache token string. Every time the large model generates a token, it will add the currently generated token to the end of the streaming output cache token string and increment the streaming output cache token counter by 1.

[0066] It is determined whether the value of the streaming output cache token counter is greater than a preset value. If it is greater than the preset value, the current streaming output cache token and the streaming output token that has been detected in the past are concatenated to obtain the text to be detected.

[0067] The user input question is passed to the big model, which generates a streaming answer and buffers the streaming output results at the size of 4 tokens. The number of buffered tokens is configurable. The current buffered content and the historical streaming output of the big model are spliced ​​together as the text to be tested for content security testing. The content security test chooses to call a third-party API or a pre-trained content security detection multi-classification model according to the length of the content to be tested. Each classification corresponds to the type of risk.

[0068] Preferably, the basis for determining whether content security detection is performed through a third-party API or a pre-trained model is the length of the text to be detected. If the length of the text to be detected is less than 512, it is detected through a pre-trained content security detection model; if it exceeds 512, it is detected through a third-party API.

[0069] In an embodiment of the present invention, the third-party API of the container security detection calls the API of the specified third-party platform;

[0070] The pre-trained content security detection multi-classification model, in which each classification corresponds to a different risk type, can be obtained by constructing a large number of questions with risk classification labels and label pairs for training.

[0071] The third-party API can call Tencent Cloud's Tianyu text content security SaaS API or the API of Baidu's content review platform, which is based on a pre-trained content security detection multi-classification model. Each classification corresponds to a risk type, such as pornography risk, drug risk, violence risk, etc. A multi-classification model can be trained by constructing a large number of questions and label pairs with risk classification labels, such as a multi-classification model based on Bert.

[0072] In the embodiment of the present invention, the content security detection multi-classification model is trained in the following way:

[0073] Read the training data file. Each line of the training data file represents a piece of data. Each piece of data contains two columns. The first column is the content, and the second column is the risk label corresponding to the content.

[0074] Shuffle the order of the data; divide the shuffled data into training set, validation set and test set according to a specific ratio;

[0075] Load a specific model and tokenizer;

[0076] Compilation of the model is completed through a specific method;

[0077] The training set, validation set, and test set are segmented by the tokenizer respectively, and then the model training method is called to implement the model training;

[0078] After the model is trained, it is evaluated. If the model accuracy on the test set meets the requirements, the model is saved locally.

[0079] Taking the Bert-based multi-classification model as an example, the pre-training process of the Bert-based content security detection multi-classification model is as follows:

[0080] Step 1: Read the training data file. Each line of the training data file represents a piece of data. Each piece of data contains two columns. The first column is the content, and the second column is the risk label corresponding to the content.

[0081] In the second step, after reading the data, you need to shuffle the data first. You can use the shuffle function in the sklearn.utils package in the scikit-learn machine learning library to shuffle the data.

[0082] Step 3: Divide the shuffled data into training set, validation set, and test set in a ratio of 6:2:2. This can be achieved through train_test_split of sklearn.model_selection.

[0083] Step 4: Load the Bert model and tokenizer, initialize the bert-base-chinese model through TFBertForSequenceClassification in the transformers library, and load the tokenizer through BertTokenizer. The loss function is

[0084] tensorflow.keras.losses.SparseCategoricalCrossentropy, the accuracy of the model training is tensorflow.tf.keras.metrics.SparseCategoricalAccuracy, the optimizer is tensorflow.keras.optimizers.Adam, and the model is compiled through the compile method of the model. The input parameters are the optimizer, loss function, and evaluation indicator.

[0085] In step 5, the training set, validation set, and test set are segmented by the tokenizer respectively, and then the fit method of the model is called to implement model training. The input parameters are the training set and validation set after token segmentation. The number of iterations can be initialized to 5, and then adjusted according to the accuracy indicators of the model on the training set and validation set after model training.

[0086] Step 6: After the model is trained, the model is evaluated through the model's evaluate method. The input parameter is the test set after word segmentation. If the model accuracy on the test set meets the effect, the model is saved locally through the model's save_pretrained method. The input parameter is the model save path.

[0087] S130, determine the content security detection result. If the detection result is risk-free, continue to generate and output the large model; if the detection result is risky, stop generating the large model and replace the current buffered content with the preset prompt information and then output it.

[0088] For example, if the detection result is risk-free, the generation of the large model continues and the current buffered characters are traversed in sequence for streaming output. If the detection result is risky, the generation of the large model is stopped and the current buffered content is replaced with ellipsis and then output.

[0089] In the embodiment of the present invention, if the detection result is no risk, the large model is continued to be generated and output, including:

[0090] Continue to generate the large model and traverse the current buffered character streaming output in sequence, initialize the streaming cache token counter to 0, and set the streaming output cache token string to empty.

[0091] Figure 3 This is a flow chart of a flow output risk control detection provided by an embodiment of the present invention. Figure 3 As shown, a streaming output cache token counter and a streaming output cache token string are created. Each time the large model generates a token, the currently generated token is concatenated to the end of the streaming output cache token string and the streaming output cache token counter is incremented by 1. Then, it is determined whether the value of the streaming output cache token counter is greater than 3. If it is greater than 3, the current streaming output cache token and the historically detected streaming output token are concatenated together as the text to be tested for content security. The content security test can call a third-party API or a pre-trained content security test multi-classification model. If the test result is no risk, the generation of the large model continues and the current buffered character streaming output is traversed in sequence, and the streaming cache token counter is initialized to 0, and the streaming output cache token string is set to empty; if the test result is risky, the generation of the large model is stopped and the current buffered content is replaced with an ellipsis and then output.

[0092] The scheme of the embodiment of the present invention includes two stages: input risk control detection and streaming output risk control detection. Risk control detection is performed on input questions based on the risk control vector library. Only risk-free input questions allow the large model to generate answers, otherwise a fixed risk control answer template is directly returned through streaming output. For the streaming output risk control detection of the large model, the streaming output content is first buffered and the text content security detection is called once every 4 tokens are buffered. If the content security detection result is risky, the generation of the large model is stopped, and the 4 tokens currently buffered are replaced with ellipsis, and then the ellipsis is streamed out; if there is no risk, the 4 tokens currently buffered are directly streamed out. The efficiency of the detection is improved through two-stage detection, and the number of content security detections is reduced through streaming buffer detection, which increases the generation speed.

[0093] like Figure 4As shown, an embodiment of the present invention provides a large model streaming output dialogue risk control device, which may include the following units:

[0094] The input risk control detection unit 410 is used to perform input risk control detection on the user's input question. If there is a risk, the default risk control answer template is queried, and each character of the risk control answer template is sequentially returned in a streaming manner; if there is no risk, the question is passed to the large model to generate a streaming output;

[0095] A content security detection unit 420, configured to generate a text to be detected based on the streaming output and perform content security detection on the text to be detected, wherein the content security detection is performed through a third-party API or based on a pre-trained content security detection multi-classification model, and each of the classifications corresponds to a different risk type;

[0096] The output unit 430 is used to determine the content security detection result. If the detection result is risk-free, the generation of the large model will continue and output; if the detection result is risky, the generation of the large model will be stopped and the current buffered content will be replaced with the preset prompt information and then output.

[0097] Optionally, the input risk control detection unit 410, when performing input risk control detection on the user's input problem, specifically performs:

[0098] Vectorizing the user input question to obtain a user input question vector;

[0099] Take the user input question vector as input and retrieve the vector with the highest similarity in the pre-built risk control vector library;

[0100] Determine whether the similarity of the matching vector exceeds the set similarity threshold. If it does not exceed the threshold, the current input question is determined to be risk-free. If it exceeds the threshold and the vector type in the current matching vector metadata is a question, the user input question is determined to be risky. Otherwise, obtain the original keyword corresponding to the matching vector from the matching vector metadata and determine whether the keyword is included in the user input question. If it is included, it means that the current input question is risky, otherwise, there is no risk.

[0101] Optionally, the input risk control detection unit 410 is used to construct a risk control vector library in the following manner:

[0102] Obtain the vectorized results of risk questions and keywords and the metadata information of the vector content, where the metadata information includes the type and content of the vector. The type includes two types: questions and keywords, and the content includes the original question or keyword corresponding to the vector.

[0103] Initialize an empty vector library and load the vectorized model. Load the risk question and keyword files in sequence. Each line of the file represents a risk record. Traverse each line of the file and input the reading results into the vectorized model.

[0104] The output result of the vectorization model, the type of the current vectorized content, and the current vectorized content are added to the vector library as an interface for inputting the vector library to obtain the risk control vector library.

[0105] Optionally, the content security detection unit 420, when generating the text to be detected based on the streaming output, specifically performs:

[0106] Create a streaming output cache token counter and a streaming output cache token string. Every time the large model generates a token, it will add the currently generated token to the end of the streaming output cache token string and increment the streaming output cache token counter by 1.

[0107] It is determined whether the value of the streaming output cache token counter is greater than a preset value. If it is greater than the preset value, the current streaming output cache token and the streaming output token that has been detected in the past are concatenated to obtain the text to be detected.

[0108] Optionally, the output unit 430, when executing if the detection result is no risk, continuing to generate and output the large model, specifically executes:

[0109] Continue to generate the large model and traverse the current buffered character streaming output in sequence, initialize the streaming cache token counter to 0, and set the streaming output cache token string to empty.

[0110] Optionally, the content security detection third-party API calls the API of a specified third-party platform;

[0111] Based on the pre-trained content security detection multi-classification model, each classification corresponds to a different risk type, which can be obtained by constructing a large number of questions and label pairs with risk classification labels.

[0112] Optionally, the content security detection unit 420 is further configured to train a content security detection multi-classification model in the following manner:

[0113] Read the training data file. Each line of the training data file represents a piece of data. Each piece of data contains two columns. The first column is the content, and the second column is the risk label corresponding to the content.

[0114] Shuffle the order of the data; divide the shuffled data into training set, validation set and test set according to a specific ratio;

[0115] Load a specific model and tokenizer;

[0116] Compilation of the model is completed through a specific method;

[0117] The training set, validation set, and test set are segmented by the tokenizer respectively, and then the model training method is called to implement the model training;

[0118] After the model is trained, it is evaluated. If the model accuracy on the test set meets the requirements, the model is saved locally.

[0119] It is understandable that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the large model streaming output dialogue risk control device. In other embodiments of the present invention, the large model streaming output dialogue risk control device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0120] The information interaction, execution process and other contents between the units in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For specific contents, please refer to the description in the embodiment of the method of the present invention, and no further description is given here.

[0121] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0122] like Figure 5As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0123] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0124] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as the large model streaming output dialogue risk control device method.

[0125] In some embodiments, the large model streaming output dialogue risk control device method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via ROM12 and / or communication unit 19. When the computer program is loaded into RAM13 and executed by processor 11, one or more steps of the large model streaming output dialogue risk control device method described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the large model streaming output dialogue risk control device method in any other appropriate manner (for example, by means of firmware).

[0126] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0128] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0129] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0130] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0131] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0132] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0133] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A large model streaming output dialogue risk control method, characterized in that: include: Perform input risk control detection on the user's input questions. If there is a risk, query the default risk control answer template and return each character of the risk control answer template in sequence in a streaming manner; if there is no risk, pass the question to the big model to generate streaming output; Generate a text to be detected based on the streaming output and perform content security detection on the text to be detected, wherein the content security detection is performed through a third-party API or based on a pre-trained content security detection multi-classification model, and each classification corresponds to a different risk type; Determine the content security test result. If the test result is risk-free, continue to generate and output the large model; if the test result is risky, stop generating the large model and replace the current buffered content with the preset prompt information and then output it.

2. The method according to claim 1, characterized in that The input risk control detection of the user's input problem includes: Vectorizing the user input question to obtain a user input question vector; Take the user input question vector as input and retrieve the vector with the highest similarity in the pre-built risk control vector library; Determine whether the similarity of the matching vector exceeds the set similarity threshold. If it does not exceed the threshold, the current input question is determined to be risk-free. If it exceeds the threshold and the vector type in the current matching vector metadata is a question, the user input question is determined to be risky. Otherwise, obtain the original keyword corresponding to the matching vector from the matching vector metadata and determine whether the keyword is included in the user input question. If it is included, it means that the current input question is risky, otherwise, there is no risk.

3. The method according to claim 2, characterized in that The risk control vector library is constructed in the following way: Obtain the vectorized results of risk questions and keywords and the metadata information of the vector content, where the metadata information includes the type and content of the vector. The type includes two types: questions and keywords, and the content includes the original question or keyword corresponding to the vector. Initialize an empty vector library and load the vectorized model. Load the risk question and keyword files in sequence. Each line of the file represents a risk record. Traverse each line of the file and input the reading results into the vectorized model. The output result of the vectorization model, the type of the current vectorized content, and the current vectorized content are added to the vector library as an interface for inputting the vector library to obtain the risk control vector library.

4. The method according to claim 1, characterized in that The step of generating the text to be detected based on the streaming output includes: Create a streaming output cache token counter and a streaming output cache token string. Every time the large model generates a token, it will add the currently generated token to the end of the streaming output cache token string and increment the streaming output cache token counter by 1. It is determined whether the value of the streaming output cache token counter is greater than a preset value. If it is greater than the preset value, the current streaming output cache token and the streaming output token that has been detected in the past are concatenated to obtain the text to be detected.

5. The method according to claim 4, characterized in that If the test result is no risk, the large model is generated and output, including: Continue to generate the large model and traverse the current buffered character streaming output in sequence, initialize the streaming cache token counter to 0, and set the streaming output cache token string to empty.

6. The method according to claim 1, characterized in that The content security detection third-party API calls the API of a specified third-party platform; The pre-trained content security detection multi-classification model, in which each classification corresponds to a different risk type, can be obtained by constructing a large number of questions with risk classification labels and label pairs for training.

7. The method according to claim 6, characterized in that The content security detection multi-classification model is trained in the following way: Read the training data file. Each line of the training data file represents a piece of data. Each piece of data contains two columns. The first column is the content, and the second column is the risk label corresponding to the content. Shuffle the order of the data; divide the shuffled data into training set, validation set and test set according to a specific ratio; Load a specific model and tokenizer; Compilation of the model is completed through a specific method; The training set, validation set, and test set are segmented by the tokenizer respectively, and then the model training method is called to implement the model training; After the model is trained, it is evaluated. If the model accuracy on the test set meets the requirements, the model is saved locally.

8. A large model streaming output dialogue risk control device, characterized in that: include: The input risk control detection unit is used to perform input risk control detection on the user's input questions. If there is a risk, the default risk control answer template is queried, and each character of the risk control answer template is returned in sequence in a streaming manner; if there is no risk, the question is passed to the large model to generate a streaming output; A content security detection unit, configured to generate a text to be detected based on the streaming output and perform content security detection on the text to be detected, wherein the content security detection is performed through a third-party API or based on a pre-trained content security detection multi-classification model, and each of the classifications corresponds to a different risk type; The output unit is used to determine the content security detection result. If the detection result is risk-free, the generation of the large model will continue and output; if the detection result is risky, the generation of the large model will be stopped and the current buffered content will be replaced with the preset prompt information and then output.

9. An electronic device, characterized in that include: at least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the large model streaming output dialogue risk control method described in any one of claims 1-7.

10. A computer readable medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the large-model streaming output dialogue risk control method described in any one of claims 1-7 when executed.

Images