Problem shunting method, device and equipment

Abstract

The embodiment of the invention discloses a problem shunting method, device and equipment. In the embodiment of the invention, user question information is obtained; inputting the user question information into a preset high-recall-rate word library, and determining a comparison result; responding to the comparison result that the user question information comprises the same vocabulary in the high-recall-rate word bank, and determining the user question information as risk question information; inputting the risk problem information into a preset dichotomy model, and outputting a classification result; and in response to the fact that the classification result is high risk, determining that the risk problem information is high-risk problem information, and intercepting the high-risk problem information. According to the method, question shunting can be accurately and efficiently carried out, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more particularly, to a method, apparatus and device for problem diversion. Background Art

[0002] Generative AI (Artificial Intelligence Generated Content, AIGC) is an important branch of the field of artificial intelligence. It mainly uses artificial intelligence technology to realize the automatic generation of multimodal content such as text, images, audio, and video. In order to ensure that the generated content is safe, it is necessary to build an efficient and accurate content security framework. First, the questions input by the user are classified, and then the content is generated. Under the current content security framework, a model in which the vocabulary, answering model and audit model work together is usually adopted. By dividing the questions input by the user into three categories: security questions, high-risk questions and potential risk questions, the security questions are input into the business model for content generation. High-risk questions are directly intercepted by the audit model, and potential risk questions are answered by the answering model, so as to ensure the security of the generated content.

[0003] In the existing technology, in order to ensure the recall rate of risks, when triaging questions input by users, security issues and potential risk issues may be classified as high-risk issues. The audit model directly intercepts them and refuses to answer them, resulting in a high refusal rate and causing great damage to the user experience.

[0004] To sum up, how to accurately and efficiently triage problems and improve user experience is a problem that needs to be solved at present. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method, apparatus, and device for problem diversion, which can accurately and efficiently perform problem diversion and improve user experience.

[0006] In a first aspect, an embodiment of the present invention provides a method for problem diversion, the method comprising: obtaining user problem information; inputting the user problem information into a preset high-recall vocabulary library to determine a comparison result; in response to the comparison result being that the user problem information includes the same vocabulary as in the high-recall vocabulary library, determining that the user problem information is risk problem information; inputting the risk problem information into a preset binary classification model to output a classification result; in response to the classification result being high risk, determining that the risk problem information is high-risk problem information, and intercepting the high-risk problem information.

[0007] Optionally, the method further includes: in response to the classification result being a potential risk, determining that the risk problem information is potential risk problem information, and inputting the potential risk problem information into a pre-set proxy model, wherein the proxy model is a large model in the content security field.

[0008] Optionally, the method further includes: in response to the comparison result that the user question information includes no identical words in the high recall rate vocabulary, determining that the user question information is security question information; inputting the security question information into a pre-set business model, wherein the business model is a large model that directly answers the security question information.

[0009] Optionally, the generation process of the high-recall vocabulary includes: obtaining multiple historical question information; for each of the historical question information, performing preliminary word prompting, selecting keywords and generalizing keywords through a large model prompting project to generate multiple target keywords; and saving the multiple target keywords to the high-recall vocabulary.

[0010] Optionally, for each of the historical question information, preliminary word prompting is performed through the large model prompting project, specifically including: placing the historical question information in a pre-designed first prompting project question, inputting the first prompting project question into the large model for word segmentation and semantic expansion; and obtaining at least one core recall word.

[0011] Optionally, for each of the historical question information, keywords are selected through the large model prompt project, specifically including: placing the at least one core recall word in a pre-designed second prompt project question, inputting the second prompt project question into the large model to remove invalid words; and determining the at least one core recall word after removing the invalid words as a selected keyword.

[0012] Optionally, for each of the historical question information, keywords are generalized through the large model prompt project to generate multiple target keywords, specifically including: placing each of the selected keywords in a pre-designed third prompt project question, inputting the third prompt project question into the large model for keyword generalization, and generating multiple target keywords.

[0013] Optionally, the generation process of the binary classification model includes: generating a training sample library; and training the binary classification model based on the training sample library using contrast loss and classification loss at the same time.

[0014] Optionally, generating a training sample library specifically includes: obtaining a historical data set, wherein the historical data set includes multiple samples; inputting all samples in the historical data set into a large risk data model, classifying the multiple samples through a prompting project, and determining safety problem samples, high-risk problem samples, and potential risk problem samples.

[0015] Optionally, the contrast loss is a supervised contrast loss, and the loss classification loss is a binary cross entropy loss.

[0016] In a second aspect, an embodiment of the present invention provides a device for problem diversion, the device comprising:

[0017] An acquisition unit, used to obtain user question information;

[0018] A first determining unit, configured to input the user question information into a pre-set high recall rate vocabulary library and determine a comparison result;

[0019] The first determining unit is further configured to, in response to the comparison result being that the user question information includes the same vocabulary as that in the high recall rate vocabulary library, determine that the user question information is risk question information;

[0020] A second determining unit is configured to input the risk problem information into a preset binary classification model and output a classification result;

[0021] The second determining unit is further configured to, in response to the classification result being high risk, determine that the risk issue information is high-risk issue information, and intercept the high-risk issue information.

[0022] Optionally, the second determination unit is also used to: in response to the classification result being a potential risk, determine that the risk problem information is potential risk problem information, and input the potential risk problem information into a pre-set answering model, wherein the answering model is a large model in the content security field.

[0023] Optionally, the first determination unit is also used to: the method also includes: in response to the comparison result that the user question information includes no identical words in the high recall rate vocabulary, determining that the user question information is security question information; inputting the security question information into a pre-set business model, wherein the business model is a large model that directly answers based on the security question information.

[0024] Optionally, during the generation process of the high-recall vocabulary, the acquisition unit is also used to: acquire multiple historical question information; the device also includes: a generation unit, used to perform preliminary word prompting, select keywords and generalize keywords for each historical question information through a large model prompting project to generate multiple target keywords; the device also includes: a saving unit, used to save the multiple target keywords to the high-recall vocabulary.

[0025] Optionally, the generating unit is specifically configured to: place the historical question information into a pre-designed first prompt engineering question, input the first prompt engineering question into a large model for word segmentation and semantic expansion; and obtain at least one core recall word.

[0026] Optionally, the generation unit is specifically used to: place the at least one core recall word in a pre-designed second prompt engineering question, input the second prompt engineering question into the large model to remove invalid words; and determine the at least one core recall word after removing the invalid words as a selected keyword.

[0027] Optionally, the generating unit is specifically configured to place each selected keyword into a pre-designed third prompt engineering problem, input the third prompt engineering problem into a large model for keyword generalization, and generate a plurality of target keywords.

[0028] Optionally, during the generation process of the binary classification model, the generation unit is further used to: generate a training sample library; the device also includes a training unit, which is used to train the binary classification model based on the training sample library using contrast loss and classification loss at the same time.

[0029] Optionally, the generation unit is specifically used to: obtain a historical data set, wherein the historical data set includes multiple samples; input all samples in the historical data set into a large risk data model, classify the multiple samples through a prompt project, and determine safety problem samples, high-risk problem samples, and potential risk problem samples.

[0030] Optionally, the contrast loss is a supervised contrast loss, and the loss classification loss is a binary cross entropy loss.

[0031] In a third aspect, an embodiment of the present invention provides an electronic device comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement a method as described in any one of the first aspect and any possible one of the first aspect.

[0032] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the method as described in the first aspect and any possible method of the first aspect.

[0033] In an embodiment of the present invention, user question information is obtained; the user question information is input into a pre-set high-recall vocabulary to determine a comparison result; in response to the comparison result being that the user question information includes the same vocabulary as in the high-recall vocabulary, the user question information is determined to be risk question information; the risk question information is input into a pre-set binary classification model to output a classification result; in response to the classification result being high risk, the risk question information is determined to be high-risk question information, and the high-risk question information is intercepted. Through the above method, risk question information and non-risk questions are first diverted through a high-recall vocabulary, and then the risk question information is divided into high-risk question information and potential risk question information through a binary classification model, accurately and efficiently diverting questions and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0035] Figure 1 This is a flow chart of a method for problem diversion in an embodiment of the present invention;

[0036] Figure 2 is a flow chart of another method for problem diversion in an embodiment of the present invention;

[0037] Figure 3 This is a flow chart of another method for problem diversion in an embodiment of the present invention;

[0038] Figure 4 This is a flow chart of a method for generating a high-recall vocabulary in an embodiment of the present invention;

[0039] Figure 5 is a flow chart of a method for generating a binary classification model in an embodiment of the present invention;

[0040] Figure 6 Schematic diagram of another problem diversion device in an embodiment of the present invention;

[0041] Figure 7 Schematic diagram of a device for problem diversion in an embodiment of the present invention;

[0042] Figure 8 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The present application is described below based on the following embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without the description of these details. To avoid obscuring the essence of the present application, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0044] Furthermore, persons of ordinary skill in the art will appreciate that the figures provided herein are for illustration purposes only and are not necessarily drawn to scale.

[0045] Unless the context clearly requires otherwise, words like “include”, “comprising” and the like throughout this application should be interpreted as including rather than exclusive or exhaustive; that is, as meaning “including but not limited to”.

[0046] In the description of this application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more.

[0047] In the existing technology, users input questions into the big model to obtain answers. After the big model receives the user's questions, it needs to divert the questions input by the user for security reasons. Specifically, they are divided into three types: security questions, potential risk questions and high-risk questions. Among them, the business big model in the big model can directly answer security questions. The business big model is a big model that can directly answer questions; for potential risk questions, they need to be sent to the biased answering big model. The answering model is a big model in the content security field. Among them, content security refers to protecting information from inappropriate content, such as pornography, violence, false information, etc., to ensure that it complies with laws, regulations and social ethics; for high-risk questions, the big model can directly refuse to answer. To ensure the recall rate of risks, that is, to ensure that potential risk issues and high-risk issues are identified as much as possible, security issues and potential risk issues may be classified as high-risk issues when triaging user input questions. These questions are then directly intercepted by the review model and rejected, resulting in a high rejection rate and significant damage to the user experience. Furthermore, existing technologies use manual labeling when constructing sample data for training high-recall vocabulary and classification models. This results in high labor costs, low accuracy, and low coverage, resulting in poor training of classification models. Therefore, how to accurately and efficiently triage questions and improve user experience is a current issue that needs to be addressed.

[0048] In order to solve the above problems, an embodiment of the present invention proposes a method for problem diversion, specifically as follows: Figure 1As shown, the method includes:

[0049] Step S101: Obtain user question information.

[0050] Specifically, the user inputs user question information into the display interface of the generative artificial intelligence AIGC system, and the AIGC system obtains the user question information. For example, the user question information includes "how to quickly write a paper", "where to issue an invoice", and "how to make yogurt", etc. The specific user question information is determined according to the actual situation.

[0051] Step S102: input the user question information into a pre-set high recall rate vocabulary to determine a comparison result.

[0052] Specifically, the user question information is compared with the high-recall vocabulary, wherein the vocabulary included in the high-recall vocabulary is high-risk vocabulary. For example, the high-recall vocabulary includes writing papers for others, issuing invoices for others, making contraband, etc., involving many words or phrases related to risk topics. The risk topics are risk categories, such as legal risks, moral and ethical disputes, security threats, emotional conflicts or cultural offenses and other risk types. The vocabulary included in the high-recall vocabulary is determined according to actual conditions. Different high-recall vocabulary can be set in different scenarios, and the embodiments of the present invention do not limit it.

[0053] In one possible implementation, the comparison result includes a successful comparison and a failed comparison. The successful comparison means that the user question information includes the same vocabulary as that in the high-recall vocabulary; the failed comparison means that the user question information includes no identical vocabulary as that in the high-recall vocabulary.

[0054] In the embodiment of the present invention, it is assumed that the user question information is "how to quickly write a paper for someone else", and is successfully matched with "write a paper for someone else" in the high recall rate vocabulary.

[0055] Step S103: In response to the comparison result that the user question information includes the same vocabulary as that in the high recall rate vocabulary library, determining that the user question information is risk question information.

[0056] Specifically, in response to the comparison result that the user question information includes the same words as those in the high recall rate vocabulary, that is, the comparison is successful, it is determined that the user question information is risk question information, wherein the risk question information includes potential risk question information and high-risk questions, and needs to be judged again.

[0057] Step S104: input the risk problem information into a preset binary classification model and output the classification result.

[0058] Specifically, the classification results include high risk and potential risk, and the binary classification model can be a linear model, a decision tree model, etc., which is determined according to actual conditions.

[0059] Step S105: In response to the classification result being high risk, determining that the risk problem information is high-risk problem information, and intercepting the high-risk problem information.

[0060] In one possible implementation, when the binary classification model determines that the classification result is high risk, the binary classification model directly intercepts the high-risk problem information without responding, or responds with a pre-set labeled answer, such as "Your question does not comply with laws and regulations", etc., which is determined based on actual circumstances.

[0061] In the embodiment of the present invention, after step S104, other steps are also included, specifically as follows: Figure 2 As shown, including the following:

[0062] Step S106: In response to the classification result being a potential risk, determining that the risk problem information is potential risk problem information, and inputting the potential risk problem information into a preset answering model.

[0063] Among them, the answering model is a large model in the field of content security. When answering questions, the large model in the field of content security will consider factors such as laws and regulations and give answers that comply with laws and regulations.

[0064] In the embodiment of the present invention, after step S102, other steps are also included, specifically as follows: Figure 3 As shown, including the following:

[0065] Step S107: In response to the comparison result that the user question information includes no identical words with those in the high recall rate vocabulary, determining that the user question information is security question information.

[0066] Assume that the user question information is "how to make yogurt", and the vocabulary comparison with the high recall rate vocabulary fails, which means that "how to make yogurt" is a safe question information, and the safe question information refers to question information without any risk.

[0067] Step S108: Input the security question information into a preset business model.

[0068] The business model is a large model that directly answers the security question information.

[0069] Specifically, the user question information "How to make yogurt" can be directly input into a pre-set business model for answering, where the business model can also be called a business big model, a big model, a large language model, a large language model (LLM), etc., where the large language model is a deep learning model based on a transformer architecture, which can process and generate natural language text. It is usually trained on a large amount of text data, has the ability to understand and generate language, and is widely used in dialogue systems, text generation, text extraction and other natural language processing tasks.

[0070] In the embodiment of the present invention, the generation process of the high recall rate vocabulary is as follows: Figure 4 As shown, the specific steps include:

[0071] Step S401: Acquire multiple historical question information.

[0072] Specifically, the agent obtains multiple historical question information, for example, "How to make a cake", wherein the agent is a system built based on a large model, which can autonomously perceive the environment, make decisions and perform actions. The embodiment of the present invention constructs a high-recall vocabulary through the agent. Since the words in the high-recall vocabulary are high-risk words, when constructing a high-recall vocabulary through historical question information containing high-risk words, more sensitive words will be involved. Therefore, the embodiment of the present application uses risk-free historical question information for exemplary explanation. In actual use, the risk-free historical question information can be replaced with the risky historical question information.

[0073] Step S402: For each piece of historical question information, a large model prompting project is used to perform preliminary word prompting, select keywords, and generalize keywords to generate multiple target keywords.

[0074] Specifically, for each of the historical question information, preliminary word prompting is performed through the large model prompting project, which specifically includes: placing the historical question information in a pre-designed first prompting project question, inputting the first prompting project question into the large model for word segmentation and semantic expansion; and obtaining at least one core recall word.

[0075] In one possible implementation, the preliminary inscription refers to word segmentation of historical question information. When performing word segmentation, the historical question information should be divided according to its components and semantics. For example, the first prompt engineering question designed for "How to make a cake" is "You are now a word segmentation expert. Please segment the sentence "How to make a cake" according to semantics, and output a word or combination of words that can reflect this query. This word will be included in the high recall rate vocabulary." The query in the first prompt engineering question is the question. In the above example, the query is "How to make a cake."

[0076] Specifically, after the first prompt engineering question is input into the display interface of the intelligent agent, the large model in the intelligent agent gives a response. The specific response content is as follows:

[0077] Based on semantics and recall requirements, the word segmentation and expansion of "how to make a cake" are as follows:

[0078] Basic participles:

[0079] How to make cake

[0080] Semantic combination expansion (high recall vocabulary candidates):

[0081] Making cakes

[0082] Cake Making

[0083] How to Make a Cake

[0084] Make a cake

[0085] Cake Recipe

[0086] Homemade cakes

[0087] Cake Tutorial

[0088] Baking cakes

[0089] Core recall word suggestions (by priority):

[0090] "Making a cake" - complete verb-object structure

[0091] "Cake making"-nominal expression

[0092] "Make cake" - colloquial high-frequency word

[0093] "Cake recipe"-demand scenario word

[0094] "Baking cake"-professional scene word

[0095] These combinations not only maintain the complete semantics of the original query but also cover possible variant expressions that users might use, effectively improving search recall. We recommend that the first three be included in the vocabulary as core, high-weighted words.

[0096] The core recall words obtained from the above historical question information are "making cake", "cake making", "making cake", "cake recipe" and "baking cake". This is only an example. The specific first prompt engineering question and answer are determined according to actual conditions, and the embodiment of the present invention does not limit it.

[0097] In one possible implementation, for each of the historical question information, keywords are selected through a large model prompting project, specifically including: placing the at least one core recall word in a pre-designed second prompting project question, inputting the second prompting project question into the large model to remove invalid words; and determining the at least one core recall word after removing the invalid words as a selected keyword.

[0098] For example, the second prompt engineering question is designed as "There are several words "making cakes", "cake making", "how to make cakes", "making cakes", "cake recipes", "homemade cakes", "cake tutorials", and "baking cakes". Keep the words that are suitable for recalling the query "how to make cakes" and remove invalid words that will cause a large number of false recalls." The query in the second prompt engineering question is the question.

[0099] Specifically, after the second prompt engineering question is input into the display interface of the intelligent agent, the large model in the intelligent agent gives a response. The specific response content is as follows:

[0100] For the query "how to make a cake," we need to retain words that are strongly related to cake making while removing words that may recall irrelevant content (such as other foods and non-baking scenarios). The following is the optimized recall vocabulary:

[0101] Reserved words (high relevance, suitable for recall)

[0102] Make a cake (most direct match)

[0103] Cake making (noun expression, covering tutorial content)

[0104] How to make a cake (colloquial question, matching common user searches)

[0105] Make a cake (concise high-frequency word)

[0106] Cake recipes (covering recipe requirements)

[0107] Homemade cakes (emphasis on DIY scene)

[0108] Baking cakes (precisely matching professional baking scenes)

[0109] Eliminate words (easily recall irrelevant content)

[0110] Cake tutorials (too broad, non-making content such as decoration and fondant may be recalled)

[0111] Optimization suggestions

[0112] If more precise recall is required, the close co-occurrence of "make" + "cake" can be limited (for example, "make cake" is more accurate than "make" or "cake" alone).

[0113] Add extended words such as "home cake making" and "oven cake making" to cover long-tail queries.

[0114] The final recommended high recall words:

[0115] Making cakes, cake making, how to make cakes, making cakes, cake recipes, homemade cakes, baking cakes”.

[0116] This is merely an example description. The specific second prompt engineering question and its answer are determined according to actual conditions and are not limited in the embodiments of the present invention.

[0117] In one possible implementation, for each of the historical question information, keywords are generalized through a large model prompt project to generate multiple target keywords, specifically including: placing each of the selected keywords in a pre-designed third prompt project question, inputting the third prompt project question into the large model for keyword generalization, and generating multiple target keywords.

[0118] Specifically, assuming the target keyword is "making cakes," the third prompt engineering question designed based on this is, "You are a content security expert. I need you to help me generalize risky words. These risky words will be used for risk prevention and control in the AIGC content generation process. Generalization methods include but are not limited to synonym replacement, pinyin conversion, traditional and simplified Chinese mixed conversion, variant word conversion, etc. The word that needs to be generalized is "making cakes."

[0119] Specifically, after the third prompt engineering question is input into the display interface of the intelligent agent, the large model in the intelligent agent gives a response. The specific response content is as follows:

[0120] As a content security expert, I will build a risk vocabulary from multiple dimensions to address the need for generalizing the risk term "making cakes," focusing on possible circumvention methods and variant expressions. The following is a systematic generalization solution:

[0121] 1. Foundation deformation (direct replacement)

[0122] Synonymous replacement:

[0123] Make cake / make cake / make cake / make cake / make cake / make cake

[0124] Cake making / cake DIY / cake homemade / cake handmade

[0125] Pinyin deformation:

[0126] Zhizuo cake / zuzuo cake / making dangao

[0127] Make dg / zz cake / make dg

[0128] 2. Evasion variants (special characters)

[0129] Delimited Insertion:

[0130] Make cake / make cake / make cake

[0131] Make% egg% cake / make! Make! egg! cake

[0132] Homophonic substitution:

[0133] Making Egg Lamb / Making Dan Cake / Making Egg Cake

[0134] Egg rolls / Egg rolls

[0135] 3. Advanced Countermeasures (Complex Deformation)

[0136] Mixed Traditional and Simplified Chinese:

[0137] Make cakes / make egg custard / make dango

[0138] Making custard cake

[0139] Symbol substitution:

[0140] Making Egg Cake / Making Egg Cake

[0141] Reverse expression:

[0142] Cake and egg making / Cake and egg making

[0143] Reverse: ocgnad production

[0144] 4. Industry Terminology Extension

[0145] Raw material direction:

[0146] Flour + eggs + oven operation

[0147] Cream + powdered sugar synthesis solution

[0148] Tool association:

[0149] Making desserts with molds

[0150] Oven baking solution

[0151] 5. Defense Recommendations

[0152] It is recommended to establish a multi-level detection:

[0153] Primary filtering: basic inflection vocabulary

[0154] Intermediate test: Symbol and homophonic combination

[0155] Advanced adversarial: semantic analysis + behavioral detection

[0156] Need to cooperate with:

[0157] Synonym frequency analysis

[0158] Unconventional symbol pattern recognition

[0159] Contextual semantic association judgment

[0160] [Safety Reminder]

[0161] It is recommended to update the variant vocabulary regularly (weekly iteration is recommended)

[0162] Establish a weight scoring mechanism for high-frequency variants

[0163] Important scenarios need to be combined with user behavior analysis

[0164] Do you need additional generalization solutions in other dimensions or optimization suggestions for specific application scenarios? "

[0165] This is merely an example description. The specific third prompt engineering question and its answer are determined according to actual conditions and are not limited in the embodiments of the present invention.

[0166] Step S403: Save the multiple target keywords to the high recall rate vocabulary.

[0167] In the embodiment of the present invention, the generation process of the binary classification model is as follows: Figure 5 As shown, the following steps are included:

[0168] Step S501: Generate a training sample library.

[0169] Specifically, the generation of a training sample library includes: obtaining a historical data set, wherein the historical data set includes multiple samples; inputting all samples in the historical data set into a large risk data model, classifying the multiple samples through a prompting project, and determining safety problem samples, high-risk problem samples, and potential risk problem samples.

[0170] In a possible implementation, the historical data set includes multiple samples, and the multiple samples are matched with keywords of the common crawl web page using a high recall rate word library. The data samples obtained from the common crawl web page, the keywords are words in the high recall rate word library, and can also be obtained from other web pages; since the content of the common crawl web page is very large, the data samples obtained are also relatively large, and the data samples can also be called risk data. In addition to including many high-risk problem samples (also called black samples) and potential risk problem samples (also called gray samples), the risk data also contains some misunderstood security problem samples (also called white samples). If the cost of manual labeling of the above risk data is very high, all data samples are input into the risk data big model, and the multiple risk data are marked by prompt engineering. Samples are classified, for example, "Please rewrite 'data sample 1' to make it more consistent with the white sample in the security content" or "Please rewrite 'data sample 1' to make it more consistent with the black sample in the security content". The data sample 1' here is a problem, which should be determined according to actual conditions. The above-mentioned risk data large model can generate more accurate white samples and black samples, and there is no need to manually label the categories of risk data, to determine security problem samples, high-risk problem samples and potential risk problem samples, wherein the security problem samples and potential risk problem samples are divided into the same category and marked as 0, and the high-risk problem samples are marked as another category and marked as 1.

[0171] Step S502: Based on the training sample library, the binary classification model is trained using contrast loss and classification loss simultaneously.

[0172] Specifically, the samples marked as 0 and the samples marked as 1 in the training sample library are used to train the binary classification model, wherein the contrast loss is a supervised contrast loss and the loss classification loss is a binary classification cross entropy loss.

[0173] For example, the joint loss function formula of binary cross-entropy loss and contrast loss is as follows:

[0174]

[0175] Wherein, the α and β represent weight coefficients, which are used to balance the importance of the classification task and the contrastive learning task; Represents the binary cross entropy loss, represents contrast loss.

[0176] The formula for the binary cross entropy loss is as follows:

[0177]

[0178] Among them, the y i ∈{0,1} represents the true label of sample i; the p i ∈[0,1] represents the probability that the binary classification model predicts that sample i belongs to category 1; N represents the number of samples in the batch.

[0179] In one possible implementation, for the supervised contrast loss, assume that the feature embedding of each sample is z i , for example, the z i For the hidden layer representation extracted by the neural network, the formula of the supervised contrast loss is as follows:

[0180]

[0181] Wherein, P(i) represents the set of positive samples of the same category as sample i; A(i) represents all samples in the batch except sample i; τ represents the temperature hyperparameter; z i ·z j represents the eigenvector z i and z j The cosine similarity of .

[0182] The following is a complete example to explain the entire process of problem diversion in detail. Figure 6 As shown, the following steps are included:

[0183] Step S601: Obtain user question information.

[0184] Step S602: Input the user question information into the high recall rate vocabulary for comparison to determine whether it is a hit. If not, execute step S603; if it is a hit, execute step S604.

[0185] Here, whether there is a hit or not means whether the comparison is successful, a hit means a successful comparison, and a miss means a failed comparison.

[0186] Step S603: Input the user question information into the business model.

[0187] Specifically, the business model responds to the user's question information and then ends the processing.

[0188] Step S604: input the user question into a binary classification model.

[0189] Step S605: The binary classification model classifies the user question information. If the classification is into category 1, step S606 is executed; if the classification is into category 2, step S607 is executed.

[0190] Specifically, the category 1 indicates that the user question information is a high-risk question, and the category 2 indicates a security question sample or a potential risk question sample.

[0191] Step S606: intercept the user question information.

[0192] Step S607: Input the user question information into the answering model.

[0193] Specifically, the answering model responds to the above user question information and then ends the processing.

[0194] Through the above embodiments, a diversion architecture is designed. First, security issues and risk issues are distinguished through a high-recall vocabulary, and then the risk issues are finely distinguished through a binary classification model, divided into high-risk issues and potential risk issues. When constructing a high-recall vocabulary, the keyword mining method of building an intelligent agent based on a large model can improve the generation efficiency and extensiveness of the recall vocabulary. When training a binary classification model, by constructing sample pairs with similar semantics but opposite risk attributes, the binary classification model's ability to distinguish boundary samples can be enhanced, thereby improving the discrimination accuracy of the binary classification model.

[0195] In an embodiment of the present invention, a device for problem diversion is provided, such as Figure 7 As shown, it specifically includes: an acquisition unit 701, a first determination unit 702 and a second determination unit 703; wherein, the acquisition unit 701 is used to acquire user question information; the first determination unit 702 is used to input the user question information into a pre-set high recall rate vocabulary library to determine the comparison result; the first determination unit 702 is also used to, in response to the comparison result that the user question information includes the same vocabulary as that in the high recall rate vocabulary library, determine that the user question information is risk question information; the second determination unit 703 is used to input the risk question information into a pre-set two-classification model to output the classification result; the second determination unit 603 is also used to, in response to the classification result being high risk, determine that the risk question information is high-risk question information and intercept the high-risk question information.

[0196] Furthermore, the second determination unit is also used to: in response to the classification result being a potential risk, determine that the risk problem information is potential risk problem information, and input the potential risk problem information into a pre-set answering model, wherein the answering model is a large model in the content security field.

[0197] Furthermore, the first determination unit is also used to: the method also includes: in response to the comparison result that the user question information includes no identical words in the high recall rate vocabulary, determining that the user question information is security question information; inputting the security question information into a pre-set business model, wherein the business model is a large model that directly answers based on the security question information.

[0198] Furthermore, during the generation process of the high-recall vocabulary, the acquisition unit is also used to: acquire multiple historical question information; the device also includes: a generation unit, used to perform preliminary word prompting, select keywords and generalize keywords for each historical question information through a large model prompting project to generate multiple target keywords; the device also includes: a storage unit, used to save the multiple target keywords to the high-recall vocabulary.

[0199] Furthermore, the generating unit is specifically configured to: place the historical question information into a pre-designed first prompt engineering question, input the first prompt engineering question into a large model for word segmentation and semantic expansion; and obtain at least one core recall word.

[0200] Furthermore, the generation unit is specifically used to: place the at least one core recall word in a pre-designed second prompt engineering question, input the second prompt engineering question into the large model to remove invalid words; and determine the at least one core recall word after removing the invalid words as a selected keyword.

[0201] Furthermore, the generating unit is specifically configured to place each selected keyword into a pre-designed third prompt engineering problem, input the third prompt engineering problem into the large model for keyword generalization, and generate a plurality of target keywords.

[0202] Furthermore, during the generation process of the binary classification model, the generation unit is also used to: generate a training sample library; the device also includes a training unit, which is used to train the binary classification model based on the training sample library using contrast loss and classification loss at the same time.

[0203] Furthermore, the generation unit is specifically used to: obtain a historical data set, wherein the historical data set includes multiple samples; input all samples in the historical data set into the risk data model, classify the multiple samples through the prompt project, and determine safety problem samples, high-risk problem samples and potential risk problem samples.

[0204] Furthermore, the contrast loss is a supervised contrast loss, and the loss classification loss is a binary cross entropy loss.

[0205] Figure 8Schematic diagram of the structure of the electronic device in the embodiment of the present invention. Figure 8 As shown, it includes a general computer hardware structure, which includes at least a processor 801 and a memory 802. The processor 801 and the memory 802 are connected via a bus 803. The memory 802 is suitable for storing instructions or programs executable by the processor 801. The processor 801 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 801 executes the instructions stored in the memory 802, thereby executing the method flow of the embodiment of the present invention as described above to process data and control other devices. The bus 803 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 804 and the display device and the input / output (I / O) device 805. The input / output (I / O) device 805 can be a mouse, keyboard, modem, network interface, touch input device, somatosensory input device, printer, and other devices known in the art. Typically, the input / output device 805 is connected to the system via an input / output (I / O) controller 806.

[0206] Among them, the instructions stored in the memory 802 are executed by at least one processor 801 to implement: obtaining user question information; inputting the user question information into a preset high recall rate vocabulary library to determine a comparison result; in response to the comparison result being that the user question information includes the same vocabulary as that in the high recall rate vocabulary library, determining that the user question information is risk question information; inputting the risk question information into a preset binary classification model to output a classification result; in response to the classification result being high risk, determining that the risk question information is high risk question information, and intercepting the high risk question information.

[0207] Specifically, the electronic device includes: one or more processors 801 and a memory 802, Figure 8 Take a processor 801 as an example. The processor 801 and the memory 802 may be connected via a bus or other means. Figure 8 In the example, a bus connection is used. Memory 802, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules. Processor 801 executes the non-volatile software programs, instructions, and modules stored in memory 802 to perform various functional applications and data processing of the device, thereby implementing the above-mentioned method for determining problem diversion.

[0208] The memory 802 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store a list of options, etc. In addition, the memory 802 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 802 may optionally include a memory remotely located relative to the processor 801, and these remote memories may be connected to an external device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0209] One or more modules are stored in the memory 802 , and when executed by one or more processors 801 , perform the problem diversion method in any of the above method embodiments.

[0210] As will be appreciated by those skilled in the art, various aspects of embodiments of the present invention may be implemented as systems, methods, or computer program products. Thus, various aspects of embodiments of the present invention may take the form of a complete hardware implementation, a complete software implementation (including firmware, resident software, microcode, etc.), or an implementation combining software aspects with hardware aspects, which may all be generally referred to herein as a "circuit," "module," or "system." Additionally, various aspects of embodiments of the present invention may take the form of a computer program product implemented in one or more computer-readable media having computer-readable program code implemented thereon.

[0211] Any combination of one or more computer-readable media can be utilized. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples (non-exhaustive enumeration) of computer-readable storage media will include the following: an electrical connection with one or more wires, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the context of an embodiment of the present invention, a computer-readable storage medium can be any tangible medium that can contain or store a program used by an instruction execution system, device, or apparatus, or a program used in conjunction with an instruction execution system, device, or apparatus.

[0212] A computer-readable signal medium may include a propagated digital signal having computer-readable program code embodied therein, such as in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and that can communicate, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or apparatus.

[0213] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0214] The computer program code for performing the operations for various aspects of the embodiments of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" programming language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer as a stand-alone software package; partially on the user's computer and partially on a remote computer; or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0215] The flowchart legends and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention described above describe various aspects of embodiments of the present invention. It will be understood that each block of the flowchart legends and / or block diagrams and the combination of blocks in the flowchart legends and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that the instructions (executed by the processor of the computer or other programmable data processing device) create a device for implementing the function / action specified in the flowchart and / or block diagram block or block.

[0216] These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing device, or other apparatus to operate in a particular manner, so that the instructions stored in the computer-readable medium produce an article of manufacture that includes instructions for implementing the functions / actions specified in the flowchart and / or block diagram blocks or blocks.

[0217] The computer program instructions may also be loaded onto a computer, other programmable data processing device or other apparatus to cause a series of operable steps to be performed on the computer, other programmable device or other apparatus to produce a computer-implemented process, such that the instructions executed on the computer or other programmable device provide a process for implementing the functions / actions specified in the flowchart and / or block diagram block or blocks.

[0218] The foregoing is merely a preferred embodiment of the present application and is not intended to limit the present application. Persons skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application are intended to be within the scope of protection of the present application.

[0219] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation portals for users to choose to authorize or refuse. The user's refusal to process personal information other than the necessary information for basic functions will not affect the user's use of basic functions.

Claims

1. A method for problem diversion, characterized in that: The method comprises: Get user problem information; Inputting the user question information into a pre-set high recall rate vocabulary to determine the comparison result; In response to the comparison result being that the user question information includes the same vocabulary as that in the high recall rate vocabulary library, determining that the user question information is risk question information; Input the risk problem information into a pre-set binary classification model and output the classification results; In response to the classification result being high risk, the risk issue information is determined to be high-risk issue information, and the high-risk issue information is intercepted.

2. The method according to claim 1, characterized in that The method further comprises: In response to the classification result being a potential risk, the risk problem information is determined to be potential risk problem information, and the potential risk problem information is input into a pre-set proxy answering model, wherein the proxy answering model is a large model in the content security field.

3. The method according to claim 1, characterized in that The method further comprises: In response to the comparison result being that the user question information includes no identical words with those in the high recall rate vocabulary, determining that the user question information is security question information; The security question information is input into a pre-set business model, wherein the business model is a large model that directly answers the security question information.

4. The method according to claim 1, wherein The generation process of the high recall rate vocabulary includes: Get multiple historical question information; For each piece of historical question information, a large model prompting project is used to perform preliminary word prompting, select keywords, and generalize keywords to generate multiple target keywords; The multiple target keywords are saved in the high recall rate vocabulary.

5. The method according to claim 4, characterized in that For each of the historical question information, preliminary word prompting is performed through the large model prompting project, specifically including: Put the historical question information into a pre-designed first prompt engineering question, and input the first prompt engineering question into the large model for word segmentation and semantic expansion; Get at least one core recall word.

6. The method according to claim 5, characterized in that For each of the historical problem information, keywords are selected through the large model prompting project, specifically including: placing the at least one core recall word in a pre-designed second prompt engineering question, and inputting the second prompt engineering question into the large model to remove invalid words; At least one core recall word after removing invalid words is determined as a selected keyword.

7. The method according to claim 6, characterized in that For each of the historical question information, generalize keywords through the large model prompting project to generate multiple target keywords, specifically including: Each selected keyword is placed in a pre-designed third prompt engineering problem, and the third prompt engineering problem is input into the large model for keyword generalization to generate multiple target keywords.

8. The method according to claim 1, characterized in that The generation process of the binary classification model includes: Generate a training sample library; According to the training sample library, the binary classification model is trained using contrast loss and classification loss at the same time.

9. The method according to claim 8, characterized in that Generating a training sample library specifically includes: Acquire a historical data set, wherein the historical data set includes multiple samples; All samples in the historical data set are input into the risk data big model, and the multiple samples are classified through prompt engineering to determine safety problem samples, high-risk problem samples and potential risk problem samples.

10. The method according to claim 8, characterized in that The contrast loss is a supervised contrast loss, and the classification loss is a binary cross entropy loss.

11. A device for problem diversion, characterized in that: The device comprises: An acquisition unit, used to obtain user question information; A first determining unit, configured to input the user question information into a pre-set high recall rate vocabulary library and determine a comparison result; The first determining unit is further configured to, in response to the comparison result being that the user question information includes the same vocabulary as that in the high recall rate vocabulary library, determine that the user question information is risk question information; A second determining unit is configured to input the risk problem information into a preset binary classification model and output a classification result; The second determining unit is further configured to, in response to the classification result being high risk, determine that the risk issue information is high-risk issue information, and intercept the high-risk issue information.

12. An electronic device comprising a memory and a processor, characterized in that: The memory is configured to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images