A method, apparatus and electronic device for question processing

Abstract

An embodiment of this specification discloses a method, device, and electronic device for question processing. The method for question processing includes: when receiving a question input by a user to a target large model, determining, from multiple protection risk dimensions, the protection risk dimension to which the question belongs, as well as the protection label and protection level of the question under the belonging protection risk dimension; determining a protection decision according to the protection label and protection level of the question under the belonging protection risk dimension; determining, from multiple content risk dimensions, the content risk dimension to which the question belongs, as well as the content label and content level of the question under the belonging content risk dimension; determining a content decision according to the content label and content level of the question under the belonging content risk dimension; determining and executing an answer acquisition strategy according to the protection decision and / or the content decision, and returning an answer obtained after executing the answer acquisition strategy to the user.

Description

Technical Field

[0001] This application belongs to the technical field of data processing, and specifically relates to a method, device, and electronic device for question processing. Background Art

[0002] Generative Artificial Intelligence (AIGC) refers to a technology that, based on artificial intelligence techniques such as generative adversarial networks and large pre-trained models, generates relevant content through learning and recognition of existing data with appropriate generalization capabilities. With the rise of generative artificial intelligence, numerous large models and their applications have emerged continuously, forming a situation of a hundred schools of thought contending.

[0003] The huge datasets behind these large models provide the large models with powerful content generation capabilities, but at the same time, the content generated by the large models also brings many security problems. The sources of these security problems usually include two aspects: on the one hand, from the contamination of data with problems such as value bias and privacy leakage, and on the other hand, from the ability of the large models to generate creative (hallucinatory) content.

[0004] To address the above problems, after pre-training, the large models can rely on Supervised Fine-Turning (SFT) and Reinforcement Learning from Human Feedback (RLHF) to perform alignment of HHH (Helpful - correctness, Honest - truthfulness, Harmless - harmlessness), improving the security of the content generated by the large models. However, more complex adversarial prompts have further exposed and exploited the security problems of the content generated by the large models. Therefore, it is necessary to provide a better question processing solution to protect against adversarial questions and align the Harmlessness of the large models. Summary of the Invention

[0005] Embodiments of this specification provide a method, device, and electronic device for question processing to provide a question processing solution.

[0006] In a first aspect, an embodiment of this specification provides a method for question processing. The method includes: when receiving a question input by a user to a target large model, determining, from multiple protection risk dimensions, the protection risk dimension to which the question belongs, as well as the protection label and protection level of the question under the belonging protection risk dimension; determining a protection decision according to the protection label and protection level of the question under the belonging protection risk dimension; determining, from multiple content risk dimensions, the content risk dimension to which the question belongs, as well as the content label and content level of the question under the belonging content risk dimension; determining a content decision according to the content label and content level of the question under the belonging content risk dimension; determining an answer acquisition strategy based on the target large model according to the protection decision and / or the content decision, and executing the answer acquisition strategy, and returning to the user the answer obtained after executing the answer acquisition strategy.

[0007] In a second aspect, an embodiment of this specification provides a device for question processing, including: a protection dimension module, configured to, when receiving a question input by a user to a target large model, determine, from multiple protection risk dimensions, the protection risk dimension to which the question belongs, as well as the protection label and protection level of the question under the belonging protection risk dimension; a protection decision module, configured to determine a protection decision according to the protection label and protection level of the question under the belonging protection risk dimension; a content dimension module, configured to determine, from multiple content risk dimensions, the content risk dimension to which the question belongs, as well as the content label and content level of the question under the belonging content risk dimension; a content decision module, configured to determine a content decision according to the content label and content level of the question under the belonging content risk dimension; an answer acquisition module, configured to determine an answer acquisition strategy based on the target large model according to the protection decision and / or the content decision, and execute the answer acquisition strategy, and return to the user the answer obtained after executing the answer acquisition strategy.

[0008] Thirdly, an embodiment of this specification provides an electronic device, which includes: a processor, and a memory arranged to store computer-executable instructions. When the executable instructions are executed, the processor is capable of: when receiving a user's question input to a target large model, determining, from multiple protection risk dimensions, the protection risk dimension to which the question belongs, as well as the protection label and protection level of the question under the protection risk dimension to which it belongs; determining a protection decision according to the protection label and protection level of the question under the protection risk dimension to which it belongs; determining, from multiple content risk dimensions, the content risk dimension to which the question belongs, as well as the content label and content level of the question under the content risk dimension to which it belongs; determining a content decision according to the content label and content level of the question under the content risk dimension to which it belongs; determining an answer acquisition strategy based on the target large model according to the protection decision and / or the content decision, and executing the answer acquisition strategy, and returning to the user the answer obtained after executing the answer acquisition strategy.

[0009] Fourthly, an embodiment of this specification provides a storage medium for storing a computer program, and the computer program can be executed by a processor to implement the following process: when receiving a user's question input to a target large model, determining, from multiple protection risk dimensions, the protection risk dimension to which the question belongs, as well as the protection label and protection level of the question under the protection risk dimension to which it belongs; determining a protection decision according to the protection label and protection level of the question under the protection risk dimension to which it belongs; determining, from multiple content risk dimensions, the content risk dimension to which the question belongs, as well as the content label and content level of the question under the content risk dimension to which it belongs; determining a content decision according to the content label and content level of the question under the content risk dimension to which it belongs; determining an answer acquisition strategy based on the target large model according to the protection decision and / or the content decision, and executing the answer acquisition strategy, and returning to the user the answer obtained after executing the answer acquisition strategy.

[0010] Fifth aspect, an embodiment of this specification provides a computer program product, including a computer program, which when executed by a processor implements the following process: When receiving a question input by a user for a target large model, determine, from multiple protection risk dimensions, the protection risk dimension to which the question belongs, as well as the protection label and protection level of the question under the belonging protection risk dimension; determine a protection decision according to the protection label and protection level of the question under the belonging protection risk dimension; determine, from multiple content risk dimensions, the content risk dimension to which the question belongs, as well as the content label and content level of the question under the belonging content risk dimension; determine a content decision according to the content label and content level of the question under the belonging content risk dimension; determine an answer acquisition strategy based on the target large model according to the protection decision and / or the content decision, and execute the answer acquisition strategy, and return to the user the answer obtained after executing the answer acquisition strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] To more clearly illustrate the technical solutions in one or more embodiments of this specification or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in one or more embodiments of this specification. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0012] Figure 1 is a schematic flowchart of a method for processing questions provided by an embodiment of this specification;

[0013] Figure 2 is a schematic flowchart of the usage process of a protection engine provided by an embodiment of this specification;

[0014] Figure 3 is a schematic flowchart of the usage process of a content risk engine provided by an embodiment of this specification;

[0015] Figure 4 is a schematic architecture diagram of a question processing system provided by an embodiment of this specification;

[0016] Figure 5 is a schematic flowchart of the operation process of a question processing system provided by an embodiment of this specification;

[0017] Figure 6 is a schematic diagram of the application scenario of a method for processing questions provided by an embodiment of this specification;

[0018] Figure 7 is a schematic structure diagram of a protection risk dimension provided by an embodiment of this specification;

[0019] Figure 8 It is a schematic flow diagram of a content risk control engine provided by an embodiment of this specification;

[0020] Figure 9 It is a schematic diagram of the interception and blocking process of high-risk questions provided by an embodiment of this specification;

[0021] Figure 10 It is a schematic process diagram of enhancing the prompt for question content provided by an embodiment of this specification;

[0022] Figure 11 It is a schematic process diagram of providing a safe answer on behalf of a large model provided by an embodiment of this specification;

[0023] Figure 12 It is a schematic structural diagram of a device for question processing provided by an embodiment of this specification;

[0024] Figure 13 It is a schematic structural diagram of an electronic device provided by an embodiment of this specification. Detailed implementation manners

[0025] Next, the technical solutions in the embodiments of this specification will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are some, but not all, of the embodiments of this specification. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope protected by this application.

[0026] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of this specification can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are usually of the same category, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the associated objects before and after.

[0027] Adversarial questions are challenging or misleading questions asked by users when they are trying to test the capabilities of the big model or find the weaknesses of the big model. Table 1 is a specific example of adversarial questions. As shown in Table 1, Table 1 shows two types of adversarial questions, one is target hijacking (or interception, hostage, etc.) questions, and the other is reverse induction (or guidance, persuasion, etc.) questions. There are many other sample types of adversarial questions like the above, such as: positive and negative views, role-playing, variant confrontation, etc. As can be seen from Table 1, the big model itself has no subject consciousness. Although it has been aligned by SFT+RLHF, it is still easily exploited and deceived by adversarial questions.

[0028] Table 1 shows specific examples of adversarial questioning.

[0029]

[0030] Among them, Supervised Fine-Turning (SFT) is a common deep learning strategy, which is usually used on pre-trained large language models. The specific approach is to use labeled data to fine-tune the large model to adapt it to specific tasks or fields.

[0031] The role of Reinforcement Learning from Human Feedback (RLHF) is to provide machines with a natural, humane, interactive learning process by incorporating human feedback into the training process. This is just like the way humans learn expertise from another professional. By building a bridge between humans, RLHF enables AI to quickly master human experience. In RLHF, reinforcement learning is combined with human feedback, and human preferences are used as reward signals to guide the training of large models (including large models), thereby enhancing the model's understanding and satisfaction of human intentions. In generative models, RLHF can also fully align the generated images with text prompts.

[0032] The question processing method, device and electronic device provided in the embodiments of this specification are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0033] Figure 1 A method for question processing provided by an embodiment of the present invention is shown. The method can be executed by an electronic device, and the electronic device may include: a server and / or a terminal device, wherein the terminal device may be, for example, a vehicle-mounted terminal or a mobile phone terminal. In other words, the method can be executed by software or hardware installed in the aforementioned electronic device, and the method for question processing includes the following steps:

[0034] Step S102: When receiving a user's question for the target large model, determine the protection risk dimension to which the question belongs, as well as the protection label and protection level of the question under the belonging protection risk dimension from multiple protection risk dimensions.

[0035] Among them, the target large model is a machine learning model with ultra-large-scale parameters (usually over one billion) and a complex computing structure. The target large model can be any type of large model, and the type of the target large model is not specifically limited in this specification and can be determined according to the actual situation. The question initiated by the user to the target large model can be unimodal text.

[0036] The protection (Guardrails) risk dimension is a dimension for understanding and protecting against risks for the questions input by users to the target large model in the large model usage scenario, which can be an intention dimension, a historical dimension, a legal and / or regulatory dimension, a social morality and / or ethical dimension, etc. The protection risk dimension can be obtained by analyzing and classifying the collected historical adversarial questions. The process of determining the protection risk dimension to which the question belongs can be: extracting the features of the question and comparing the extracted features with the features corresponding to each protection type under the protection risk dimension. If the two are similar, it means the question belongs to this protection risk dimension; otherwise, it does not.

[0037] Since the intention of the user's question is the purpose for which the user initiates the question, by understanding the intention of the user's question, adversarial questions with harmful intentions (or high-risk intentions) can be filtered, significantly reducing the risks generated in the user question session. Therefore, among these protection risk dimensions, the intention dimension representing the intention of the user to initiate the question can be used as an essential protection risk dimension. In addition to the intention dimension, other protection risk dimensions can be used to supplement the intention dimension. The number and content of the protection risk dimensions are not specifically limited in this specification and can be determined according to the actual situation.

[0038] To achieve refined protection of the question, for each protection risk dimension, there are multiple protection labels and multiple protection levels corresponding to it. For example, the protection labels corresponding to each protection type under the intention dimension can include: target hijacking, reverse induction, positive and negative viewpoints, role-playing, variant confrontation, etc. The protection level corresponding to each protection label can include high risk, low risk, medium risk, etc. The protection labels and protection levels included in different protection risk dimensions are not specifically limited in this specification and can be determined according to the actual situation.

[0039] Figure 2 Shows a schematic diagram of the usage process of the protection engine. As Figure 2 shown, a protection engine (a system for pre-protecting the risks of questions) can be set to determine the protection risk dimension of the questions input by users to the target large model.Figure 2 The protection engine therein includes 3 protection risk dimensions. The input question hits the target hijacking label in the intent dimension, and the risk level is high risk.

[0040] In this protection engine, multiple multi-classification protection dimension models can be set. Each protection risk dimension corresponds to a protection dimension model. The protection dimension model enhances the generalization ability for adversarial questions based on a rich word library and keywords. When receiving a question input by the user to the target large model, the question can be input into each protection dimension model simultaneously to determine the protection risk dimension to which the question belongs, as well as the protection label and protection level of the question under the belonging protection risk dimension. It should be noted that a question can belong to one or more protection risk dimensions. The structure of the protection dimension model is not specifically limited in this specification and can be determined according to the actual situation.

[0041] Step S104: Determine a protection decision based on the protection label and protection level of the question under the belonging protection risk dimension.

[0042] Among them, the protection decision is a risk strategy determined for the question based on the protection risk dimension, which is used to obtain specific decision actions and cooperate with the target large model for security defense. The protection decision can be roughly divided into two categories: pass and not pass. The former is used to indicate that it can enter the next link (i.e., the determination of the content risk dimension), and the latter is used to indicate that it cannot enter the next link and directly return relevant information of refusing to answer to the user who sent the question as the answer to the question. Further, these two types of decisions of pass and not pass can be further subdivided. For example, the former can include transaction suggestions, question interactions, etc., and the latter can include refusing to answer, terminating the session, etc.

[0043] In an example, a protection decision can be determined through a policy tree. Specifically, the policy tree can include N tree nodes, and each tree node can be composed of 1 - N policies; each policy can include a rule and multiple actions. The rule can include: left value, comparison operator, right value, and the actions include: pass, reject, all enter for review, only hit enter for review, transaction suggestion, etc. By inputting the protection label and protection level of the question under the belonging protection risk dimension into the policy tree, the protection decision corresponding to the question can be obtained.

[0044] Step S106: Determine the content risk dimension to which the question belongs, as well as the content label and content level of the question under the belonging content risk dimension from multiple content risk dimensions.

[0045] Among them, the content risk dimension is a risk dimension determined from the content of the question, which can include content bottom line dimensions (such as including relevant legal bottom lines), technology ethics dimensions, data privacy dimensions, etc.

[0046] To achieve refined control over the content of questions, for each content risk dimension, there are corresponding multiple content tags (corresponding to content risk types) and multiple content levels. For the content risk dimensions in the compliance dimension, the content tags may include: labor, environment, etc. The content levels corresponding to each content tag may include high risk, low risk, medium risk, etc. In this specification, the content tags and content levels included in different content risk dimensions are not specifically defined and can be determined according to the actual situation.

[0047] In one example, a content risk engine (a system for identifying, evaluating, and managing risks based on the content of questions) can be set up to determine the content risk dimension of the questions input by users to the target large model. In this content risk engine, for each content risk dimension, one to N policy trees can be set (the number of policy trees is related to the number of tags of the content risk dimension), and each policy tree can contain N tree nodes, and each node can be composed of one to N policies. The policy tree can refer to the policy tree corresponding to the aforementioned protection decision. In addition, the left value of each content risk dimension can also cover rich data sources such as models, databases, caches, etc. The content risk engine and the protection engine can be independent of each other, or the two can be combined into a new engine. In this specification, the relationship between the content risk engine and the protection engine is not specifically defined and can be determined according to the actual situation.

[0048] When receiving the questions input by users to the target large model, the questions can be input into the content risk engine to determine the content risk dimension to which the questions belong, as well as the content tags and content levels of the questions under the belonging content risk dimension.

[0049] In one example, determining the content risk dimension to which the questions belong can be executed after the protection decision is determined (that is, the questions should be input into the protection engine first and then into the content risk engine). In this way, when the protection decision is not passed, the process of determining the content risk dimension to which the questions belong can be avoided, improving the efficiency of question processing.

[0050] Step S108: Determine the content decision according to the content tags and content levels of the questions under the belonging content risk dimension.

[0051] Among them, content decision-making is a risk strategy determined for a question based on content risk dimensions, which is used to obtain specific decision-making actions. Content decisions can be roughly divided into two categories: pass and fail. The former is used to indicate that the next step can be entered (i.e., determining an answer acquisition strategy based on the target large model), and the latter is used to indicate that the next step cannot be entered, and relevant information rejecting the answer is directly returned to the user who issued the question as the answer to the question. Further, these two types of decisions, pass and fail, can be further subdivided. For example, the former can include transaction suggestions, question interactions, etc., and the latter can include rejecting the answer, terminating the session, etc.

[0052] Similar to the process of determining the aforementioned protection decision, a decision tree can be used to determine the content decision using the content tags and content levels of the question in the content risk dimension to which it belongs.

[0053] Figure 3 The usage process diagram of the content risk engine is shown. As Figure 3 shown, the content risk engine can include a content qualitative layer and a comprehensive decision-making layer. After the question entered by the user into the target large model is input into the content risk engine, the content qualitative layer can understand the content risk dimension, obtain the content risk dimension to which the question belongs, as well as the content tags and content levels of the question in the content risk dimension to which it belongs. The comprehensive qualitative layer can then obtain the content decision based on the obtained content tags and content levels. In the case of obtaining multiple content decisions, these content decisions can be merged to obtain the final content decision.

[0054] In addition to the content qualitative layer and the comprehensive strategy layer, in order to reduce the pressure on the content risk control engine, as Figure 3 shown, a quick decision-making layer can also be set in the content risk control engine. There are multiple sample libraries such as a list library, a keyword library, and a link library that serve as a whitelist or a blacklist in the quick decision-making layer. The obtained tags of the intent dimension are compared with the data in these sample libraries. When the comparison result indicates that the question has a high risk, the high-risk question can be quickly intercepted and processed to directly obtain the content decision, reducing the pressure on the content risk prevention and control engine.

[0055] Step S110: Determine an answer acquisition strategy based on the protection decision and / or content decision, execute the answer acquisition strategy, and return the answer obtained after executing the answer acquisition strategy to the user.

[0056] Among them, the answer acquisition strategy is a strategy for using a large model to obtain the answer corresponding to a question. Since the protection label and protection level are the labels and levels of the question determined from the protection risk dimension, while the content label and content level are the labels and levels of the question determined from the content risk dimension, therefore, the protection decision corresponding to the former and the content decision corresponding to the latter can be combined to obtain the answer acquisition strategy. The answer acquisition strategy can be to refuse to answer, input the large model to obtain the answer, etc. Table 2 shows some answer acquisition strategies corresponding to different adversarial questions.

[0057] Table 2 Answer Acquisition Strategies Corresponding to Different Adversarial Questions

[0058]

[0059] After obtaining the answer acquisition strategy, the answer acquisition strategy can be executed, and the answer obtained after the execution is returned to the user. Figure 4 shows a schematic diagram of the architecture of a question processing system. As Figure 4 shown, the system includes a protection engine, a content risk engine, and a decision center. The decision center is used to determine the answer acquisition strategy based on the protection decision and the content decision.

[0060] Figure 5 shows Figure 4 the schematic diagram of the operation process of the question processing system in Figure 5 shown. As shown, after the user inputs a question, the question is first input into the protection engine, and the question is understood through multiple protection risk dimensions to obtain the protection risk dimension to which the question belongs, as well as the protection label and protection level of the question under the belonging protection risk dimension. Furthermore, the protection decision can be obtained based on the obtained protection label and protection level. In the case where the protection decision is passed, the question is input into the content risk engine to obtain the content risk dimension to which the question belongs, as well as the content label and content level of the question under the belonging content risk dimension. Furthermore, the content decision can be obtained based on the obtained content label and content level. In the case where the content decision is passed, the decision center determines and executes the answer acquisition strategy, and the answer acquisition strategy is not to use the large model to obtain the answer, and a refused answer is returned to the user.

[0061] In the embodiments of this specification, obtain the protection label and protection level of the question under the protection risk dimension to which it belongs, and obtain the protection decision corresponding to the obtained protection label and protection level. Obtain the content label and content level of the question under the content risk dimension to which it belongs, and obtain the content decision corresponding to the obtained content label and content level. Combine the risk label and risk level of the content risk dimension, and combine the protection decision and the content decision to determine the answer acquisition strategy, and then execute the answer acquisition strategy to return the answer obtained by executing the answer acquisition strategy to the user. This process combines the protection risk dimension and the content risk dimension to obtain the answer acquisition strategy. By adding the protection risk dimension, it is possible to improve the protection against adversarial questions for large models, and thus align the harmlessness of large models.

[0062] Figure 6 An application scenario diagram of a method for question processing is provided, as Figure 6 shown. The user sends a service request to the server through a terminal such as a mobile phone. The service request includes: performing a question processing task for the target question initiated by the user to the target large model. After receiving the service request, the server uses the question processing method in this specification to obtain the answer corresponding to the question, and sends the answer to the terminal for the user to refer to.

[0063] As mentioned above, the intent dimension is an important protection risk dimension. Adversarial questions are complex and variable, and the intent dimension covers a wide range. In order to better understand the question intent, it is necessary to perform a fine-grained division of the intent dimension. In one implementation, the protection risk dimension includes the intent dimension. Step S102 can be performed as the following steps A1 - A2:

[0064] Step A1, when receiving a question, determine the protection risk dimension to which the question belongs from multiple protection risk dimensions;

[0065] Step A2, when the protection risk dimension to which the question belongs includes the intent dimension, determine the sub-intent dimension to which the question belongs, as well as the protection label and protection level of the question under the sub-intent dimension, from the sub-intent dimensions included in the intent dimension.

[0066] Figure 7 The structural diagram of the protection risk dimension is shown, as Figure 7 shown. The protection risk dimension can include the topic dimension, the intent dimension, and the domain dimension. The widely covered intent dimension is finely divided to obtain the attack sub-dimension, the emotion sub-dimension, and other sub-dimensions. Tables 3 - 5 successively show the protection types included in the three sub-intent dimensions.

[0067] Table 3 Protection types included in the attack sub-dimension in the intent dimension

[0068]

[0069] As shown in Table 3, in the attack sub-dimension of the intention dimension, it mainly targets three types of adversarial questions: target hijacking, reverse induction, and variant countermeasure.

[0070] Table 4 Protection types included in the emotion sub-dimension of the intention dimension

[0071]

[0072] As shown in Table 4, in the emotion sub-dimension of the intention dimension, it mainly targets four types of questions: factual description - positive, factual description - negative, subject emotion - positive, and subject emotion - negative.

[0073] Except for the question types in Table 3 and Table 4, the remaining question types are regarded as other sub-intention dimensions. The types included in other sub-intention dimensions can be referred to Table 5.

[0074] Table 5 Protection types included in other sub-dimensions of the intention dimension

[0075]

[0076] Similar to the protection risk dimension, each intention sub-dimension also contains multiple protection labels and protection levels. Since the intention dimension includes multiple sub-intention dimensions, therefore, in the case of receiving a question input by the user to the target large model, the protection risk dimension to which the question belongs can be determined first. In the case where the protection risk dimension to which the question belongs includes the intention dimension, the intention sub-dimension to which the question belongs can be further determined, and the protection label and protection level of the question under the belonging intention sub-dimension can be determined.

[0077] In the embodiments of this specification, in the case where the protection risk dimension to which the question belongs includes the intention dimension, it is necessary to further determine the intention sub-dimension to which the question belongs to better identify the protection risk dimension of the question, thereby reducing the risk of the answer obtained in the question processing process.

[0078] Determining the protection label and protection level of the question under the belonging protection risk type can be regarded as a type division of the question. This type division can be used to facilitate the subsequent determination process of the content risk dimension. In one implementation, in the case where the protection risk dimension includes the domain dimension, step S106 can be executed as the following steps B1 - B2:

[0079] Step B1, based on the protection label of the question under the belonging domain dimension, determine the target content risk dimension corresponding to the question from the content risk dimension;

[0080] Step B2: From the target content risk dimension, determine the content risk dimension to which the question belongs, as well as the content label and content level of the question under the content risk dimension to which it belongs.

[0081] Among them, the Domain dimension is used to indicate the scope or area to which the question belongs. When collecting historical adversarial questions, the adversarial questions can be divided by domain to obtain multiple domain dimensions. Table 6 shows some protection types included in the domain dimension. As shown in Table 6, the adversarial questions can be related to the fields of politics, medicine, and government affairs.

[0082] Table 6 Protection Types Included in the Domain Dimension

[0083]

[0084] The target content risk dimension is the content risk dimension to which the question may belong based on the domain dimension. In one example, the mapping relationship between the domain dimension and the content risk dimension can be set. In this way, when at least one preset domain dimension is hit during the determination process of the protection risk dimension, through the obtained protection label and the mapping relationship, the multiple content risk dimensions (i.e., the target content risk dimension) to which the question may belong can be determined.

[0085] Furthermore, it is possible to determine the specific content risk dimension to which the question belongs from the target content risk dimension, and determine the content label and content level of the question under the content risk dimension to which it belongs.

[0086] Figure 8 Shows the schematic flow diagram of the content risk control engine. As Figure 8 shown, determining the target content risk dimension based on the hit domain dimensions (Domain 1 and Domain 2) can avoid the execution of irrelevant policies.

[0087] In the above process, determining the target content risk dimension based on the protection label under the domain dimension circumscribes the scope of the content risk dimension to which the question belongs, reduces the execution of irrelevant policies, shrinks the size of the decision-making process, realizes the routing function of the content policy in the content risk dimension and the refined prevention and control of adversarial questions, reduces various costs such as machines, models, and storage, and better realizes cost reduction and efficiency improvement.

[0088] Furthermore, the protection label and protection level of the question under the protection risk dimension to which it belongs can also be used to assist the determination process of the content risk dimension and / or the determination process of the content decision. In one implementation, step B2 can be executed as the following step C1:

[0089] Step C1, based on the protection label and protection level of the question under the protection risk dimension, determine the content risk dimension to which the question belongs and the content label and content level of the question under the content risk dimension from the target content risk dimension;

[0090] And / or step S108 may be executed as the following step C2:

[0091] Step C2, determining the content decision according to the protection label and protection level of the question under the protection risk dimension to which it belongs, and the content label and content level of the question under the content risk dimension to which it belongs.

[0092] Since the protection label and protection level of the question under the protection risk dimension are the understanding of the question in the protection risk dimension, in one example, the protection label and protection level of the question under the protection risk dimension can be used in the determination process of the content risk dimension to which the question belongs. Similarly, the protection label and protection level of the question under the protection risk dimension can also be used in the determination process of content decision.

[0093] like Figure 8 As shown, the protection engine is externally mounted on the content risk control engine. The protection label and protection level obtained by the protection engine enter the content risk control engine. The protection label and protection level of the question under the protection risk dimension to which it belongs are input into the content qualitative layer. This can improve the accuracy of the content risk dimension to which the question belongs as determined by the content qualitative layer, as well as the content label and content level of the question under the content risk dimension to which it belongs; the protection label and protection level of the question under the protection risk dimension to which it belongs are input into the comprehensive decision-making layer. This can improve the accuracy of the content decision determined by the comprehensive decision-making layer.

[0094] Since the protection engine includes time-consuming capacity pre-calculation and parallel calculation of tree strategies, the protection engine can be set to process questions less than or equal to 100ms to ensure the timeliness of the question processing process. In addition to the content qualitative layer and the comprehensive strategy layer, in order to reduce the pressure on the content risk control engine, Figure 8 As shown, a quick decision layer can also be set up in the content risk control engine. The quick decision layer is provided with multiple sample libraries such as a list library, a keyword library, and a link library as a whitelist or a blacklist. The labels of the obtained intent dimension are compared with the data in these sample libraries. When the comparison result indicates that the question has a high risk, the high-risk question can be quickly intercepted and handled, and a content decision that is not passed can be directly obtained, thereby reducing the pressure on the content risk prevention and control engine.

[0095] In the embodiments of the present specification, by using the protection labels and protection levels of the question under the corresponding protection risk dimension for the determination process of the content risk dimension to which the question belongs and / or the determination process of content decision-making, the accuracy of the determination process of the content risk dimension to which the question belongs and / or the determination process of content decision-making can be improved.

[0096] In one implementation, step S110 can be performed as the following steps D1 - D2:

[0097] Step D1, when the protection decision or content decision is not passed, determine the answer acquisition strategy as not using the target large model to obtain the answer;

[0098] Step D2, when both the protection decision and content decision are passed, determine the answer acquisition strategy as using the target large model to obtain the answer.

[0099] Specifically, when the protection decision or content decision is not passed, it usually indicates that the question sent by the user to the target large model is a high-risk adversarial question, and such questions need to be directly intercepted and blocked. Figure 9 Illustrates the interception and blocking process of high-risk questions. As Figure 9 shown, for the case where the protection decision or content decision is not passed, the decision center will make an answer of refusing to answer.

[0100] When both the protection decision and content decision are passed, using the target large model to obtain the answer can be used as the answer acquisition strategy. In a question, the topic is the subject that the question is about. During the training process of the target large model, the detailed information of some topics is not open to the public for special purposes. Thus, for questions under these topics, if the questions are directly input into the target large model, the simple answers obtained may not satisfy the users. In one implementation, the protection risk dimension includes the topic dimension, and step S110 can be performed as the following steps E1 - E2:

[0101] Step E1, when the protection risk dimension to which the question belongs includes the topic dimension, obtain the supplementary information corresponding to the question from the data source corresponding to the topic dimension to which the question belongs;

[0102] Step E2, when the answer acquisition strategy is to use the target large model to obtain the answer, input the question and the supplementary information source corresponding to the question into the target large model to obtain the answer.

[0103] Among them, the topic dimension is the dimension of the subject that the question is about, and the topic dimension is the dimension for supplementing information to the question outside the intention dimension. The topic dimension can be determined based on the collected historical questions and their corresponding answers. Table 7 shows some of the protection types included in the topic dimension.

[0104] Table 7 Protection Types Included in Topic Dimensions

[0105]

[0106] Supplementary information is information that supplements a question. Since the target large model has insufficient understanding of some topic dimensions, relevant supplementary information needs to be found to improve the user experience in the large model application scenario. Usually, the ways to obtain supplementary information corresponding to different topic dimensions can be different. For example, the ways to obtain supplementary information can be databases, specific large models (different from the large model used by the user), etc.

[0107] In one example, the data source for obtaining supplementary information can be determined based on the topic dimension to which the question belongs, and then the supplementary information can be obtained from the data source. Specifically, it can first be determined whether the protection dimension to which the question belongs includes the topic dimension. If it does, the data source can be obtained based on the topic dimension, and the supplementary information can be obtained from the data source. In one implementation, the process of obtaining supplementary information can be performed as the following steps F1 - F2:

[0108] Step F1, when the data source is a database, extract keywords from the question, and based on the keywords, obtain the supplementary information corresponding to the question from the database;

[0109] Step F2, when the data source is a supplementary information large model, input the question into the supplementary information large model, and use the answer corresponding to the question output by the supplementary information large model as the supplementary information corresponding to the question.

[0110] Specifically, when the data source of the supplementary information is a database, the supplementary information can be obtained from the database (such as a knowledge base, etc.) according to the keywords in the question, so as to enhance the hint for the content of the question. Figure 10 Shows a schematic diagram of the process of enhancing the hint for the content of the question. As Figure 10 shown, a knowledge base can be used to obtain supplementary information.

[0111] When the data source of the supplementary information is a supplementary information large model, the question can be input into the supplementary information large model, and the answer can be used as the supplementary information to achieve secure proxy answering for the large model. Figure 11 Shows a schematic diagram of the process of secure proxy answering for the large model. As Figure 11 shown, a supplementary information large model can be used to obtain supplementary information.

[0112] Furthermore, the supplementary information and the question can be input into the target large model together to obtain the answer to be fed back to the user.

[0113] In the above process, by supplementing information to prompt and enhance the question content and / or provide a safe answer substitution, it is possible to increase the safety level of the large model application scenario while reducing the rejection rate of the large model, meeting the special prevention and control requirements in the large model application scenario.

[0114] Before feeding the answer back to the user, the answer can be subject to a security check to determine whether the answer presented to the user is safe, so as to further improve the risk prevention and control ability of the large model application scenario. In one implementation, step S110 can be executed as the following steps G1 - G3:

[0115] Step G1: Determine the content risk dimension to which the answer belongs, as well as the content label and content level of the answer under the content risk dimension to which it belongs, from multiple content risk dimensions;

[0116] Step G2: Determine the risk identification result of the answer based on the content label and content level of the answer under the content risk dimension to which it belongs;

[0117] Step G3: When the risk identification result of the answer is a preset risk level, return the answer to the user.

[0118] Among them, the risk identification result is the risk degree of the question determined based on the protection dimension and the content risk dimension, and it can be a specific risk value.

[0119] Specifically, different risk values can be set for different content labels and content levels. In this way, after obtaining the content label and content level of the answer under the content risk dimension to which it belongs, the risk value of the question can be determined, and then the risk identification result corresponding to the question can be determined according to the risk value. The risk identification result can be high risk, low risk, and medium risk.

[0120] When the risk identification result is high risk, it indicates that the answer has a relatively high risk and it is not appropriate to directly return the answer to the user. The answer can be processed to exclude the risk. Only when the processed answer is converted to the preset risk level can the answer be returned to the user; when the risk identification result is the preset risk level (such as low risk and / or medium risk), the answer can be directly returned to the user.

[0121] In the embodiments of this specification, by determining the risk identification result of the answer, the risk of the answer reaching the user can be reduced, thereby improving the risk prevention and control ability of the large model application scenario.

[0122] It can be understood that, without violating the principle logic, the above-mentioned method embodiments described in this specification can be combined with each other to form combined embodiments. Due to space limitations, they will not be elaborated in this specification. Those skilled in the art can understand that in the above methods of the specific implementation manner, the specific execution order of each step should be determined according to its function and possible internal logic.

[0123] It should be noted that for the method for question processing provided in the embodiments of this specification, the execution subject can be a device for question processing, or a control module in the device for question processing that executes the method for question processing. In the embodiments of this specification, taking the device for question processing executing the method for question processing as an example, the device for question processing provided in the embodiments of this specification is described.

[0124] Figure 12 It is a schematic structural diagram of a device for question processing according to an embodiment of the present invention. As Figure 12 shown, the device 1200 for question processing includes:

[0125] A protection dimension module 1210, configured to determine the protection risk dimension to which the question belongs, as well as the protection label and protection level of the question under the belonging protection risk dimension, from multiple protection risk dimensions when receiving a question input by a user to a target large model;

[0126] A protection decision module 1220, configured to determine a protection decision according to the protection label and protection level of the question under the belonging protection risk dimension;

[0127] A content dimension module 1230, configured to determine the content risk dimension to which the question belongs, as well as the content label and content level of the question under the belonging content risk dimension, from multiple content risk dimensions;

[0128] A content decision module 1240, configured to determine a content decision according to the content label and content level of the question under the belonging content risk dimension;

[0129] An answer acquisition module 1250, configured to determine an answer acquisition strategy based on the target large model according to the protection decision and / or the content decision, and execute the answer acquisition strategy, and return the answer obtained after executing the answer acquisition strategy to the user.

[0130] In one embodiment, the answer acquisition module 1250 includes:

[0131] A first answer acquisition unit, configured to determine that the answer acquisition strategy is not to obtain an answer using the target large model when the protection decision or the content decision fails;

[0132] A second answer acquisition unit, configured to determine the answer acquisition strategy as obtaining an answer using a target large model when both the protection decision and the content decision are passed.

[0133] In one embodiment, the protection risk dimension includes a topic dimension. The answer acquisition module 1250 includes:

[0134] A supplementary information acquisition unit, configured to obtain supplementary information corresponding to the question from a data source corresponding to the topic dimension to which the question belongs when the protection risk dimension to which the question belongs includes a topic dimension;

[0135] An answer acquisition unit, configured to input the question and the supplementary information source corresponding to the question into the target large model to obtain an answer when the answer acquisition strategy is to obtain an answer using the target large model.

[0136] In one embodiment, the process of obtaining supplementary information includes:

[0137] When the data source is a database, extract keywords from the question and obtain supplementary information corresponding to the question from the database according to the keywords;

[0138] When the data source is a supplementary information large model, input the question into the supplementary information large model and use the answer corresponding to the question output by the supplementary information large model as the supplementary information corresponding to the question.

[0139] In one embodiment, the content dimension module 1230 includes:

[0140] A target unit, configured to determine a target content risk dimension corresponding to the question from the content risk dimension based on the protection label of the question in the domain dimension to which it belongs when the protection risk dimension includes a domain dimension;

[0141] A dimension unit, configured to determine the content risk dimension to which the question belongs, the content label and the content level of the question in the content risk dimension to which it belongs from the target content risk dimension.

[0142] In one embodiment, determining the content risk dimension to which the question belongs, the content label and the content level of the question in the content risk dimension to which it belongs from the target content risk dimension includes:

[0143] Based on the protection label and the protection level of the question in the protection risk dimension to which it belongs, determine the content risk dimension to which the question belongs, the content label and the content level of the question in the content risk dimension to which it belongs from the target content risk dimension;

[0144] And / or the content decision module 1240 includes:

[0145] A content decision-making unit, configured to determine a content decision according to the protection label and protection level of the question under the protection risk dimension to which the question belongs, and the content label and content level of the question under the content risk dimension to which the question belongs.

[0146] In one embodiment, the protection risk dimension includes an intent dimension, and the protection dimension module 1210 includes:

[0147] A protection risk dimension unit, configured to, when receiving a question, determine the protection risk dimension to which the question belongs from multiple protection risk dimensions;

[0148] A sub-intent dimension unit, configured to, when the protection risk dimension to which the question belongs includes an intent dimension, determine the sub-intent dimension to which the question belongs, as well as the protection label and protection level of the question under the sub-intent dimension to which the question belongs, from the sub-intent dimensions included in the intent dimension.

[0149] In one embodiment, the answer acquisition module 1250 includes:

[0150] A content risk dimension unit, configured to determine the content risk dimension to which the answer belongs, as well as the content label and content level of the answer under the content risk dimension to which the answer belongs, from multiple content risk dimensions;

[0151] A risk identification result unit, configured to determine the risk identification result of the answer according to the content label and content level of the answer under the content risk dimension to which the answer belongs;

[0152] An answer return unit, configured to return the answer to the user when the risk identification result of the answer is a preset risk level.

[0153] It should be noted that the embodiments of the apparatus for question processing in this specification and the embodiments of the method for question processing in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the corresponding implementation of the method for question processing described above, and the repeated parts will not be elaborated.

[0154] Each module in the above-mentioned apparatus for question processing can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor in the terminal device or the processor in the server in the form of hardware, or can be stored in the memory in the terminal device or the memory in the server in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0155] Furthermore, corresponding to the method for question processing described above, based on the same technical concept, one or more embodiments of this specification also provide an electronic device, which is used to execute the above-mentioned method for question processing. Figure 13 It is a schematic structural diagram of an electronic device provided by one or more embodiments of this specification.

[0156] Based on the same idea, one or more embodiments of this specification also provide an electronic device, as Figure 13 shown. The electronic device can vary greatly due to configuration or performance differences, and may include one or more processors 1301 and a memory 1302. One or more stored application programs or data can be stored in the memory 1302. Among them, the memory 1302 can be transient storage or persistent storage. The application programs stored in the memory 1302 can include one or more modules (not shown in the figure), and each module can include a series of computer-executable instructions for the electronic device. Further, the processor 1301 can be set to communicate with the memory 1302 and execute a series of computer-executable instructions in the memory 1302 on the electronic device. The electronic device can also include one or more power supplies 1303, one or more wired or wireless network interfaces 1304, one or more input / output interfaces 1305, and one or more keyboards 1306.

[0157] In a specific embodiment, the electronic device includes a memory and one or more programs. One or more of the programs are stored in the memory, and one or more of the programs can include one or more modules. Each module can include a series of computer-executable instructions for the electronic device and is configured to be executed by one or more processors. The one or more programs include the following computer-executable instructions for:

[0158] In the case of receiving a user's question about the target large model, determine the protection risk dimension to which the question belongs, as well as the protection label and protection level of the question under the belonging protection risk dimension from multiple protection risk dimensions;

[0159] Determine a protection decision based on the protection label and protection level of the question under the belonging protection risk dimension;

[0160] Determine the content risk dimension to which the question belongs, as well as the content label and content level of the question under the belonging content risk dimension from multiple content risk dimensions;

[0161] Determine a content decision based on the content label and content level of the question under the belonging content risk dimension;

[0162] Determine an answer acquisition strategy based on the target large model according to the protection decision and / or content decision, and execute the answer acquisition strategy to return the answer obtained after executing the answer acquisition strategy to the user.

[0163] It should be noted that the embodiments of the electronic device in this specification and the embodiments of the method for question processing in this specification are based on the same inventive concept. Therefore, for the specific implementation of this embodiment, reference may be made to the corresponding implementation of the method for question processing described above, and repeated parts will not be elaborated.

[0164] Furthermore, corresponding to the method for question processing described above and based on the same technical concept, one or more embodiments of this specification also provide a storage medium for storing computer-executable instructions. In a specific embodiment, the storage medium may be a USB flash drive, an optical disc, a hard disk, etc. When the computer-executable instructions stored in the storage medium are executed by a processor, the following processes can be implemented:

[0165] In the case of receiving a question from a user input to a target large model, determine the protection risk dimension to which the question belongs, as well as the protection label and protection level of the question under the protection risk dimension to which it belongs, from multiple protection risk dimensions;

[0166] Determine a protection decision based on the protection label and protection level of the question under the protection risk dimension to which it belongs;

[0167] Determine the content risk dimension to which the question belongs, as well as the content label and content level of the question under the content risk dimension to which it belongs, from multiple content risk dimensions;

[0168] Determine a content decision based on the content label and content level of the question under the content risk dimension to which it belongs;

[0169] Determine an answer acquisition strategy based on the target large model according to the protection decision and / or the content decision, and execute the answer acquisition strategy, and return the answer obtained after executing the answer acquisition strategy to the user.

[0170] It should be noted that the embodiments of the storage medium in this specification and the method for question processing in this specification are based on the same inventive concept. Therefore, for the specific implementation of this embodiment, reference may be made to the corresponding implementation of the method for question processing described above, and repeated parts will not be elaborated.

[0171] Furthermore, corresponding to the method for question processing described above and based on the same technical concept, one or more embodiments of this specification also provide a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following processes can be implemented:

[0172] In the case of receiving a question from a user input to a target large model, determine the protection risk dimension to which the question belongs, as well as the protection label and protection level of the question under the protection risk dimension to which it belongs, from multiple protection risk dimensions;

[0173] Determine the protection decision based on the protection label and protection level under the protection risk dimension to which the question belongs;

[0174] From multiple content risk dimensions, determine the content risk dimension to which the question belongs, as well as the content label and content level of the question under the content risk dimension to which it belongs;

[0175] Determine the content decision based on the content label and content level of the question under the content risk dimension to which it belongs;

[0176] Based on the protection decision and / or content decision, determine the answer acquisition strategy based on the target large model, and execute the answer acquisition strategy, and return to the user the answer obtained after executing the answer acquisition strategy.

[0177] It should be noted that the embodiments of the computer program product in this specification and the embodiments of the method for question processing in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the corresponding implementation of the aforementioned method for question processing, and the repeated parts will not be elaborated.

[0178] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the specific order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0179] In the 1990s, it was obvious to distinguish whether an improvement to a technology was a hardware improvement (e.g., improvement to the circuit structures such as diodes, transistors, switches, etc.) or a software improvement (improvement to the method flow). However, with the development of technology, many improvements to method flows today can be regarded as direct improvements to hardware circuit structures. Almost all designers obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement to a method flow cannot be implemented with a hardware entity module. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user's programming of the device. The designer can program by himself to "integrate" a digital system on a PLD, without having to ask a chip manufacturer to design and fabricate a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, which is similar to the software compiler used in program development and writing. The original code before compilation also has to be written in a specific programming language, which is called a Hardware Description Language (HDL), and there is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones currently are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be aware that as long as the method flow is slightly logically programmed with the above-mentioned several hardware description languages and programmed into the integrated circuit, it is easy to obtain the hardware circuit that implements the logical method flow.

[0180] The controller can be implemented in any suitable manner. For example, the controller can take the form of, for example, a microprocessor or a processor and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of the controller include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that, in addition to implementing the controller in the form of pure computer-readable program code, it is entirely possible to make the controller implement the same function in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as either software modules for implementing the method or structures within the hardware component.

[0181] The systems, devices, modules, or units illustrated in the above embodiments can be specifically implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0182] For the convenience of description, when describing the above devices, they are described separately as various units according to their functions. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0183] Those skilled in the art should understand that one or more embodiments of this specification can be provided as a method, a system, or a computer program product. Therefore, one or more embodiments of this specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0184] This specification is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the specification. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general purpose computers, special purpose computers, embedded processors, or other programmable data processing devices to produce a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0185] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0186] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0187] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0188] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0189] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information accessible by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0190] It should also be noted that the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article or apparatus comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.

[0191] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

[0192] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and reference can be made to the corresponding parts of the method embodiments for relevant content.

[0193] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A method for question processing, comprising: When receiving a question input by a user to a target large model, determining, from multiple protection risk dimensions, the protection risk dimension to which the question belongs, as well as the protection label and protection level of the question under the belonging protection risk dimension, wherein the protection risk dimension includes an intent dimension, and the protection label under the intent dimension is used to filter adversarial questions with high-risk intents. The protection labels under the attack sub-dimension in the intent dimension include a target hijacking protection label and a reverse induction protection label. The target hijacking protection label is used for a situation where a normal question instruction is sent first, followed immediately by an imperative instruction related to the previous instruction. The reverse induction protection label is used for a situation where harmful information is deliberately obtained through a positive scenario setting; Determining a protection decision according to the protection label and protection level of the question under the belonging protection risk dimension; Determining, from multiple content risk dimensions, the content risk dimension to which the question belongs, as well as the content label and content level of the question under the belonging content risk dimension; Determining a content decision according to the content label and content level of the question under the belonging content risk dimension; Determining an answer acquisition strategy based on the target large model according to the protection decision and / or the content decision, and executing the answer acquisition strategy, and returning an answer obtained after executing the answer acquisition strategy to the user.

2. The method according to claim 1, wherein determining an answer acquisition strategy based on the target large model according to the protection decision and / or the content decision includes: When the protection decision or the content decision is not passed, determining the answer acquisition strategy as not using the target large model to obtain the answer; When both the protection decision and the content decision are passed, determining the answer acquisition strategy as using the target large model to obtain the answer.

3. The method according to claim 2, wherein the protection risk dimension includes a topic dimension, and executing the answer acquisition strategy includes: When the protection risk dimension to which the question belongs includes the topic dimension, obtaining supplementary information corresponding to the question from a data source corresponding to the topic dimension to which the question belongs; When the answer acquisition strategy is to use the target large model to obtain the answer, inputting the question and the supplementary information source corresponding to the question into the target large model to obtain the answer.

4. The method according to claim 3, wherein the process of obtaining the supplementary information includes: When the data source is a database, extracting keywords from the question, and obtaining supplementary information corresponding to the question from the database according to the keywords; When the data source is a supplementary information large model, inputting the question into the supplementary information large model, and using the answer corresponding to the question output by the supplementary information large model as the supplementary information corresponding to the question.

5. According to the method described in claim 1, when the protection risk dimension includes the domain dimension, determining the content risk dimension to which the question belongs, the content label, and the content level of the question under the content risk dimension from multiple content risk dimensions includes: Based on the protection label of the question under the domain dimension to which it belongs, determining the target content risk dimension corresponding to the question from the content risk dimensions; Determining the content risk dimension to which the question belongs, the content label, and the content level of the question under the content risk dimension from the target content risk dimension.

6. According to the method described in claim 5, determining the content risk dimension to which the question belongs, the content label, and the content level of the question under the content risk dimension from the target content risk dimension includes: Based on the protection label and protection level of the question under the protection risk dimension to which it belongs, determining the content risk dimension to which the question belongs, the content label, and the content level of the question under the content risk dimension from the target content risk dimension; and / or determining the content decision according to the content label and content level of the question under the content risk dimension to which it belongs, including: Determining the content decision according to the protection label and protection level of the question under the protection risk dimension to which it belongs, and the content label and content level of the question under the content risk dimension to which it belongs.

7. According to the method described in claim 1, when receiving a question input by the user to the target large model, determining the protection risk dimension to which the question belongs, the protection label, and the protection level of the question under the protection risk dimension from multiple protection risk dimensions includes: When receiving the question, determining the protection risk dimension to which the question belongs from multiple protection risk dimensions; When the protection risk dimension to which the question belongs includes the intention dimension, determining the sub-intention dimension to which the question belongs, the protection label, and the protection level of the question under the sub-intention dimension from the sub-intention dimensions included in the intention dimension.

8. According to the method described in claim 1, returning the answer obtained after executing the answer acquisition strategy to the user includes: Determining the content risk dimension to which the answer belongs, the content label, and the content level of the answer under the content risk dimension from multiple content risk dimensions; Determining the risk identification result of the answer according to the content label and content level of the answer under the content risk dimension to which it belongs; When the risk identification result of the answer is a preset risk level, returning the answer to the user.

9. A question processing device includes: A protection dimension module, which is used to determine the protection risk dimension to which the question belongs, the protection label and protection level of the question under the belonging protection risk dimension from multiple protection risk dimensions when receiving a question input by the user to the target large model. Among them, the protection risk dimensions include an intention dimension, and the protection label under the intention dimension is used to filter adversarial questions with high-risk intentions. The protection labels under the attack sub-dimension in the intention dimension include a target hijacking protection label and a reverse induction protection label. The target hijacking protection label is used for a situation where a normal question instruction is issued first, followed by an imperative instruction related to the previous instruction. The reverse induction protection label is used for a situation where harmful information is deliberately obtained through a positive scenario setting; A protection decision-making module, which is used to determine a protection decision according to the protection label and protection level of the question under the belonging protection risk dimension; A content dimension module, which is used to determine the content risk dimension to which the question belongs, the content label and content level of the question under the belonging content risk dimension from multiple content risk dimensions; A content decision-making module, which is used to determine a content decision according to the content label and content level of the question under the belonging content risk dimension; An answer acquisition module, which is used to determine an answer acquisition strategy based on the target large model according to the protection decision and / or the content decision, execute the answer acquisition strategy, and return the answer obtained after executing the answer acquisition strategy to the user.

10. An electronic device, comprising: a processor, and a memory arranged to store computer-executable instructions, which when executed, can cause the processor to: When receiving a question input by the user to the target large model, determine the protection risk dimension to which the question belongs, the protection label and protection level of the question under the belonging protection risk dimension from multiple protection risk dimensions. Among them, the protection risk dimensions include an intention dimension, and the protection label under the intention dimension is used to filter adversarial questions with high-risk intentions. The protection labels under the attack sub-dimension in the intention dimension include a target hijacking protection label and a reverse induction protection label. The target hijacking protection label is used for a situation where a normal question instruction is issued first, followed by an imperative instruction related to the previous instruction. The reverse induction protection label is used for a situation where harmful information is deliberately obtained through a positive scenario setting; Determine a protection decision according to the protection label and protection level of the question under the belonging protection risk dimension; Determine the content risk dimension to which the question belongs, the content label and content level of the question under the belonging content risk dimension from multiple content risk dimensions; Determine a content decision according to the content label and content level of the question under the belonging content risk dimension; Determine an answer acquisition strategy based on the target large model according to the protection decision and / or the content decision, execute the answer acquisition strategy, and return the answer obtained after executing the answer acquisition strategy to the user.

Images