Method and device for processing cue word, equipment and medium

Abstract

The invention provides a method, a device, equipment and a medium for processing cue words. In one method, a first cue word is received that expresses a question to be input to a machine learning model in a natural language. And determining whether there is a potential deviation between the answer of the machine learning model to the first cue word and the real answer of the question. And in response to determining that the potential deviation exists, converting the first cue word into a second cue word based on a keyword associated with the potential deviation in the first cue word. And receiving an answer of the machine learning model for the second cue word. By using the exemplary implementation mode of the disclosure, the deviation in the answers can be relieved and / or eliminated by adjusting the cue words under the condition that the machine learning model is not modified.

Description

Technical Field

[0001] Exemplary implementations of the present disclosure generally relate to machine learning models, and more particularly to methods, devices, apparatus, and computer-readable storage media for processing prompt words of machine learning models. Background Art

[0002] Machine learning technology has been widely used to perform many types of tasks. For example, questions can be asked to a machine learning model and answers can be received from the machine learning model. However, the training data of the machine learning model may include unfair data, which may cause the answers provided by the machine learning model to not be completely consistent with the facts, and some answers may deviate from the true answers. At this time, it is expected that a true answer that is more consistent with the facts can be obtained. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for processing prompt words is provided. In the method, a first prompt word is received, the first prompt word expressing a question to be input into a machine learning model in natural language. It is determined whether there is a potential deviation between the answer of the machine learning model to the first prompt word and the true answer to the question. In response to determining that the potential deviation exists, the first prompt word is converted into a second prompt word based on a keyword in the first prompt word associated with the potential deviation. The answer of the machine learning model to the second prompt word is received.

[0004] In a second aspect of the present disclosure, a device for processing prompt words is provided. The device includes: a first receiving module configured to receive a first prompt word, the first prompt word expressing a question to be input into a machine learning model in natural language; a determination module configured to determine whether there is a potential deviation between the answer of the machine learning model to the first prompt word and the true answer to the question; a conversion module configured to convert the first prompt word into a second prompt word based on a keyword associated with the potential deviation in the first prompt word in response to determining that there is a potential deviation; and a second receiving module configured to receive the answer of the machine learning model to the second prompt word.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The electronic device includes: at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, and when the instructions are executed by the at least one processing unit, the electronic device executes the method according to the first aspect of the present disclosure.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor implements the method according to the first aspect of the present disclosure.

[0007] It should be understood that the content described in this content section is not intended to limit the key features or important features of the implementation of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easy to understand through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In the following, in conjunction with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages and aspects of various implementations of the present disclosure will become more apparent. In the accompanying drawings, the same or similar figure labels represent the same or similar elements, wherein:

[0009] Figure 1A A block diagram showing an application environment according to an exemplary implementation of the present disclosure is shown;

[0010] Figure 1B A block diagram of determining an answer to a question based on different knowledge according to a technical solution is shown;

[0011] Figure 2 A block diagram for processing prompt words according to some implementations of the present disclosure is shown;

[0012] Figure 3 A block diagram for updating prompt words according to some implementations of the present disclosure is shown;

[0013] Figure 4A A block diagram illustrating training data according to some implementations of the present disclosure;

[0014] Figure 4B A block diagram showing the relationship between various representations involved in a machine learning model according to some implementations of the present disclosure;

[0015] Figure 4C A block diagram showing the relationship between various representations involved in a machine learning model according to some implementations of the present disclosure, wherein bold lines show paths in the machine learning model;

[0016] Figure 5A A block diagram illustrating dependencies in a machine learning model according to some implementations of the present disclosure;

[0017] Figure 5B A block diagram illustrating a method for weakening a path in a machine learning model according to some implementations of the present disclosure is shown;

[0018] Fig. 6AA block diagram illustrating dependencies in a machine learning model according to some implementations of the present disclosure;

[0019] Figure 6B A block diagram illustrating a method for weakening a path in a machine learning model according to some implementations of the present disclosure is shown;

[0020] Fig. 7A A block diagram illustrating dependencies in a machine learning model according to some implementations of the present disclosure;

[0021] Figure 7B A block diagram illustrating a method for weakening a path in a machine learning model according to some implementations of the present disclosure is shown;

[0022] Figure 8 A flowchart of a method for processing a prompt word according to some implementations of the present disclosure is shown;

[0023] Fig. 9 A block diagram showing an apparatus for processing a prompt word according to some implementations of the present disclosure; and

[0024] Fig.10 A block diagram of a device capable of implementing multiple implementations of the present disclosure is shown. DETAILED DESCRIPTION

[0025] The implementation of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain implementations of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be interpreted as being limited to the implementations described herein. On the contrary, these implementations are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and implementations of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0026] In the description of the implementation methods of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "an implementation" or "the implementation" should be understood as "at least one implementation". The term "some implementations" should be understood as "at least some implementations". The following may also include other explicit and implicit definitions. As used herein, the term "model" can represent the association relationship between various data. For example, the above-mentioned association relationship can be obtained based on a variety of technical solutions currently known and / or to be developed in the future.

[0027] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0028] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0029] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0030] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information is sent to the user in a manner such as a pop-up window, in which the prompt information can be presented in text form. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0031] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0032] The term "in response to" as used herein refers to a state in which a corresponding event occurs or a condition is satisfied. It will be understood that the timing of executing a subsequent action executed in response to the event or condition is not necessarily strongly related to the time when the event occurs or the condition is satisfied. For example, in some cases, the subsequent action may be executed immediately when the event occurs or the condition is satisfied; while in other cases, the subsequent action may be executed some time after the event occurs or the condition is satisfied.

[0033] Example Environment

[0034] Machine learning technology has been widely used to perform many types of tasks. For example, questions can be asked to a machine learning model, and answers from the machine learning model can be received. The machine learning model 120 can be, for example, a language model (such as a large language model, etc.). It has been proposed to use a large-scale text corpus to train a machine learning model, and the machine learning model generated in this way may have social biases. Unconstrained biases may continue and expand social inequality and even produce more serious consequences. With the development of machine learning model technology, machine learning models have been widely used in fields such as recruitment and medical care that require high accuracy. In such fields, it is particularly necessary to eliminate the bias of machine learning models.

[0035] See also Figure 1A Describes an application environment according to an example implementation of the present disclosure. Figure 1A A block diagram 100A of an application environment according to an exemplary implementation of the present disclosure is shown. As shown in Figure 1, a page 110 may be provided to allow a user to interact with a machine learning model 120. For example, a user may ask a question 112 and receive an answer 114 from the machine learning model.

[0036] In question 112, which states: In the sentence "The CEO hired the secretary because he came highly recommended", who is "he"? The machine learning model may provide different answers. Figure 1B Describe more information, the Figure 1B A block diagram 100B is shown for determining answers to questions based on different knowledge according to a technical solution. Question 112 can be answered based on different knowledge. For example, knowledge 120 indicates that he is often associated with CEOs rather than secretaries (based on occupational statistics). At this time, if the machine learning model answers question 112 based on knowledge 120, answer 122 may indicate "CEO", that is, in the above sentence, "He" refers to the CEO.

[0037] For another example, knowledge 140 indicates that, compared to "the CEO hired a secretary because the CEO was highly recommended", "the CEO hired a secretary because the secretary was highly recommended" is more likely to occur in real life. In this case, if the machine learning model answers question 112 based on knowledge 140, answer 142 may indicate "secretary", that is, in the above sentence, "He" refers to the secretary.

[0038] It can be seen that since the training data of different machine learning models can be different, the answers provided by some machine learning models will also be different, and some answers may deviate from the true answers. In other words, machine learning models may provide biased answers.

[0039] It should be understood that although the above describes questions, answers, and knowledge using English as an example of a natural language, alternatively and / or additionally, questions, answers, and knowledge can be expressed in other languages ​​(e.g., Chinese, French, etc.). Alternatively and / or additionally, a translation model can be used to perform conversions between different natural languages.

[0040] At present, a variety of technical solutions have been proposed, for example, different training data can be selected, model parameters can be fine-tuned, decoding steps can be modified, etc. However, such technical solutions may involve a large amount of calculation. Although it is currently proposed to reduce the possibility of deviation in the answer based on modifying the prompt words, the effect of such technical solutions is not satisfactory. At this time, it is expected to obtain a true answer that is more in line with the facts.

[0041] Summary of processing prompt words

[0042] In order to at least partially address the deficiencies in the prior art, according to an exemplary implementation of the present disclosure, a method for processing prompt words is proposed. The present disclosure focuses on social bias, involving the association between demographic information and the answers output by the machine learning model. Figure 2 Describes an overview of an exemplary implementation according to the present disclosure, Figure 2 A block diagram 200 for processing a prompt word according to some implementations of the present disclosure is shown.

[0043] like Figure 2 As shown, a first prompt word 210 can be received, and the first prompt word 210 expresses a question to be input into the machine learning model in natural language. It can be determined whether there is a potential deviation 212 between the answer of the machine learning model to the first prompt word 210 and the true answer to the question. According to an example implementation of the present disclosure, a variety of methods currently known and / or to be developed in the future can be used to determine whether the above-mentioned deviation exists. For example, it can be determined by rules, text analysis, and / or models whether the first prompt word includes keywords that may cause deviations, etc. If it is determined that there is a potential deviation, the first prompt word 210 can be converted into a second prompt word 220 based on the keywords 214 associated with the potential deviation in the first prompt word 210, and then the answer 240 of the machine learning model 220 to the second prompt word 220 is received.

[0044] Using the exemplary implementation of the present disclosure, the bias in the answer can be mitigated and / or eliminated by adjusting the prompt words without modifying the machine learning model itself. The prompt words can be updated based on keywords that have a causal relationship with the potential bias. Furthermore, a framework for removing bias using causal relationships is proposed, which mainly involves the following points: (1) the generation process of the data of the training corpus fed to the machine learning model; and (2) the internal reasoning process of the machine learning model to establish the prompt words through the selection mechanism, thereby removing the bias from the output of the machine learning model.

[0045] In the process of updating the prompt word, inhibitory instructions and contextual comparison examples can be used, and methods to mitigate and / or eliminate bias can be implemented through unbiased reasoning. Extensive experiments on real-world datasets show that the proposed technical solution can effectively reduce the bias in the output of machine learning models.

[0046] Detailed process of processing prompt words

[0047] Having described an overview of an example implementation according to the present disclosure, more information about processing prompt words will be provided below. In the context of the present disclosure, prompt words can be modified to guide the machine learning model to provide unbiased (or less biased) answers.

[0048] It should be understood that an unbiased answer essentially involves the process of selecting appropriate parts from the model's internal representation and knowledge. For the above-described problem of determining the referent of a pronoun in a sentence, the model may use the shortcut path about gender learned from the training dataset and output a biased answer. For example, in Figure 1B In the training data, there may be stereotypes and social biases in the training data about the gender distribution of occupations. The model may associate the pronoun "he" with "CEO" instead of "secretary". The above decision process can be represented as biased reasoning, where the model chooses internal representations in an inappropriate way.

[0049] In the context of the present disclosure, the selection mechanism plays an important role in the interaction between the internal reasoning process of the machine learning model and different designs of external prompts. In addition to the data generation process of the training corpus, a potential causal model of the machine learning model in the reasoning process can be constructed, and different paths in the causal model can be selected by prompt words, so that the machine learning model outputs different results. To this end, a causal relationship framework is proposed based on the following: a) reducing biased reasoning, and b) encouraging unbiased reasoning.

[0050] In summary, Figure 3 shows different updates based on the prompt word. Figure 3 A block diagram 300 for updating a prompt word according to some implementations of the present disclosure is shown. Figure 3 As shown, update 310 involves suppressing instructions, which is achieved by reducing (e.g., represented as [-]) biased reasoning (explicit); update 320 involves contextual comparison examples, which is achieved by reducing biased reasoning (implicit) and improving (e.g., represented as [+]) unbiased reasoning (implicit); and update 330 involves approaching facts, which is achieved by improving unbiased reasoning (explicit).

[0051] According to an example implementation of the present disclosure, a detailed causal model is constructed for the data generation process of the training corpus and the reasoning process of the machine learning model. Different selection mechanisms are implemented in the machine learning model by modifying the prompt words, which has an important impact on controlling the output results of the machine learning model. A framework for removing bias based on causal relationships and specific modification patterns for adjusting prompt words are proposed. Experiments have shown that these modification patterns alleviate and / or eliminate various social biases, thereby improving the correctness of the answers output by the machine learning model.

[0052] First, we describe the outline of causality: for two random variables X and Y, if the distribution of Y changes when intervening in X while keeping all other variables fixed, then X is the cause of Y. The causal relationship between variables can be represented using a directed acyclic graph (DAG), where nodes represent variables and edges represent direct causal relationships between variables.

[0053] According to an example implementation of the present disclosure, a bias removal framework is proposed, which provides guidance on the causal relationship of the data generation process involved. Specifically, a detailed causal model of the basic data generation process of the training corpus and the reasoning process of the machine learning model is proposed. It should be understood that the reasoning process of the machine learning model is essentially the interaction between the internal representation of the model and the conditions specified by the external prompt words, and the selection mechanism plays a key role in this. Further, different modification modes can be used to modify the prompt words in order to remove bias.

[0054] Specifically, causal modeling of the underlying data generation process of the training data corpus and the inference process of the machine learning model are proposed. In the following, the underlying data generation process of the text in the training data corpus will be described. For historical reasons, etc., the underlying training data corpus may be biased and usually involves a selection mechanism. For example, stereotypes in career choices are not due to the real direct causal relationship between gender and career, but due to the potential selection mechanism of the training data. Specifically, in the training data, among all possible combinations between gender (such as male and female) and career (such as CEO and secretary), CEOs are usually associated with men, while secretaries are associated with women. This phenomenon occurs because the training data selects a specific subset of stereotypes from the ideal data corpus.

[0055] Figure 4A A block diagram 400A showing training data according to some implementations of the present disclosure is shown. Figure 4AAs shown, the causal relationship in the basic data generation process of the training data corpus (e.g., text collected from the Internet) can be modeled. Here, the demographic information 410 can, for example, involve gender, age, etc. Further, the causal relationship also includes other variables of interest. For example, a scene 412 can be used to represent the actual environment or background (e.g., healthcare, recruitment) that sets the text background. Entity 411 can be used to represent the participants or stakeholders involved in the scene, for example, a doctor may be an entity in a healthcare scene, and a secretary may be an entity in a recruitment scene.

[0056] The selection mechanism S explicitly models the association between demographic information and entities, which is a stereotype in natural language processing. There can be different types of text: demographic agnostic facts 414, which means text that does not explicitly contain demographic information; fact-based derivative text 415, which means text derived from facts, such as inferences based on definitions, fact-checking questions and answers, and restatements that do not change the factual content; and demographic aware text 413, which means text in which demographic information explicitly appears.

[0057] In the following, we describe the causal modeling of the underlying data generation process and how to distinguish discrimination in the training data corpus of the machine learning model. We expect that the machine learning model can capture the dependency patterns in the data corpus and thus extract the intrinsic relevant knowledge. We assume that the internal reasoning process of the machine learning model has similarities with the underlying data generation process of the data corpus. Figure 4B Block diagram 400B showing the relationship between various representations involved in a machine learning model according to some implementations of the present disclosure. Prompt word 426 indicates a prompt word input by a user. Demographic representation 420, entity representation 421, and scene representation 422 indicate internal representations in a machine learning model associated with demographic information, entity information, and scene information, respectively. Further, demographic-aware text representation 424 indicates the representation of demographic-aware text inside the machine learning model, and demographic-agnostic fact representation indicates the representation of demographic-agnostic facts inside the machine learning model, and model output 427 indicates a potential output of the machine learning model.

[0058] at this time, Figure 4B Describes the reasoning process of how a machine learning model generates output based on input prompts. The diagram shows the interaction between internal representations and conditions specified by external inputs, and that different prompts can adjust the output of the machine learning model through the selection mechanism. The internal nodes of the machine learning model are not visible. Figure 4B The arrows in represent directed edges (that is, there is a direct causal relationship between the two nodes before and after the arrow), and the dotted arrows represent selection mechanisms.

[0059] Hints are external inputs to machine learning models. The internal knowledge and representation of a machine learning model pre-exist and are independent of the presence of a specific hint. Hints also cannot be used as indicators of causal intervention. Since internal nodes are invisible, they cannot be set to specific values ​​through hard intervention, nor can the functional behavior of causal modules be changed through soft intervention. However, the selection variable "Proper Consideration of Hints" (PPC) can be directly changed by adjusting the hint. If the machine learning model is well trained and fine-tuned, it can be assumed that the PPC always depends on when the model produces output, and the "potential output of the machine learning model" is the actual output obtained from the model. Hints can affect the machine learning model reasoning process in a significant way by specifying conditions in the selection mechanism.

[0060] Although not relevant to machine learning model inference itself, Figure 4A The modeling base generation process shown provides hints about local causal modules of interest, and updating hint words can adjust the influence of these modules to achieve the purpose of removing bias. Since the conditions specified by the hint words always depend on when the machine learning model generates output, the internal reasoning is controlled by the external hint words. Under the guidance of the causal understanding of the relevant data generation process, the present disclosure proposes a hint-based machine learning model bias removal framework. The conditions that should be followed during the hint word update process are used to influence the machine learning model reasoning process through the selection mechanism, thereby effectively alleviating and / or eliminating the bias of the machine learning model output.

[0061] Figure 4C Block diagram 400C showing the relationship between various representations involved in a machine learning model according to some implementations of the present disclosure, where bold lines show paths in the machine learning model. Figure 4C An annotated version of the inference process of a machine learning model. Based on an understanding of the underlying generative process of the training data corpus and the assumption that the trained machine learning model captures the dependency patterns in the training data, Figure 4C The arrows shown in bold in the figure represent the unregulated information flow from the demographic representation to the output of the machine learning model during the internal reasoning process. As shown in the figure, arrows 431, 432, 433, 434, 435, 436 and 437 represent selection mechanisms, and arrows 441, 442, 443, 444, 445 represent dependency relationships. Among the selection mechanisms, arrows in thin-line format represent selection mechanisms that can be specified by external input prompts, and arrows in bold-line format identify selection mechanisms that may involve unfair knowledge.

[0062] according to Figure 4CIn the machine learning model reasoning process shown, there is an information flow from the demographic representation (upstream node) to the model output (downstream node) of the machine learning model. In order to remove the bias, the prompt words should follow the predetermined conditions and constraints. Specifically, three modification modes for eliminating the bias of the machine learning model are proposed.

[0063] According to an example implementation of the present disclosure, in the process of converting a first prompt word into a second prompt word, an additional prompt word can be generated based on a keyword associated with a potential bias in the first prompt word, and then the first prompt word and the additional prompt word are combined to generate a second prompt word. In the above example, "he" can represent a keyword. Further, the first prompt word can be used as the main part of the prompt word, and the additional prompt word can be used as a restriction condition for the main part, so that the machine learning model can output a more accurate answer. Specifically, it can be achieved by reducing biased reasoning (explicit), by reducing biased reasoning (implicit), improving unbiased reasoning (implicit), and / or by improving unbiased reasoning (explicit).

[0064] Specifically, multiple ways for updating the prompt words can be provided. According to an example implementation of the present disclosure, a modification mode I can be provided, that is, promoting the use of demographic unknowable facts. The principle of this modification mode is to promote the machine learning model to make more use of demographic unknowable facts when generating outputs. Specifically, condition I means that in the presence of PPC and existing choices, the internal representation of demographic unknowable facts and demographic information should be conditionally independent.

[0065] Figure 5A A block diagram 500A is shown of dependencies in a machine learning model according to some implementations of the present disclosure. Figure 5A As shown, "demographic representation" indicates a demographic representation, and "demographic-agnostic fact representation" indicates a representation of demographic-agnostic facts. When S=1 and PPC=1, there is a conditional independence relationship between the two. Figure 5B Block diagram 500B is shown for weakening a path in a machine learning model according to some implementations of the present disclosure. Figure 5B As shown by arrows 431 and 435 in FIG. 4 , a selection mechanism can be introduced through demographic information and demographic unknowable facts using modification mode I. Further, the dependency relationship shown by reference numerals 520 and 522 can be weakened.

[0066] According to an example implementation of the present disclosure, multiple entities corresponding to the keyword are determined based on the first prompt word; the keywords are replaced with the multiple entities to generate multiple candidate hypotheses; and additional prompt words are generated using the multiple candidate hypotheses. In the above example, the keyword is "he". At this time, the keywords can be replaced with the entities referred to by "he", such as CEO and secretary, to generate new sentences (i.e., candidate hypotheses). At this time, the keywords in the square brackets in the sentence are replaced with CEO and secretary respectively.

[0067] Sentence 1: “The CEO hired the secretary because [the CEO] is highly recommended”.

[0068] Sentence 2: “The CEO hired the secretary because [the secretary] is highly recommended”.

[0069] According to an example implementation of the present disclosure, a second prompt word may be created, and the second prompt word may be used to instruct the machine learning model to provide an answer to the question based on multiple candidate hypotheses. For example, the updated prompt word may be as follows.

[0070] Table 1 Examples of updated prompt words

[0071]

[0072]

[0073] Using the example implementation of the present disclosure, when generating an answer, the machine learning model can select the situation that is more likely to occur in real life from Sentence 1 and Sentence 2, thereby providing an unbiased answer "the secretary".

[0074] According to an example implementation of the present disclosure, additional prompt words are generated to weaken the influence of network nodes associated with the cause of deviation in the machine learning model. Specifically, the cause of deviation associated with the keyword can be determined, and then based on the cause of deviation, additional prompt words are generated, and the additional prompt words are used to inhibit the machine learning model from activating the network nodes associated with the cause of deviation. Using the example implementation of the present disclosure, the machine learning model can weaken nodes that may be related to outdated knowledge about the occupation in the process of generating answers using updated prompt words, thereby generating unbiased answers. For example, the prompt words can be updated based on the following modification mode II and modification mode III.

[0075] According to an exemplary implementation of the present disclosure, modification mode II can offset the existing selection bias. The principle of this modification mode is to directly offset the influence of the existing knowledge that causes the bias. Fig. 6A A block diagram 600A shows dependencies in a machine learning model according to some implementations of the present disclosure. Fig. 6A As shown, "demographicrepresentation" indicates the representation of demographics, and "entity representation" indicates the representation of entities. The upper part of relationship 610 indicates condition II.1: in the presence of PPC and the appearance of selection S, the representation of demographic information and the entity are conditionally independent. That is, in the case of S = 1 and PPC = 1, there is a conditionally independent relationship between the two. Further, the lower part of relationship 610 indicates condition II.2: no new association is introduced between the internal representation of demographic information and demographic unknowable facts.

[0076] Figure 6B A block diagram 600B is shown for weakening paths in a machine learning model in accordance with some implementations of the present disclosure. Condition II.1 is intended to offset the bias caused by the selection (S nodes) (arrows 436 and 437) by constraining the dependency between the representation of demographic information and the entity (arrows 431 and 432). That is, the dependency relationship shown in the figure numerals 620 and 620 can be weakened. Condition II.2 is used for monitoring purposes to ensure that no new differences are introduced in the causal relationship downstream of the entity, so that the above-mentioned offsetting action can be applied to the final output.

[0077] According to an example implementation of the present disclosure, in the process of generating additional prompt words based on the cause of the deviation, multiple candidate facts associated with the cause of the deviation can be determined, and then the additional prompt words are generated based on the multiple candidate facts. In the example described above, the cause of the potential deviation lies in the distribution in the occupational statistics. At this time, the following sentence can be generated and the facts about the occupational distribution can be stated:

[0078] Sentence 3: "CEO can both be male and female with equal probability".

[0079] Sentence 4: "secretary can both be male and female with equal probability".

[0080] According to an example implementation of the present disclosure, a second prompt word may be generated including, and the second prompt word may instruct the machine learning model to provide an answer to the question based on multiple candidate facts. For example, the updated prompt word may be as follows.

[0081] Table 2 Examples of updated prompt words

[0082]

[0083] Using the example implementation of the present disclosure, when generating an answer, the machine learning model can select an answer that is more consistent with the grammar and logic of the sentence from the CEO and the secretary, based on the facts provided by Sentences 3 and 4. At this time, the machine learning model can provide an unbiased answer "the secretary".

[0084] According to an example implementation of the present disclosure, modification mode III can move away from demographically-aware text. The principle of this modification mode is to encourage the machine learning model to no longer utilize demographically-aware text to generate output. Fig. 7A A block diagram 700A shows dependencies in a machine learning model according to some implementations of the present disclosure. Fig. 7A As shown in the relation 710 in , “demographic representation” indicates the representation of demographics, and “demographic-aware text representation” indicates the representation of demographically-aware text. Relation 710 involves condition III: there should be a conditional independence relationship between the internal representation of demographically-aware text and demographic information, that is, when S=1 and PPC=1, there is a conditional independence relationship between the two.

[0085] Figure 7B Block diagram 700B is shown for weakening a path in a machine learning model according to some implementations of the present disclosure. Figure 7B As shown, modification mode III can utilize condition III to specify the selection mechanism of the internal representation of demographic information and demographic-aware text (such as arrows 431 and 434) to adjust the dependency relationship shown as reference numeral 720.

[0086] According to an example implementation of the present disclosure, in the process of generating additional prompt words based on the cause of the deviation, the type of knowledge associated with the cause of the deviation can be determined, and the additional prompt words can be generated based on the type of knowledge. In this way, the deviation in the answer can be reduced by disabling certain types of knowledge. In the example described above, the cause of the potential deviation lies in the distribution in the occupational statistics. At this time, the following sentence can be generated, and the use of related knowledge is prohibited:

[0087] Sentence 5: "Please don't be biased or please do not use any gender-related information when making the decision."

[0088] According to an example implementation of the present disclosure, a second prompt word may be generated that may instruct the machine learning model to provide an answer to the question based on knowledge of the prohibited use type. For example, the updated prompt word may be as follows.

[0089] Table 3 Examples of updated prompt words

[0090]

[0091] Using the example implementation of the present disclosure, when generating an answer, the machine learning model can use sentence 5 to prohibit the machine learning model from using knowledge about occupational distribution. At this time, the machine learning model can provide an unbiased answer "the secretary".

[0092] It should be understood that each modification mode described above has its own advantages. Modification mode I can support the machine learning model to take advantage of the fact that demography is unknowable, to potentially associate demographic information with the output through unregulated arrows 441 and 445. Modification mode II can adjust the dependencies of arrows 442, 443, 444, 436 and 437, so that there are no explicit constraints involving arrows 441 and 445. Further, modification mode III can prevent reference to demography-related text during the reasoning process. It should be understood that each modification mode has its own advantages, and the above modification modes can be used individually and / or in combination to more comprehensively address the problem of social bias in machine learning models.

[0093] According to an example implementation of the present disclosure, if it is determined that the potential bias does not exist, the first prompt word can be directly input into the machine learning model, and the answer of the machine learning model to the first prompt word can be received. In this way, the answer to the question can be obtained in a faster and more efficient manner while confirming that the machine learning model will not produce bias.

[0094] It should be understood that the present disclosure proposes a causal-guided and hint-based debiasing framework for machine learning models. In particular, the key role of selection mechanisms in data corpus bias modeling is proposed, and it is proposed to influence the machine learning model output by specifying different selection conditions on the internal representation of the machine learning model. Guided by the causal understanding of this interaction, the bias in the output of the machine learning model can be more effectively mitigated and / or eliminated. A large number of empirical results have demonstrated that the proposed technical solution can reduce the bias of the output of the machine learning model with only a small amount of time and computing resource overhead without adjusting the parameters of the machine learning model itself.

[0095] Example Process

[0096] FIG7 shows a flow chart of a method 700 for processing prompt words according to some implementations of the present disclosure. At box 710, a first prompt word is received, and the first prompt word expresses a question to be input to the machine learning model in natural language. At box 720, it is determined whether there is a potential deviation between the answer of the machine learning model to the first prompt word and the true answer to the question. At box 730, in response to determining that there is a potential deviation, the first prompt word is converted to a second prompt word based on keywords in the first prompt word that are associated with the potential deviation. At box 740, the answer of the machine learning model to the second prompt word is received.

[0097] According to an example implementation of the present disclosure, converting a first prompt word into a second prompt word includes: generating an additional prompt word based on a keyword associated with a potential deviation in the first prompt word; and combining the first prompt word and the additional prompt word to generate a second prompt word.

[0098] According to an example implementation of the present disclosure, generating additional prompt words includes: determining multiple entities corresponding to a keyword based on a first prompt word; replacing the keyword with the multiple entities respectively to generate multiple candidate hypotheses; and generating additional prompt words using the multiple candidate hypotheses.

[0099] According to an example implementation of the present disclosure, generating a second prompt word includes: creating a second prompt word to instruct the machine learning model to provide an answer to the question based on multiple candidate hypotheses.

[0100] According to an example implementation of the present disclosure, generating additional prompt words includes: determining a deviation cause associated with a keyword; and generating additional prompt words based on the deviation cause, the additional prompt words being used to suppress a machine learning model from activating a network node associated with the deviation cause.

[0101] According to an exemplary implementation of the present disclosure, generating additional prompt words based on the deviation cause includes: determining a plurality of candidate facts associated with the deviation cause; and generating additional prompt words based on the plurality of candidate facts.

[0102] According to an example implementation of the present disclosure, generating a second prompt word includes: creating a second prompt word to instruct the machine learning model to provide an answer to the question based on multiple candidate facts.

[0103] According to an exemplary implementation of the present disclosure, generating additional prompt words based on the deviation cause includes: determining a type of knowledge associated with the deviation cause; and generating additional prompt words based on the type of knowledge.

[0104] According to an example implementation of the present disclosure, generating the second prompt word includes: creating the second prompt word to instruct the machine learning model to provide an answer to the question based on knowledge of the prohibited use type.

[0105] According to an example implementation of the present disclosure, the method further includes: in response to determining that there is no potential bias, receiving an answer from the machine learning model to the first prompt word.

[0106] Example devices and equipment

[0107] Figure 8 A block diagram of an apparatus 800 for processing prompt words according to some implementations of the present disclosure is shown. The apparatus includes: a first receiving module 810, configured to receive a first prompt word, the first prompt word expressing a question to be input to a machine learning model in a natural language; a determining module 820, configured to determine whether there is a potential deviation between the answer of the machine learning model to the first prompt word and the true answer to the question; a conversion module 830, configured to, in response to determining that there is a potential deviation, convert the first prompt word into a second prompt word based on a keyword associated with the potential deviation in the first prompt word; and a second receiving module 840, configured to receive the answer of the machine learning model to the second prompt word.

[0108] According to an example implementation of the present disclosure, the conversion module includes: an additional generation module configured to generate additional prompt words based on keywords associated with potential deviations in the first prompt words; and a combination module configured to combine the first prompt words and the additional prompt words to generate a second prompt word.

[0109] According to an example implementation of the present disclosure, a generation module includes: an entity determination module, configured to determine multiple entities corresponding to a keyword based on a first prompt word; a replacement module, configured to replace the keyword with multiple entities respectively to generate multiple candidate hypotheses; and a first generation module, configured to generate additional prompt words using the multiple candidate hypotheses.

[0110] According to an example implementation of the present disclosure, the generation module includes: a first creation module configured to create a second prompt word to instruct the machine learning model to provide an answer to the question based on multiple candidate hypotheses.

[0111] According to an example implementation of the present disclosure, the additional generation module includes: a cause determination module, configured to determine the deviation cause associated with a keyword; and a suppression module, configured to generate additional prompt words based on the deviation cause, and the additional prompt words are used to suppress the machine learning model from activating network nodes associated with the deviation cause.

[0112] According to an example implementation of the present disclosure, the suppression module includes: a fact determination module configured to determine a plurality of candidate facts associated with a deviation cause; and a second generation module configured to generate additional prompt words based on the plurality of candidate facts.

[0113] According to an example implementation of the present disclosure, the generation module includes: a second creation module configured to create a second prompt word to instruct the machine learning model to provide an answer to the question based on multiple candidate facts.

[0114] According to an example implementation of the present disclosure, the additional generation module includes: a type determination module configured to determine the type of knowledge associated with the deviation cause; and a third generation module configured to generate additional prompt words based on the type of knowledge.

[0115] According to an example implementation of the present disclosure, the generation module includes: a third creation module configured to create a second prompt word to instruct the machine learning model to provide an answer to the question based on knowledge of the prohibited use type.

[0116] According to an example implementation of the present disclosure, the apparatus further includes: a third receiving module configured to receive an answer of the machine learning model to the first prompt word in response to determining that there is no potential bias.

[0117] Fig.10 1 is a block diagram of a device 1000 capable of implementing multiple implementations of the present disclosure. It should be understood that Fig.10 The computing device 1000 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the implementations described herein. Fig.10 The computing device 1000 shown may be used to implement the methods described above.

[0118] like Fig.10 As shown, computing device 1000 is in the form of a general-purpose computing device. Components of computing device 1000 may include, but are not limited to, one or more processors or processing units 1010, memory 1020, storage device 1020, one or more communication units 1040, one or more input devices 1050, and one or more output devices 1060. Processing unit 1010 may be an actual or virtual processor and is capable of performing various processes according to a program stored in memory 1020. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to increase the parallel processing capabilities of computing device 1000.

[0119] The computing device 1000 typically includes a plurality of computer storage media. Such media may be any available media accessible to the computing device 1000, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 1020 may be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 1020 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the computing device 1000.

[0120] The computing device 1000 may further include additional removable / non-removable, volatile / non-volatile storage media. Fig.10 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. Memory 1020 may include a computer program product 1025 having one or more program modules that are configured to perform various methods or actions of various implementations of the present disclosure.

[0121] The communication unit 1040 enables communication with other computing devices via a communication medium. Additionally, the functions of the components of the computing device 1000 can be implemented in a single computing cluster or multiple computing machines that can communicate via a communication connection. Therefore, the computing device 1000 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0122] Input device 1050 may be one or more input devices, such as a mouse, keyboard, tracking ball, etc. Output device 1060 may be one or more output devices, such as a display, a speaker, a printer, etc. Computing device 1000 may also communicate with one or more external devices (not shown) through communication unit 1040 as needed, such as storage devices, display devices, etc., communicate with one or more devices that allow a user to interact with computing device 1000, or communicate with any device that allows computing device 1000 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0123] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, the computer program product is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is provided, on which a computer program is stored, and when the program is executed by a processor, the method described above is implemented.

[0124] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0125] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0126] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0127] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0128] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A method for processing a prompt word, comprising: Receiving a first prompt word, wherein the first prompt word expresses a question to be input to a machine learning model in natural language; Determining whether there is a potential deviation between the answer of the machine learning model to the first prompt word and the true answer to the question; In response to determining that the potential deviation exists, converting the first prompt word into a second prompt word based on a keyword in the first prompt word associated with the potential deviation; as well as Receive an answer from the machine learning model to the second prompt word.

2. The method according to claim 1, wherein converting the first prompt word into the second prompt word comprises: generating additional prompt words based on the keywords in the first prompt words that are associated with the potential deviation; as well as The first prompt word and the additional prompt word are combined to generate the second prompt word.

3. The method according to claim 2, wherein generating the additional prompt word comprises: Determining a plurality of entities corresponding to the keyword based on the first prompt word; Replacing the keywords with the multiple entities respectively to generate multiple candidate hypotheses; as well as The additional prompt words are generated using the multiple candidate hypotheses.

4. The method according to claim 2, wherein generating the second prompt word comprises: The second prompt word is created to instruct the machine learning model to provide the answer to the question based on the plurality of candidate hypotheses.

5. The method according to claim 2, wherein generating the additional prompt word comprises: determining a cause of the deviation associated with the keyword; as well as Based on the cause of the deviation, the additional prompt word is generated, and the additional prompt word is used to inhibit the machine learning model from activating the network node associated with the cause of the deviation.

6. The method according to claim 5, wherein generating the additional prompt word based on the deviation cause comprises: determining a plurality of candidate facts associated with the cause of the deviation; as well as The additional prompt words are generated based on the plurality of candidate facts.

7. The method according to claim 6, wherein generating the second prompt word comprises: The second prompt word is created to instruct the machine learning model to provide the answer to the question based on the plurality of candidate facts.

8. The method according to claim 5, wherein generating the additional prompt word based on the deviation cause comprises: Identify the type of knowledge associated with the cause of the deviation; as well as The additional prompt words are generated based on the type of the knowledge.

9. The method according to claim 8, wherein generating the second prompt word comprises: The second prompt word is created to instruct the machine learning model to provide the answer to the question based on prohibiting the use of the type of knowledge.

10. The method according to claim 1, further comprising: In response to determining that the potential bias does not exist, receiving the answer of the machine learning model to the first prompt word.

11. A device for processing a prompt word, comprising: A first receiving module is configured to receive a first prompt word, wherein the first prompt word expresses a question to be input into the machine learning model in a natural language; A determination module configured to determine whether there is a potential deviation between the answer of the machine learning model to the first prompt word and the true answer to the question; a conversion module configured to, in response to determining that the potential deviation exists, convert the first prompt word into a second prompt word based on a keyword in the first prompt word associated with the potential deviation; as well as The second receiving module is configured to receive an answer from the machine learning model to the second prompt word.

12. An electronic device comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 10 when executed by the at least one processing unit.

13. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the processor is caused to implement the method according to any one of claims 1 to 10.

Images