Reducing Corpus-Based Bias Self-Diagnosis and Debiasing Methods and Systems in NLP

Abstract

The present invention discloses a method and system for reducing corpus-based bias self-diagnosis and debiasing in NLP. The method comprises the following steps: obtaining a textual description of the bias; performing self-diagnosis on the language model based on the language model's internal knowledge and the textual description of the bias; and performing self-debiasing on the language model based on a preset self-debiasing algorithm. The present invention demonstrates that language models, particularly large language models, have the ability to self-diagnose. Based on this ability, a simple method can be constructed to reduce the probability of the model outputting inappropriate language through model output, self-diagnosis, and removal of undesirable occurrences.

Description

Technical Field

[0001] The present invention relates to the field of language processing technology, and in particular to a method and system for reducing corpus-based bias self-diagnosis and debiasing in NLP. Background Art

[0002] When trained on large, unfiltered scrapes of the internet, language models pick up and reproduce various undesirable biases that may be present in the data. Since large models require millions of training examples to achieve good performance, it is difficult to completely prevent them from being imposed on this content.

[0003] Simple solutions like using banword lists don't alleviate this problem; first, they can't reliably prevent language models from generating biased text. It's easy to generate biased text by simply using words that are perfectly harmless in themselves, and since many of these words are important in the English vocabulary, they're required for meaningful text generation and shouldn't be on banword lists. Second, banwords also prevent the language model from acquiring topical knowledge related to the banned words, which might be necessary for some applications. Therefore, banning words is inherently difficult without compromising the model's capabilities.

[0004] Another solution is to carefully build and maintain training datasets, which is particularly effective for improving linguistic and cultural diversity in online and other forms of communication. However, for large language models that can be used for common global languages, additional mechanisms are needed to address bias, as datasets can be complex and cumbersome to organize and document given the amount of data required. Furthermore, building different training sets and training different models for each desired behavior can lead to high environmental impact.

[0005] Currently, no effective solutions have been proposed for the problems in related technologies. Summary of the Invention

[0006] In response to the problems in the related art, the present invention proposes a method and system for reducing corpus-based bias self-diagnosis and debiasing in NLP to overcome the above-mentioned technical problems existing in the existing related art.

[0007] To this end, the specific technical solutions adopted in the present invention are as follows:

[0008] According to one aspect of the present invention, a method and system for reducing corpus-based bias self-diagnosis and debiasing in NLP is provided, the method comprising the following steps:

[0009] S1. Obtain a textual description of the bias;

[0010] S2, self-diagnose the language model based on its internal knowledge and textual description of biases;

[0011] S3. Perform self-biasing on the language model based on a preset self-biasing algorithm.

[0012] Furthermore, the self-diagnosis of the language model based on the internal knowledge of the language model and the text description of the bias includes the following steps:

[0013] S21, define a language model M and word sequences w1, ...wk;

[0014] S22. Define the probability that the next word output by the language model is ω when the input is a word sequence w1, ...wk, expressed as p M (ω|ω1, ...ωk), the word with the highest probability is taken as the output of the language model M;

[0015] S23. Based on the sentence x and attribute description y generated by the language model M, construct the self-diagnosis input sdg(x,y).

[0016] S24. Calculate the probability that statement x exhibits attribute y using a preset probability calculation formula.

[0017] Furthermore, the probability calculation formula is:

[0018]

[0019] Where M represents the language model;

[0020] x represents the sentence generated by the language model;

[0021] y represents the attribute description;

[0022] sdg(x,y) represents the constructed autocorrelation input;

[0023] ω represents the two possible outputs of the language model M after receiving the self-diagnosis input sdg(x,y), including Yes and No;

[0024] p M (ω|sdg(x,y)) represents the probability that the language model M outputs ω when the input of the language model M is sdg(x,y);

[0025] p(x, y) represents the probability that statement x exhibits attribute y, and when p(x, y) is greater than a preset threshold σ, it means that statement x exhibits attribute y.

[0026] Furthermore, the self-debiasing of the language model based on the preset self-debiasing algorithm includes the following steps:

[0027] S31. Based on the preset self-debiasing algorithm, construct the self-debiasing input sdb(x,y) corresponding to the input x;

[0028] S32: Input the original input x and the self-debiased input sdb(x,y) into the language model M. The language model M calculates the probability distribution of the output marked words and obtains p M (ω|x) and p M (ω|sdb(x,y));

[0029] S33, the probability distribution p corresponding to the original input x M The probability distribution p of (ω|x) and the self-debiased input sdb(x,y) M (ω|sdb(x,y)) is input into the algorithm and a new probability distribution is obtained through calculation;

[0030] S34, based on the calculation to obtain a new probability distribution p M (ω|x), by comparing the corresponding probabilities of the labeled words, output the labeled word with the maximum probability.

[0031] Furthermore, the calculation formula of the new probability distribution is:

[0032] p M (ω|x)∝α(Δ(ω,x,y))·p M (ω|x)

[0033] Where M represents the language model;

[0034] x represents the original input, that is, the sentence or word passed into the language model M;

[0035] y represents the text description of the attribute;

[0036] ω represents the possible output of model M after receiving input;

[0037] p M (ω|x) represents the probability distribution corresponding to the original input x;

[0038] p M (ω|sdb(x,y)) represents the probability distribution corresponding to the self-debiased input sdb(x,y);

[0039] Δ(ω, x, y) represents p M (ω|x) and p M The difference of (ω|sdb(x,y));

[0040] p M (ω|x) represents the new probability distribution;

[0041] α: R→[0, 1] represents a scaling function, which ranges between [0, 1] and is used to change the probability of the bias word based on the difference Δ(ω, x, y), and R represents the real number domain.

[0042] According to one aspect of the present invention, a corpus-based self-diagnosis and debiasing system for reducing bias in NLP is provided, the system comprising a text acquisition module, a self-diagnosis module, and a self-debiasing module;

[0043] The text acquisition module is used to obtain a text description of the bias;

[0044] The self-diagnosis module is used to perform self-diagnosis on the language model based on the internal knowledge of the language model and the text description of the bias;

[0045] The self-debiasing module is used to perform self-debiasing on the language model based on a preset self-debiasing algorithm.

[0046] Furthermore, the self-diagnosis of the language model based on the internal knowledge of the language model and the text description of the bias includes the following steps:

[0047] Define the language model M and word sequence w1, ...wk;

[0048] Defined as the input word sequence w1, ...wk, the probability that the language model outputs the next word is ω is expressed as p M (ω|ω1, ...ωk), the word with the highest probability is taken as the output of the language model M;

[0049] Based on the sentence x and attribute description y generated by the language model M, the self-diagnosis input sdg(x, y) is constructed.

[0050] The probability that statement x exhibits attribute y is calculated using a preset probability calculation formula.

[0051] Furthermore, the probability calculation formula is:

[0052]

[0053] Where M represents the language model;

[0054] x represents the sentence generated by the language model;

[0055] y represents the attribute description;

[0056] sdg(x, y) represents the constructed autocorrelation input;

[0057] ω represents the two possible outputs of the language model M after receiving the self-diagnosis input sdg(x, y), including Yes and No;

[0058] p M (ω|sdg(x, y)) represents the probability that the language model M outputs ω when the language model M input is sdg(x, y);

[0059] p(x, y) represents the probability that statement x exhibits attribute y, and when p(x, y) is greater than a preset threshold σ, it means that statement x exhibits attribute y.

[0060] Furthermore, the self-debiasing of the language model based on the preset self-debiasing algorithm includes the following steps:

[0061] Based on the preset self-debiasing algorithm, construct the self-debiasing input sdb(x, y) corresponding to the input x;

[0062] The original input x and the self-debiased input sdb(x, y) are input into the language model M, which calculates the probability distribution of the output labeled words and obtains p M (ω|x) and p M (ω|sdb(x,y));

[0063] The probability distribution p corresponding to the original input x M (ω|x) and the probability distribution p of the self-debiased input sdb(x, y) M (ω|sdb(x, y)) is input into the algorithm and a new probability distribution is obtained by calculation;

[0064] Based on the calculation, the new probability distribution p M (ω|x), by comparing the corresponding probabilities of the labeled words, output the labeled word with the maximum probability.

[0065] Furthermore, the calculation formula of the new probability distribution is:

[0066] p M (ω|x)∝α(Δ(ω,x,y))·p M (ω|x)

[0067] Where M represents the language model;

[0068] x represents the original input, that is, the sentence or word passed into the language model M;

[0069] y represents the text description of the attribute;

[0070] ω represents the possible output of model M after receiving input;

[0071] p M (ω|x) represents the probability distribution corresponding to the original input x;

[0072] p M(ω|sdb(x, y)) represents the probability distribution corresponding to the self-debiased input sdb(x, y);

[0073] Δ(ω, x, y) represents p M (ω|x) and p M The difference of (ω|sdb(x, y));

[0074] p M (ω|x) represents the new probability distribution;

[0075] α: R→[0, 1] represents a scaling function, which ranges between [0, 1] and is used to change the probability of the bias word based on the difference Δ(ω, x, y), and R represents the real number domain.

[0076] The beneficial effects of the present invention are:

[0077] 1. This invention demonstrates that language models, especially large language models, have the ability to self-diagnose. Based on this ability, a simple method can be constructed to reduce the probability of the model outputting inappropriate language through model output, self-diagnosis, and deletion of unexpected situations.

[0078] 2. The present invention constructs a self-debiasing algorithm by using self-debiasing input and scaling function to reduce the probability (score) of the model outputting inappropriate words, thereby reducing the output of inappropriate words.

[0079] 3. The present invention introduces a hyperparameter attenuation constant λ in the scaling function to avoid forcing the probability of certain words to zero in certain circumstances, making it impossible to evaluate the quality of the language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0081] Figure 1 4 is a flowchart of a method for reducing corpus-based bias self-diagnosis and debiasing in NLP according to an embodiment of the present invention. DETAILED DESCRIPTION

[0082] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention. They are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. By referring to these contents, ordinary technicians in this field should be able to understand other possible implementation methods and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0083] According to an embodiment of the present invention, a method and system for reducing corpus-based bias self-diagnosis and debiasing in NLP are provided.

[0084] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to one embodiment of the present invention, a method for reducing corpus-based bias self-diagnosis and debiasing in NLP is provided, and the method includes the following steps:

[0085] S1. Obtain a textual description of the bias;

[0086] S2, self-diagnose the language model based on its internal knowledge and textual description of biases;

[0087] The self-diagnosis of the language model based on the internal knowledge of the language model and the text description of the bias includes the following steps:

[0088] S21, define a language model M and word sequences w1, ...wk;

[0089] S22. Define the probability that the next word output by the language model is ω when the input is a word sequence w1, ...wk, expressed as p M (ω|ω1, ...ωk), the word with the highest probability is taken as the output of the language model M;

[0090] S23. Based on the sentence x and attribute description y generated by the language model M, construct the self-diagnosis input sdg(x,y).

[0091] Specifically, the specific construction method of the self-diagnosis input sdg(x,y) is to ask the language model M a question for the sentence x generated by the language model M, and require the language model M to use the internal knowledge of the model to answer whether the input sentence x contains attribute y.

[0092] Specifically, the constructed self-diagnosis input sdg(x,y) can also be expressed in the form of Table 1:

[0093] Table 1

[0094] ″x″ Question: Does the above text contain attribute y? Answer: Yes or No

[0095] S24. Calculate the probability that statement x exhibits attribute y using a preset probability calculation formula; wherein the probability calculation formula is:

[0096]

[0097] Where M represents the language model;

[0098] x represents the sentence generated by the language model;

[0099] y represents the attribute description;

[0100] sdg(x, y) represents the constructed autocorrelation input;

[0101] ω represents the two possible outputs of the language model M after receiving the self-diagnosis input sdg(x, y), including Yes and No;

[0102] p M (ω|sdg(x, y)) represents the probability that the language model M outputs ω when the language model M input is sdg(x, y);

[0103] p(x, y) represents the probability that statement x exhibits attribute y, and when p(x, y) is greater than a preset threshold σ, it means that statement x exhibits attribute y.

[0104] S3. Perform self-debiasing on the language model based on a preset self-debiasing algorithm;

[0105] Specifically, language models have been shown to have self-diagnostic capabilities, meaning that the model can use internal knowledge to determine whether its own output contains undesirable attributes; by generating a set of sentences in a regular way and then performing self-diagnosis, discarding all those sentences that exhibit undesirable biases, the probability of poor model output can be reduced; however, this approach is inefficient and does not provide a viable alternative, so a more efficient algorithm was designed to leverage the internal knowledge of the language model to reduce undesirable behavior.

[0106] Define the self-debiasing model as a language model that uses only its internal knowledge to reduce the probability of generating biased text; assume that M is a pre-trained language model, and y is a text description of an attribute. Let x be an input text, and we want M to generate a continuation (i.e., output) for it. By requiring the language model M to generate a continuation containing the attribute y based on the local domain of the input text x, we construct the self-debiasing input sdb(x, y); similar to self-diagnosis, calculate the distribution p of the next word output by the model given the original input x M (w|x), and the next word distribution p output by the model when using self-debiased input sdb(x, y) M(ω|sdb(x, y)); the construction of the self-debiasing input sdb(x, y) is shown in Table 2:

[0107] Table 2

[0108] The following text contains the y attribute x ？

[0109] The step of performing bias removal on the language model based on a preset bias removal algorithm includes the following steps:

[0110] S31. Based on the preset self-debiasing algorithm, construct the self-debiasing input sdb(x, y) corresponding to the input x; S32. Input the original input x and the self-debiasing input sdb(x, y) into the language model M. The language model M calculates the probability distribution of the output marked words and obtains p M (ω|x) and p M (ω|sdb(x,y));

[0111] Specifically, when a sentence x is input into the language model M, the language model M will automatically generate its possible continuation words and their probabilities. When the input is a word sequence w1, ...wk, the probability that the next word output by the language model M is ω is expressed as p M (ω|ω1, ...ωk), the language model M selects the word with the highest probability as the output.

[0112] S33, the probability distribution p corresponding to the original input x M (ω|x) and the probability distribution p of the self-debiased input sdb(x, y) M (ω|sdb(x, y)) is input into the algorithm and a new probability distribution is obtained by calculation;

[0113] Specifically, according to the construction of the self-debiasing input sdb(x, y), the self-debiasing input will encourage the language model to produce undesirable text; therefore, undesirable words will be assigned a higher probability, which is the difference between the two distributions:

[0114] Δ(ω, x, y) = p M (ω|x)-p M (ω|sdb(x,y))

[0115] will be less than zero for unpopular words;

[0116] Using this fact, a new probability distribution is obtained.

[0117] The calculation formula of the new probability distribution is:

[0118] p M (ω|x)∝α(Δ(ω,x,y))·p M(ω|x)

[0119] Where M represents the language model;

[0120] x represents the original input, that is, the sentence or word passed into the language model M;

[0121] y represents the text description of the attribute;

[0122] ω represents the possible output of model M after receiving input;

[0123] p M (ω|x) represents the probability distribution corresponding to the original input x;

[0124] p M (ω|sdb(x, y)) represents the probability distribution corresponding to the self-debiased input sdb(x, y);

[0125] Δ(ω, x, y) represents p M (ω|x) and p M The difference of (ω|sdb(x, y));

[0126] p M (ω|x) represents the new probability distribution;

[0127] α: R→[0, 1] represents a scaling function, which ranges from [0, 1] and is used to change the probability of bias words based on the difference Δ(ω, x, y), where R represents the real number domain;

[0128] If the scaling function is used to force the probability of certain words to zero, it will be difficult to ensure the quality of the language model output. This is because as long as the model makes a wrong judgment on the "correct output", it may lead to incoherent sentences, confusion of parts of speech, or even more serious problems. Therefore, instead of forcing the probability of biased words to zero, a variant is adopted, that is, reducing their probability according to the size of the difference Δ(ω, x, y):

[0129]

[0130] Here, the decay constant λ is a hyperparameter of our proposed algorithm.

[0131] With only a slight modification, the self-debiasing algorithm can also be used to perform self-debiasing on multiple attributes simultaneously given a set of descriptions Y = {y1, ...yn}; to do this, we simply replace Δ(ω, x, y) with

[0132] Where x represents the original input, that is, the sentence or word passed into the language model M;

[0133] y represents the text description of the attribute, and Y is a set of attribute text descriptions y;

[0134] ω represents the possible output of the language model M after receiving the input;

[0135] p M (ω|x) represents the probability distribution corresponding to the original input x;

[0136] p M (ω|sdb(x, y)) represents the probability distribution corresponding to the self-debiased input sdb(x, y), which is p M (ω|x) and p M The difference of (ω|sdb(x, y)) is the minimum difference in a set of attribute descriptions y.

[0137] Therefore, if there is a higher probability of using word ω as a continuation of x based on at least one self-debiased input, it will be penalized; when the next output word of the model is a bad word ω, Δ(ω, x, y) < 0, and the corresponding weight is reduced through the reduction function to achieve the purpose of penalizing and reducing the probability of bad word output.

[0138] S34, based on the calculation to obtain a new probability distribution p M (ω|x), by comparing the corresponding probabilities of the labeled words, output the labeled word with the maximum probability.

[0139] Specifically, the present invention utilizes the language model's ability to self-diagnose and self-debias to reduce the probability of the model generating biased text in the decoding algorithm. First, we use the language model's internal knowledge and text description of the bias to detect whether the language model's own output exhibits undesirable attributes (bias), which is the language model's self-diagnosis. Then, we use this knowledge advantage of the language model's self-diagnosis to discard undesirable behaviors in a completely unsupervised manner, which is the language model's self-debiasing.

[0140] According to another embodiment of the present invention, a corpus-based self-diagnosis and debiasing system for reducing bias in NLP is provided, the system comprising a text acquisition module, a self-diagnosis module, and a self-debiasing module;

[0141] The text acquisition module is used to obtain a text description of the bias;

[0142] The self-diagnosis of the language model based on the internal knowledge of the language model and the text description of the bias includes the following steps:

[0143] Define the language model M and word sequence w1, ...wk;

[0144] Defined as the input word sequence w1, ...wk, the probability that the language model outputs the next word is ω is expressed as p M(ω|ω1, ...ωk), the word with the highest probability is taken as the output of the language model M;

[0145] Based on the sentence x and attribute description y generated by the language model M, the self-diagnosis input sdg(x, y) is constructed.

[0146] The probability that statement x exhibits attribute y is calculated using a preset probability calculation formula.

[0147] Wherein, the probability calculation formula is:

[0148]

[0149] Where M represents the language model;

[0150] x represents the sentence generated by the language model;

[0151] y represents the attribute description;

[0152] sdg(x, y) represents the constructed autocorrelation input;

[0153] ω represents the two possible outputs of the language model M after receiving the self-diagnosis input sdg(x, y), including Yes and No;

[0154] p M (ω|sdg(x, y)) represents the probability that the language model M outputs ω when the language model M input is sdg(x, y);

[0155] p(x, y) represents the probability that statement x exhibits attribute y, and when p(x, y) is greater than the preset threshold σ, it means that statement x exhibits attribute y;

[0156] The self-diagnosis module is used to perform self-diagnosis on the language model based on the internal knowledge of the language model and the text description of the bias;

[0157] The self-debiasing module is used to perform self-debiasing on the language model based on a preset self-debiasing algorithm;

[0158] The self-debiasing of the language model based on the preset self-debiasing algorithm includes the following steps:

[0159] Based on the preset self-debiasing algorithm, construct the self-debiasing input sdb(x, y) corresponding to the input x;

[0160] The original input x and the self-debiased input sdb(x, y) are input into the language model M, which calculates the probability distribution of the output labeled words and obtains p M (ω|x) and p M (ω|sdb(x,y));

[0161] The probability distribution p corresponding to the original input x M (ω|x) and the probability distribution p of the self-debiased input sdb(x, y) M (ω|sdb(x, y)) is input into the algorithm and a new probability distribution is obtained by calculation;

[0162] The calculation formula of the new probability distribution is:

[0163] p M (ω|x)∝α(Δ(ω,x,y))·p M (ω|x)

[0164] Where M represents the language model;

[0165] x represents the original input, that is, the sentence or word passed into the language model M;

[0166] y represents the text description of the attribute;

[0167] ω represents the possible output of model M after receiving input;

[0168] p M (ω|x) represents the probability distribution corresponding to the original input x;

[0169] p M (ω|sdb(x, y)) represents the probability distribution corresponding to the self-debiased input sdb(x, y);

[0170] Δ(ω, x, y) represents p M (ω|x) and p M The difference of (ω|sdb(x, y));

[0171] p M (ω|x) represents the new probability distribution;

[0172] α: R→[0, 1] represents a scaling function, which ranges between [0, 1] and is used to change the probability of the bias word based on the difference Δ(ω, x, y), and R represents the real number domain.

[0173] Based on the calculation, the new probability distribution p M (ω|x), by comparing the corresponding probabilities of the labeled words, output the labeled word with the maximum probability.

[0174] In summary, with the help of the above technical solutions of the present invention, the present invention can prove that language models, especially large language models, have the ability of self-diagnosis. Based on this ability, a simple method can be constructed to reduce the probability of the model outputting inappropriate language through model output, self-diagnosis and deletion of unexpected situations; the present invention constructs a self-debiasing algorithm by using self-debiasing input and scaling function to reduce the probability size (score) of the model outputting inappropriate words, thereby reducing the output of inappropriate words; the present invention introduces a hyperparameter attenuation constant λ in the scaling function to avoid forcing the probability of certain words to be zero in specific circumstances, making it impossible to evaluate the quality of the language model.

[0175] In the present invention, unless otherwise clearly stipulated and limited, the terms "installation", "setting", "connection", "fixation", "screw-on" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integrated connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, ordinary technicians in this field can understand the specific meanings of the above terms in the present invention according to the specific circumstances.

[0176] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. Reduce corpus-based bias self-diagnosis and debiasing methods in NLP, characterized by: The method comprises the following steps: S1. Obtain a textual description of the bias; S2, self-diagnose the language model based on its internal knowledge and textual description of biases; S3. Perform self-debiasing on the language model based on a preset self-debiasing algorithm; The self-diagnosis of the language model based on the internal knowledge of the language model and the text description of the bias includes the following steps: S21, define a language model M and word sequences w1, ...wk; S22. Define the probability that the next word output by the language model is ω when the input is a word sequence w1, ...wk, expressed as p M (ω|ω1, ...ωk), the word with the highest probability is taken as the output of the language model M; S23, constructing the self-diagnosis input sdg(x,y) based on the sentence x and attribute description y generated by the language model M; The self-diagnosis input sdg(x,y) is constructed by asking the language model M a question based on the sentence x generated by the language model M, requiring the language model M to use its internal knowledge to answer whether the input sentence x contains attribute y. S24. Calculate the probability that statement x exhibits attribute y using a preset probability calculation formula; The self-debiasing of the language model based on the preset self-debiasing algorithm includes the following steps: S31. Based on the preset self-debiasing algorithm, construct the self-debiasing input sdb(x,y) corresponding to the input x; S32: Input the original input x and the self-debiased input sdb(x,y) into the language model M. The language model M calculates the probability distribution of the output marked words and obtains p M (ω|x) and p M (ω|sdb(x,y)); S33, the probability distribution p corresponding to the original input x M The probability distribution p of (ω|x) and the self-debiased input sdb(x,y) M (ω|sdb(x,y)) is input into the algorithm and a new probability distribution is obtained through calculation; S34, based on the calculation to obtain a new probability distribution p M (ω|x), by comparing the corresponding probabilities of the marked words, output the marked word with the maximum probability; Among them, the self-debiased input sdb(x,y) is constructed by requiring the language model M to generate a continuation containing attribute y based on the input text local domain x.

2. The method for reducing corpus-based bias self-diagnosis and debiasing in NLP according to claim 1, characterized in that: The probability calculation formula is: Among them, M represents the language model; x represents the sentence generated by the language model; y represents the attribute description; sdg(x,y) represents the constructed autocorrelation input; ω represents the two possible outputs of the language model M after receiving the self-diagnosis input sdg(x,y), including Yes and No; p M (ω|sdg(x,y)) represents the probability that the language model M outputs ω when the input of the language model M is sdg(x,y); p(x, y) represents the probability that statement x exhibits attribute y, and when p(x, y) is greater than a preset threshold σ, it means that statement x exhibits attribute y.

3. The method for reducing corpus-based bias self-diagnosis and debiasing in NLP according to claim 2, characterized in that: The calculation formula of the new probability distribution is: p M (ω|x)∝α(Δ(ω,x,y))·p M (ω|x) Where M represents the language model; x represents the original input; y represents the text description of the attribute; ω represents the possible output of model M after receiving input; p M (ω|x) represents the probability distribution corresponding to the original input x; p M (ω|sdb(x,y)) represents the probability distribution corresponding to the self-debiased input sdb(x,y); Δ(ω, x, y) represents p M (ω|x) and p M The difference of (ω|sdb(x,y)); p M (ω|x) represents the new probability distribution; α: R→[0, 1] represents a scaling function, which ranges between [0, 1] and is used to change the probability of the bias word based on the difference Δ(ω, x, y), and R represents the real number domain.

4. A system for reducing corpus-based bias self-diagnosis and debiasing in NLP, used in the operating steps of the method for reducing corpus-based bias self-diagnosis and debiasing in NLP according to any one of claims 1 to 3, characterized in that: The system includes a text acquisition module, a self-diagnosis module and a self-debiasing module; The text acquisition module is used to obtain a text description of the bias; The self-diagnosis module is used to perform self-diagnosis on the language model based on the internal knowledge of the language model and the text description of the bias; The self-debiasing module is used to perform self-debiasing on the language model based on a preset self-debiasing algorithm; The self-diagnosis of the language model based on the internal knowledge of the language model and the text description of the bias includes the following steps: Define the language model M and word sequence w1, ...wk; Defined as the input word sequence w1, ...wk, the probability that the language model outputs the next word is ω is expressed as p M (ω|ω1, ...ωk), take the word with the highest probability as the output of the language model M; construct the self-diagnosis input sdg(x, y) based on the sentence x and attribute description y generated by the language model M; The self-diagnosis input sdg(x, y) is constructed by asking the language model M a question based on the sentence x generated by the language model M, requiring the language model M to use its internal knowledge to answer whether the input sentence x contains attribute y. Calculate the probability that statement x exhibits attribute y using a preset probability calculation formula; The self-debiasing of the language model based on the preset self-debiasing algorithm includes the following steps: Based on the preset self-debiasing algorithm, construct the self-debiasing input sdb(x, y) corresponding to the input x; The original input x and the self-debiased input sdb(x, y) are input into the language model M, which calculates the probability distribution of the output labeled words and obtains p M (ω|x) and p M (ω|sdb(x,y)); The probability distribution p corresponding to the original input x M (ω|x) and the probability distribution p of the self-debiased input sdb(x, y) M (ω|sdb(x, y)) is input into the algorithm and a new probability distribution is obtained by calculation; Based on the calculation, the new probability distribution p M (ω|x), by comparing the corresponding probabilities of the marked words, output the marked word with the maximum probability; Among them, the self-debiased input sdb(x,y) is constructed by requiring the language model M to generate a continuation containing attribute y based on the input text local domain x.

5. The corpus-based bias self-diagnosis and debiasing system for reducing NLP according to claim 4, characterized in that: The probability calculation formula is: Where M represents the language model; x represents the sentence generated by the language model; y represents the attribute description; sdg(x,y) represents the constructed autocorrelation input; ω represents the two possible outputs of the language model M after receiving the self-diagnosis input sdg(x,y), including Yes and No; p M (ω|sdg(x,y)) represents the probability that the language model M outputs ω when the input of the language model M is sdg(x,y); p(x, y) represents the probability that statement x exhibits attribute y, and when p(x, y) is greater than a preset threshold σ, it means that statement x exhibits attribute y.

6. The corpus-based bias self-diagnosis and debiasing system for reducing bias in NLP according to claim 5, characterized in that: The calculation formula of the new probability distribution is: p M (ω|x)∝α(Δ(ω,x,y))·p M (ω|x) Where M represents the language model; x represents the original input; y represents the text description of the attribute; ω represents the possible output of model M after receiving input; p M (ω|x) represents the probability distribution corresponding to the original input x; p M (ω|sdb(x, y)) represents the probability distribution corresponding to the self-debiased input sdb(x, y); Δ(ω, x, y) represents p M (ω|x) and p M The difference of (ω|sdb(x, y)); p M (ω|x) represents the new probability distribution; α: R→[0, 1] represents a scaling function, which ranges between [0, 1] and is used to change the probability of the bias word based on the difference Δ(ω, x, y), and R represents the real number domain.

Images