Interpretable debiasing of vectorized language representations with iterative orthogonalization

Abstract

A computer-implemented method for debiasing a vectorized language representation can include identifying two (or more) pairs of concepts that need to be debiased, calculating an average vector for each concept, determining a center point for a rotation operation for orthogonalization based on the average vectors, and debiasing the vectorized language representation based on the center point. And shifting the vectors to the center point prior to performing a correction operation, which can be a hierarchical rotation, after which the vectors can be moved back from the center point. The process can be iteratively performed if desired.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 354,554, filed on June 22, 2022, the disclosure of which is incorporated herein by reference. Background Art

[0003] The present disclosure relates generally to natural language processing systems and methods, and particularly to debiasing vectorized language representations.

[0004] "Natural language processing" generally refers to a computer-implemented technology for interacting with a user using a human language with a natural vocabulary and grammar, which is relative to an artificial language or a prescribed command set that is traditionally used to interact with a computer. Typically, natural language processing is implemented by training a neural network or other machine learning model using a training corpus of documents, which may include dictionaries, news reports, textbooks, works of fiction, and / or other documents. Language models attempt to reflect the relationship between words, such as synonyms or associations. Language models typically rely on vectorized representations, in which words are represented as vectors in a high-dimensional space (e.g., thousands of dimensions). Vector components are learned via a training process, and the result is that the relationship between words (e.g., similarity in meaning) is encoded in the vector components. Such vectorized language representations (sometimes also referred to as "word embeddings") have been shown to be useful tools in enabling computers to interpret natural language input and / or generate natural language output.

[0005] Unfortunately, the training process may result in word embeddings that reflect biases present in the training corpus. As used herein, "bias" refers to any unwanted association between terms. For example, "doctor" may be associated with "man" and "nurse" with "woman" due to prevalent gender stereotypes reflected in the training corpus. Therefore, it is desirable to "de-bias" the representations of various words, such as by modifying the vectors learned during the training process to remove unwanted associations. Summary of the invention

[0006] Certain embodiments of the present invention relate to systems and methods for debiasing vectorized language representations. These systems and methods are applicable to multiple vectorized language representations. In some embodiments, the debiasing method may include identifying two pairs (or more pairs) of contrasting concepts that need debiasing, calculating the subspace direction for each concept, determining the center point for the correction operation to orthogonalize the subspace direction, and centering the word vector on the center point before performing the correction operation (which may be a hierarchical rotation), after which the word vector may be recentered (or moved back from the center point). In some embodiments, the process may be performed iteratively.

[0007] Some embodiments relate to a computer-implemented method, comprising: obtaining a vectorized language representation for a plurality of terms, wherein the vectorized language representation comprises a plurality of vectors in a vector space such that each term has an associated vector; identifying two pairs of concepts to be debiased; obtaining a representative word list for each concept in each pair of concepts; for each concept, calculating a corresponding concept average from vectors associated with terms in the representative word list for the concept; calculating a center point for the corresponding concept average; calculating a corresponding subspace direction for each pair of concept averages; and for one or more of the plurality of vectors in the vector space, calculating a debiasing vector, wherein calculating the debiasing vector comprises: realigning the vector on the center point; performing a correction operation on the vector with respect to the corresponding subspace direction; and unrealigning the vector. The debiasing vector can be added to the vectorized language representation to replace or augment the original vector as desired.

[0008] Some embodiments relate to a computer system having a memory for storing a vectorized language representation and a processor coupled to the memory. The processor can be configured to: identify two pairs of concepts to be debiased; obtain a representative word list for each concept in each pair of concepts; for each concept, calculate a corresponding concept average from vectors associated with words in the representative word list for the concept; calculate a center point for the corresponding concept average; calculate a corresponding subspace direction for each pair of concept averages; and for one or more vectors in a plurality of vectors in a vector space, re-center the vector on the center point; perform a correction operation on the vector with respect to the corresponding subspace direction; and un-re-center the vector.

[0009] Some embodiments relate to a computer-readable storage medium having program code instructions stored therein, which, when executed by a processor in a computer system, cause the computer system to perform a method, the method comprising: obtaining a vectorized linguistic representation for a plurality of words, wherein the vectorized linguistic representation comprises a plurality of vectors in a vector space such that each word has an associated vector; identifying two pairs of concepts to be debiased; generating a representative word list for each concept in each pair of target concepts; for each concept, calculating a corresponding concept average from vectors associated with words in the representative word list for the concept; calculating a center point for the corresponding concept average; calculating a corresponding subspace direction for each pair of concept averages; centering each vector in the vectorized linguistic representation on the center point; performing a first correction of each vector relative to the first two subspace directions in the subspace directions; projecting a third subspace direction in the subspace directions onto the span of the first two subspace directions; performing a second correction of each vector relative to the third subspace direction and the projection; and uncentering each vector. In some embodiments, the first two subspace directions in the subspace directions may be a pair of subspace directions that are most nearly orthogonal.

[0010] In these and other embodiments, the following actions are performed until a stopping criterion is met: calculating the corresponding concept average; calculating the center point; calculating the corresponding subspace direction; realigning the vector; performing a correction operation; and not realigning the vector. The stopping criterion may include, for example, a convergence criterion based on a change in a performance metric and / or a fixed number of iterations.

[0011] In these and other embodiments, performing the correction operation on the vector may include: determining a rotation angle to be applied to the vector; and rotating the vector by the rotation angle. The rotation angle may be based at least in part on a relative similarity of the vector to a corresponding subspace direction for each pair of concept averages, or based at least in part on an angle between the vector and one of the subspace directions.

[0012] In these and other embodiments, the vectorized language representation may be obtained from a variety of sources including structured text and / or unstructured text.

[0013] In these and other embodiments, the representative word list can be generated in whole or in part by a human. In some embodiments, the processor can be further configured such that obtaining a representative word list for each concept in each pair of concepts includes: for at least one of the concepts: receiving an initial word list generated by a human; determining a word vector average corresponding to the words in the initial word list; and selecting words for the representative word list based on vector similarity to the word vector average.

[0014] The following detailed description, together with the accompanying drawings, will provide a better understanding of the nature and advantages of the claimed invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figures 1A to 1D The OSCaR method for debiasing using a simplified two-dimensional representation of word vectors is described.

[0016] Figure 2 A flowchart of a process for iterative subspace modification is shown according to some embodiments.

[0017] Figure 3 A flowchart of a process for iterative subspace modification is shown according to some embodiments.

[0018] Figure 4 A table summarizing WEAT scores for a number of different debiasing processes, including processes according to specific embodiments, is shown.

[0019] Figure 5 A table summarizing the WEAT and dot-integral scores at different iterations for a conventional debiasing process and a process according to a specific embodiment is shown.

[0020] Figure 6 A table summarizing WEAT scores for a number of different debiasing processes applied to different concept pairs, including processes according to specific embodiments, is shown.

[0021] Figure 7 and Figure 8 A table summarizing the results of a cross-validation study for a number of different debiasing processes (including processes according to specific embodiments) is shown. WEAT scores obtained for different debiasing processes applied to different concept pairs are shown; Figure 7 In , a test / train split of the word list is used, while in Figure 8 In , all words are used for training and testing.

[0022] Fig. 9 A table summarizing SWEAT scores for various concept pairs and various debiasing processes, including processes according to specific embodiments, is shown.

[0023] Fig.10 A table summarizing the WEAT scores and dot-product scores for an iterative debiasing process according to a particular embodiment performed on three concept pairs is shown.

[0024] Fig.11 A table summarizing the SWEAT scores for each concept pair at each iteration of an iterative debiasing process according to a particular embodiment performed on three concept pairs is shown. DETAILED DESCRIPTION

[0025] The following description of exemplary embodiments of the present invention is presented for the purpose of illustration and description. The following description is not intended to be exhaustive or to limit the claimed invention to the precise form described, and those of ordinary skill in the art will appreciate that many modifications and variations are possible. These embodiments are selected and described in order to best explain the principles of the invention and its practical application, thereby enabling others skilled in the art to best implement and use the invention in various embodiments and with various modifications to suit the particular use contemplated.

[0026] Certain embodiments of the present invention relate to systems and methods for debiasing vectorized language representations. Vectorized language representations can include contextualized embeddings (such as known embedding processes ELMO, BERT, or RoBERTA) based on natural language data, and decontextualized embeddings (such as known embedding processes Word2Vec, GloVe, or FastText) based on structured data. Vectorized language representations or embeddings map each word to a vector in a high-dimensional space. In some representations, the (cosine) similarity between words (vectors) captures the similarity of meaning based on the similarity in the context of using specific words. The technology described herein can be applied to a variety of vectorized language representations, including embeddings based on structured data.

[0027] As used herein, the term "bias" refers to unwanted associations between words that can be reflected in word embeddings or vectorized representations. This bias can be reflected in the cosine similarity between vectors, which would be unrelated (or orthogonal) in the absence of bias. An example is gender bias. In an unbiased language representation, words that represent work (such as "doctor," "nurse," "programmer," or "teacher") are not associated with words that identify or imply a specific gender (such as "man," "father," "king," or "woman," "mother," "queen." However, if a corpus including such correlations is used to train a language model, the resulting language model can reflect these correlations. As used herein, "debiasing" refers to (post-training) operations on vectorized language representations that remove correlations that may have been learned during training.

[0028] A variety of techniques have been developed to debias the language model by modifying the vectors for certain words to eliminate unwanted correlations. The modification is usually performed between linear subspaces. Some debiasing techniques rely on, for example, using principal component analysis to project into subspaces. Examples include: linear projection (LP) (described in S. Dev et al. in AISTATS "Attenuating bias in word vectors", Proceedings of Machine Learning Research, pp. 879 to 887, April 16 to 18, 2019); hard debiasing (HD) (described in T. Bolukbasi et al. "Man is to computer programmer as woman is to homemaker? debiasing word embeddings", Advances in Neural Information Processing Systems 29 (2016)) and iterative null space projection (INLP) (described in S. Ravfogel et al. "Null it out: Guarding protected attributes by iterative nullspace projection", 2020). A recently developed technique is called orthogonal subspace association and rectification (OSCaR) (described in S. Dev et al., "Oscar: Orthogonal Subspace correction and rectification of biases in word embeddings," in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 5034-5050, November 7-11, 2021). OSCaR involves identifying two subspaces (e.g., a male-female gender subspace and a work subspace) and performing a continuous deformation of the embeddings across the span of the two subspaces that orthogonalizes the two subspaces (and the concepts they represent).

[0029] Figures 1A to 1D The OSCaR method using a simplified two-dimensional representation of word vectors is described. In step 1 ( Figure 1A), two concept subspaces are identified for which orthogonality is desired. The subspaces can be identified by manually crafting lists of appropriate representative words of each type. These lists need not be exhaustive and may include, for example, on the order of a dozen to a hundred words. In this example, one subspace, represented by vector 102, is defined by considering pairs of words that are similar except for gender, such as "man" / "woman", "boy" / "girl", "he" / "she", "uncle" / "aunt", etc. Another subspace, represented by vector 104, includes words that identify jobs, such as "doctor", "engineer", "nurse", "maid", etc.; these are words that represent jobs that people can do regardless of gender. A subspace vector can be calculated for each subspace by subtracting the vectors for pairs of words and determining the average of the differences, or by determining an average vector for each concept and then performing the subtraction. As Figure 1A As shown in , subspace vectors 102 (gendered pairs) and 104 (jobs) are not orthogonal, indicating that gender terms and job terms are related.

[0030] like Figure 1B As shown in , the subspace vectors can be made orthogonal by applying a rotation to one of the subspace vectors. In this example, the working vector 104 is rotated to a vector 104' that is orthogonal to the gender vector 102.

[0031] Next, if Figure 1C and Figure 1D As shown in , the correction can be applied to any word that has a vector representation in the language model. The OSCaR method uses a graded rotation to perform the correction, where the amount of rotation for a given word is determined based on its similarity to the gender vector 102 and the working vector 104. For example, the maximum rotation angle may be equal to the rotation angle between vector 104 and vector 104′, and the minimum rotation angle may be zero. The rotation angle for a given word is selected based on the relative similarity to the gender vector 102 and the working vector 104, where the angle is lower for words whose vectors are more similar to the gender vector 102, and higher for words whose vectors are more similar to the working vector 104. In Figure 1C In , vector 116 represents any word that is neither a work word nor a gender word, such as "car", "family", or "football". Figure 1D, vector 116 is rotated to vector 116′. Hierarchical rotation can help avoid removing desired associations. For example, a word such as "actress" will be similar to gender vector 102 and will be rotated little or not at all, while a word such as "driver" will be similar to working vector 104 and will be rotated accordingly. The correction operation can be applied to each word in the language model, including the words used to determine gender vector 102 and working vector 104.

[0032] Certain embodiments of the present invention provide debiasing techniques that may be more effective than OSCaR in removing biases without removing desired associations. Some embodiments may use hierarchical rotations similar to OSCaR. However, the center of rotation is selected in a different manner than OSCaR. In some embodiments, corrections may be performed iteratively to further improve the debiasing, and the number of iterations may be a fixed number or a number selected based on a convergence criterion, examples of which are described below. In some embodiments, the debiasing techniques may be extended to more than two subspaces.

[0033] According to some embodiments, "concept" can be regarded as a set of words with a high degree of mutual similarity and can be represented as, for example, the average point of those words. For example, a concept can include clear male words (such as "man", "he", "his", "him", "boy"). Another concept can include clear female words (such as "woman", "she", "her", "hers", "girl"). (For the sake of simplicity of description, gender is regarded as binary in this article, while admitting that gender is in fact not limited to binary; "male" and "female" can also be understood as the end area in the gender spectrum.) Given a pair of concepts, a direction (or "concept vector") can be defined as a vector between the average values ​​of the two concepts. For two pairs of concepts, two concept vectors can be defined. Subsequently, by defining a center point, a given word vector is translated to the center point, and then a hierarchical rotation is applied, and correction can be performed in the subspace spanned by the two concept vectors.

[0034] Figure 2 A flow chart of a process 200 for iterative subspace revision is shown according to some embodiments. The process 200, which may be implemented in a computer system, performs debiasing associated with two concept pairs.

[0035] At block 202, a vectorized language model is obtained. The vectorized language model may be obtained by training the model using a document corpus, which may include any document in the modeled language. Existing models may be used, including non-contextual models such as Word2Vec, GloVe, FastText, etc. Other algorithms and techniques for language modeling may also be used, and the debiasing operations described herein may be applied to a range of vectorized language models. In some embodiments, block 202 may include obtaining a pre-trained language model from some other source.

[0036] At frame 204, a word list can be created for each concept pair to be applied for debiasing. As used herein, "concept" generally refers to a word set with a high degree of mutual similarity, and "concept pair" refers to two concepts (such as male / female, pleasant / unpleasant, occupation / family, etc.) that are considered to be mutually exclusive, mutually tense or mutually opposed in a sense. For example, it is assumed that gender and occupation / family terms need to be debiased (removing the correlation between gender and occupation / family terms). In this example, four concepts are involved (male and female form a concept pair, and occupation and family form another concept pair), and four word lists will be created. It should be understood that concept pairs can be defined as needed, and any two concept pairs can be selected for debiasing. In practice, these selections can be driven by people's understanding and intuition about language and culture (for example, "female" words may be biased towards "family" words, and "male" words may be biased towards "occupation" words).

[0037] In some embodiments, word lists can be manually created for various concepts, such as by having a person or a group of people generate a customized word list representing each concept. In other embodiments, each list can be manually seeded (e.g., by having a person or a group of people list a dozen or so representative words) and further expanded using vector representation. For example, a seed word list provided by a person can be expanded by calculating an average vector according to the seed word list and identifying up to a certain number of nearest neighbor words in the vector space. In various embodiments, a word list can include about a dozen to a hundred words; the specific scale is not important. The same word can be included in multiple lists; for example, "uncle" may be both a male word and a family word. Some examples are provided below.

[0038] At block 206, a "concept mean" (μ) may be calculated for each word list, for example by calculating the mean of the vectors of the words in each list. Each concept mean may be viewed as representing one of the concepts. In mathematical terms, assume that debiasing is to be performed between two pairs of concepts, such as male / female gender and occupation / family. The word lists for the first pair of concepts may be represented as sets A and B, and the word lists for the second pair of concepts may be represented as sets X and Y. The mean of set A may be defined as:

[0039]

[0040] where |A| is the number of elements (words) in set A, and a represents the vector coordinates of a particular word in set A. Means μ(B), μ(C), and μ(D) can be defined in a corresponding manner for other concepts.

[0041] At block 208, a center point for the subspace rotation may be calculated; for example, the center point may be the average of the conceptual averages calculated at block 206. In mathematical terms, the center point may be calculated using the following formula:

[0042]

[0043] At block 210, a subspace direction (or subspace vector) may be calculated for each concept pair. For example, the subspace direction for the gender subspace may be calculated by subtracting the concept mean of the "male" word list from the concept mean of the "female" word list (or vice versa). Similarly, the subspace direction for the occupation / family terms may be calculated by subtracting the concept mean of the "occupation" word list from the concept mean of the "family" word list (or vice versa). More generally, a subspace vector may be defined as:

[0044] v 1 =μ(A)-μ(B); (5)

[0045] v 2 =μ(X)-μ(Y). (6)

[0046] Centered and projected to span v 1 and v 2 After going up, the midpoint c AB and c XY Close to the origin, especially in the gap ‖c AB -c XY ‖ Smaller and / or concatenated vector c AB -c XY Almost the same as v 1 and v 2 When orthogonal.

[0047] After block 210, the center point c and the subspace vector v 1 and v 2 Can be used to modify any or all of the word vectors in the language model. This word vector can include word vectors corresponding to words in the word list for each concept, as well as word vectors corresponding to words that are not in any of the word lists.

[0048] More specifically, at block 212, the word vectors in the vectorized language model may be re-centered using the center point calculated at block 208, for example, by subtracting the center point from the vector (a translation operation). In mathematical terms, for a word vector w, the re-centered vector w c can be calculated as:

[0049] w c =wc, (7)

[0050] where c is given by the following equation (4).

[0051] At block 214, the re-centered word vectors may be modified, for example, by rotating within the span defined by the subspace vectors. In some embodiments, the modification may use a hierarchical rotation similar to or the same as that used in OSCaR. For example, a rotation matrix may be defined as rotating v 2 Angle θ to v 1 Orthogonal vector v 2 ′. For any other re-centered word vector w c The rotation angle can be based on the word vector and v 1 For example, similar to OSCaR, the rotation angle θ for the re-centered word vector is w Can be defined as:

[0052]

[0053] in And θ′=arccos <v 1 ,v 2 >. (The symbols <·,·> indicate vector dot products, and ‖·‖ indicates the magnitude of a vector.)

[0054] Equation (8) is mathematically similar to the hierarchical rotation used for correction in OSCaR. However, in process 200, the hierarchical rotation is applied after re-centering at block 212, which may produce very different results from OSCaR. It should be understood that other hierarchical rotations or other correction techniques may be substituted.

[0055] At block 216 , the revised term vector may be de-centered, for example, by adding the center point calculated at block 208 to the revised term vector (reversing the translation applied at block 212 ).

[0056] By executing blocks 212 through 216 for each word vector, modified vectors may be generated for any or all words in the language model, including but not limited to words in a word list used to define concept pairs.

[0057] One round of correction may not make the concept vectors completely orthogonal. That is, if the concept means μ(A), μ(B), μ(C), and μ(D) and the vector v 1 and v 2 Using the original word list produced by process 200 and the revised word vectors, v 1 and v 2 may not be orthogonal. In some embodiments, process 200 may iterate to approach orthogonality of concept vectors. Therefore, at block 220, it may be determined whether another revision iteration should be performed. Various stopping criteria may be used. For example, a predetermined fixed number of iterations (e.g., 1, 2, 4, 10, or some other number) may be selected. As another example, a convergence criterion may be defined. The convergence criterion may be based on modifying the word vectors for the words in the word list and determining the subspace direction (or vector v 1 With v 2 The subspace direction is recalculated after the dot product between (i.e., the word vectors) has shifted by how much; iterations can continue until the shift drops below a certain threshold. (As shown in the example below, process 200 can converge within ten or fewer iterations.) If the correction procedure should be iterated, process 200 returns to block 206, using the revised word vectors as input. (The same word list can be used in each iteration.) Once the last iteration is complete, process 200 can return the revised word vectors at block 222.

[0058] In process 200, the only extension to the word vectors is the correction (e.g., hierarchical rotation) applied at box 214. As noted, this correction can be applied to all word vectors in the language model as a (sub)differentiable continuous shift, and thus generalizes to all other vectorized representations that may carry some connotations in the connotation of the concept (including words that are not in the word list generated at box 204). For example, a statistically gendered name (such as Amy or John) can carry or represent gender information in the embedding, but it may not be desirable to assign a gender to the name because the person with the name may not identify with the statistically most likely gender. It should also be noted that after rotating in the subspace, the full dimensionality of the vector is restored. Therefore, the rotation may affect 2 dimensions out of a large number (e.g., 300 or possibly more) of dimensions on an appropriate basis, and the overall impact on most word representations may be small, with the words that are most strongly associated with the target concept vector being most affected.

[0059] Process 200 provides iterative modification of two subspaces. In some embodiments, more than two subspaces may be modified simultaneously. Figure 3 A flow chart of a process 300 for iterative modification of three subspaces according to some embodiments is shown. Process 300 may be similar to process 200, except that modification is performed in stages for each subspace during each iteration.

[0060] At block 302, Figure 2 Similar to block 202 of FIG. 3 , a vectorized language model is obtained. At block 304, Figure 2 Similar to block 204 of , a word list can be created for each concept to which debiasing is to be applied. In this case, assume that there are three pairs of concepts to consider (e.g., male / female, career / family, and pleasant / unpleasant). In mathematical terms, multiple pairs of concepts can be represented as sets A and B, X and Y, and R and S.

[0061] At block 306, a "concept average" (μ) may be calculated for each word list, for example by calculating the average of the vectors of the words in each list. Each concept average may be considered to represent one of the concepts. Block 306 may be similar to Figure 2 The block 206 of is similar (eg, using equation (1) to calculate each concept mean).

[0062] At block 308, Figure 2 Similar to block 208 of , the center point for the subspace rotation can be calculated. In some embodiments, the center point can be defined as:

[0063]

[0064] At block 310, a subspace direction (or subspace vector) may be calculated for each concept pair. Block 310 may be similar to Figure 2 In some embodiments, the direction may be defined as a vector:

[0065] v 1 =μ(A)-μ(B); (10)

[0066] v 2 =μ(X)-μ(Y); (11)

[0067] v 3 =μ(R)-μ(S). (12)

[0068] After block 310, the center point c and the subspace vector v 1 、v 2 and v 3 Can be used to modify any or all of the word vectors in the language model. This word vector can include word vectors corresponding to words in the word list for each concept, as well as word vectors corresponding to words that are not in any of the word lists.

[0069] More specifically, at block 312 , the word vectors in the vectorized language model may be re-centered using the center point calculated at block 308 , similar to block 212 , such as by subtracting the center point from the vector.

[0070] At block 314, the first pair of subspaces (referred to as v for convenience) may be 1 and v 2 ) performs a first correction operation on the word vector. In some embodiments, the first pair of subspaces can be a pair of subspaces that are most nearly orthogonal. The correction can use a hierarchical rotation similar to or the same as the hierarchical rotation used at block 214 of process 200.

[0071] At block 316, the third subspace v 3 Can be projected into the subspace v 1 and v 2 The projection is represented in this paper as

[0072] At block 318, the pair of subspaces v 3 and A second modification operation is performed on the term vector (as modified by the first modification operation at block 314). As at block 314, the modification operation may use a hierarchical rotation similar to or the same as the hierarchical rotation used at block 214 of process 200.

[0073] At block 320 , the revised term vector produced by block 318 may be un-centered, for example, by adding the center point calculated at block 308 to the vector (reversing the translation applied at block 312 ).

[0074] By executing blocks 312 through 318 for each word vector, modified vectors may be generated for any or all words in the language model, including but not limited to words in a word list used to define concept pairs.

[0075] As in process 200, one round of revision may not make the concept vectors completely orthogonal. Therefore, at box 322, it is determined whether another revision iteration should be performed. Various stopping criteria can be used, including any of the criteria described in box 220 of reference process 200 above. If the revision procedure should be iterated, process 300 returns to box 306, using the revised word vector as input. (The same word list can be used in each iteration.) It should be noted that this makes each iteration perform two revision steps, rather than iterating only on a pair of subspaces. Once the last iteration is completed, process 300 can return to the revised word vector at box 324.

[0076] Processes 200 and 300 are illustrative, and variations and modifications are possible. The operations described in sequence can be performed in parallel, and unless otherwise specified by logic, the order of operations can be modified as needed. The correction can be applied to language models of any size, including vectors of any number of words and any desired dimension. The number of iterations can be 1 or more, and the stopping criterion can be a predetermined number of iterations (e.g., 4 or 10 iterations) or can be dynamically determined, for example, based on the analysis of the results after each iteration. In addition, although process 200 illustrates the correction for two subspaces (two pairs of concepts), and process 300 illustrates the correction for three subspaces (three pairs of concepts), those skilled in the art who can access the present disclosure will understand that by generalizing process 300 to continuously project additional subspaces into the previous subspace and applying the correction, the correction process can be extended to a larger number of subspaces. The orthogonalization technique described herein does not remove information about words, but represents the information in a subspace orthogonal to other attributes.

[0077] To further illustrate the Iterative Subspace Revision (ISR) process according to various embodiments, example implementations will now be described. It should be understood that these examples are intended to be illustrative, not restrictive. In these examples, performance metrics are defined as estimating how well a given debiasing process corrects concepts (or orthogonalizes concepts) and how well a given debiasing process reduces bias. In particular, the dot product score is used herein to measure the level of orthogonality between two concept pairs (sometimes also referred to as "linear learning concepts"). The dot product can be represented as <v 1 ,v 2 >. If the concept pair is orthogonal, the dot product should be 0.

[0078] To measure bias, the Word Embedding Association Test (WEAT) can be used. The goal of WEAT is to measure the level of human-like stereotype bias associated with words in word embeddings. WEAT uses four word sets: two target word sets X and Y and two attribute word sets A and B. For each word w∈(X∪Y), the association of w with sets A and B can be calculated as:

[0079]

[0080] The WEAT score is obtained by averaging equation (13) across all words in sets X and Y:

[0081]

[0082] The scores WEAT(X,Y,A,B) can be normalized by the standard deviation s(w,A,B) across all words w∈(X∪Y). Normalized WEAT scores are usually in the range [-1,1], and values ​​closer to 0 indicate less biased associations. The effect of debiasing can be measured by comparing the WEAT scores before and after debiasing.

[0083] In a first example, the association between gender words and "pleasure / unpleasure" words (ie, words with strong pleasant or unpleasant emotional resonance) was analyzed. Table 1 lists the gender words used, and Table 2 lists the pleasant / unpleasant words.

[0084]

[0085] Table 1

[0086]

[0087]

[0088] Table 2

[0089] Debiasing is performed on the initial language model using two different implementations of process 200 described above. In the following description, "SR" denotes an implementation with a single iteration of process 200, while "ISR" denotes an iterative implementation with 10 iterations. For comparison, debiasing is also performed on the same initial language model using each of five different conventional techniques, specifically: linear projection (LP); hard debiasing (HD); iterative null space projection (INLP); OSCaR; and an iterative version of OSCaR, which is referred to herein as iOSCaR.

[0090] Figure 4 4 is a table 400 summarizing WEAT scores for various debiasing processes. For reference, the raw WEAT scores for the initial language model, before any debiasing, are shown at column 401. The results of the conventional debiasing process are shown in columns 402 (LP), 403 (HD), 404 (INLP), 405 (OSCaR), and 407 (iOSCaR). The results obtained using an implementation of process 200 are shown at columns 406 (SR) and 408 (ISR).

[0091] The convergence of the ISR is also studied by generating the WEAT score and the dot-product score (dotP) after each iteration. Figure 5 5 is a table summarizing the scores at each iteration for ISR. Each column corresponds to a different iteration. For comparison, the corresponding scores for iOSCaR are shown. It should be noted that ISR converges to a point integral score close to zero, indicating successful debiasing. In contrast, iOSCaR does not converge to any particular point integral score. It should also be noted that the WEAT score for ISR converges quickly to a stable value, and only 2 to 4 iterations may be sufficient.

[0092] As further examples, the same process is applied to other concept pairs. Figure 6 600 which summarizes the WEAT scores for different processes applied to different concept pairs. "Gender (M / F)" represents the gender words listed in Table 1 above. "Pleasant / Unpleasant" represents the pleasant / unpleasant words listed in Table 2 above. "Career / Family" words are listed in Table 3. "Math / Art" words are listed in Table 4, and "Science / Art" words are listed in Table 5. "Name (M / F)" words are listed in Table 6. "Flowers / Insects" words are listed in Table 7. "Music / Weapons" words are listed in Table 8. Figure 6 As shown in , for most dataset pairs, ISR achieves the minimum WEAT scores of all tested methods.

[0093]

[0094] Table 3

[0095]

[0096] Table 4

[0097]

[0098]

[0099] Table 5

[0100]

[0101] Table 6

[0102]

[0103] Table 7

[0104]

[0105] Table 8

[0106] As yet another example, a study of the effects of cross-validation is performed. In cross-validation, different word lists are used for training (e.g., determining the word lists used to represent v in an ISR). 2 Map to v 2 ' center point and rotation angle) and test (e.g., calculate WEAT score). To support cross-validation, a larger word list is constructed by using a small word list (in Tables 1 to 8) and determining the mean, then selecting the 60 words closest to each mean. Each list is randomly split 50 / 50 into a test subset and a training subset. Debiasing is performed on the training subset, and WEAT scores are evaluated on the test subset. This process is repeated 10 times (10 different random splits), and the WEAT scores are averaged across the random splits. Figure 7 shows a table 700 of WEAT scores obtained for various debiasing procedures with test / train splits, and Figure 8 Table 800 shows WEAT scores obtained using the same 60-word list but without a test / train split. Figure 7 and Figure 8 It is shown that ISR consistently performs best, with gendered names getting the worst results. It should also be noted that projection-based methods (such as LP, HD, and INLP) perform better for test / train splits, while rotation-based methods (such as SR and ISR) perform better without splits. This may be because rotation-based methods are more precise and therefore more affected by smaller word lists.

[0107] Another consideration in evaluating debiasing algorithms is the extent to which they destroy important information in the vectorized representation. For example, if the gender subspace is removed (e.g., by a projection-based debiasing method), certain task-specific challenges (such as pronoun resolution involving gender) may be adversely affected.

[0108] Task-based information retention can be quantified using a score referred to herein as "self-WEAT" or "SWEAT". Given a pair of word lists A, B that define a concept pair (e.g., masculine and feminine gendered terms), the SWEAT score measures how the consistency within each word list compares to the cross-consistency of the other word lists. To determine the SWEAT score, each word list can be randomly split: List A can be split into List A 1 and A 2 , and list B can be split into list B 1 and B 2 WEAT scoreWEAT(A 1 ,A 2 ,B 1 ,B 2 ) can be calculated using equations (13) and (14) above. The SWEAT score can be defined as the average of this WEAT score across ten different random splits. If lists A and B retain their distinct meanings after debiasing, the SWEAT scores before and after debiasing should be similar. If the dissimilarity is reduced or destroyed, the SWEAT score will decrease towards 0 after debiasing.

[0109] Fig. 9 Table 900 shows SWEAT scores for various concept pairs and various debiasing methods. "Concept 1" (column 901) is the concept to which linear debiasing is applied, and "Concept 2" (column 902) is the second concept to which rotation-based debiasing is applied. Column 903 shows the SWEAT scores before debiasing. It should be noted that SR (column 908) and ISR (column 910) have little effect on the SWEAT scores, indicating that ISR preserves most or all of the relevant information. In contrast, conventional debiasing methods can significantly reduce the SWEAT scores.

[0110] As yet another example, an implementation of process 300 is applied to debias the following three concept pairs: gendered male / female terms; pleasant / unpleasant terms; and statistically gendered male / female names. A large word list generated as described above is used. It is observed that the gendered terms and pleasant / unpleasant terms have the smallest dot product; therefore, a first correction is performed using these two concept pairs, followed by correction using statistically gendered names. Fig.10Table 1000 shows the WEAT scores and dot product scores for three pairs of ISR processes across ten iterations for each pair of concepts. In Table 1000, "GT" represents gendered terms, "GN" represents gendered names, and "P / U" represents pleasant / unpleasant terms. Table 1000 shows that after 10 iterations, all pairwise WEAT scores decrease significantly (to about 0.04), while the pairwise dot products also decrease toward zero.

[0111] Fig.11 Table 1100 shows the SWEAT scores for each concept pair at each iteration of the three-pair ISR process. Note that even after 10 iterations, the pleasant / unpleasant terms still retain most of their SWEAT scores, while the SWEAT scores for both the gendered terms and the gendered names decrease. This is likely due to the fact that the gendered terms and gendered names start out with a high correlation (dot product score of 0.79), and significant distortion occurs to orthogonalize the concept vectors. Even so, Fig.11 The SWEAT scores shown in are also higher than the corresponding SWEAT scores for other methods (such as Fig. 9 ).

[0112] As shown in the aforementioned examples, debiasing according to some embodiments can significantly improve the amount of debiasing compared to conventional methods. For example, when measured using standard WEAT scores, debiasing according to some embodiments can achieve a 95% improvement, rather than an improvement of approximately 20% to 30%. This significant improvement was maintained in test-train split experiments (which are rarely attempted in this field). In addition, while some conventional debiasing techniques (e.g., hard debiasing, INLP) are based on projections and therefore destroy information about the concept whose bias is attenuated (e.g., gender), debiasing according to some embodiments can be shown to retain relevant information. In addition, debiasing according to some embodiments can be extended to multiple subspace debiasing (e.g., as described above with reference to Figure 3 ), which can help resolve the intersection problem. The resulting representation creates multiple subspaces, all of which are orthogonal. The resulting representation is also more interpretable than other debiased representations. After applying this orthogonalization to multiple subspaces, it is possible to perform a basis rotation (without changing any cosine similarities or Euclidean distances), which aligns each of the identified and orthogonalized concepts with one of the coordinate axes. Therefore, the power, flexibility, and compression of the distributed representation can be maintained, while still being able to recover an intuitive and simple coordinate representation of those features, at least for select concepts. In downstream tasks, features associated with the debiased concepts can be ignored if they should not be involved in some aspect of the training (such as gender for resume classification), or these features can be retained for coreference resolution.

[0113] Although the foregoing description refers to specific embodiments, it will be appreciated by those skilled in the art that the description is not exhaustive of all embodiments. There may be many variations and modifications. For example, although male / female, occupation / family and other specific concept pairs are used as examples, debiasing can be performed for any two or more pairs of concepts in a similar manner. As another example, a financial services institution may maintain data (which may be anonymous data) related to financial transactions of various users, and may wish to create a vectorized language model from the data to support operations such as fraud detection or to recommend merchants to be patronized or articles to be purchased based on past behavior patterns. There may be an unwanted association between a location and a specific merchant or article that may be desired to be removed. More generally, concepts may be defined by clustering in a language model, and subspaces may be defined by selecting concept pairs.

[0114] Furthermore, the embodiments described above use concept pairs where two word lists are defined to represent distinct concepts. An alternative approach uses a subspace defined by a single word list (e.g., work). In this approach, a single set subspace can be defined as the top principal components of the vectors in the word lists. Thus, given two word lists, The row l in the (high-dimensional vector space of the language model) 1 and l 2 To identify the center, we can analytically determine a pair of points p that are as close as possible 1 ∈l 1 and p 2 ∈l 2 The center c can be chosen to be p 1 With p 2 The midpoint between 1 +p 2 ) / 2. The correction can be performed as described above with reference to process 200 (or process 300). Applying this method iteratively makes the dot product converge to zero; however, in the absence of a contrast concept, assessing information preservation becomes challenging.

[0115] The vectorized language representation may include representations of any number of words and may correspond to any natural language. The word vector in the representation may include any number of vector components. The techniques described herein may be used to remove unwanted associations (referred to herein as "biases") between two or more pairs of concepts. Whether the association is desired or unwanted may depend on the specific purpose for which the vectorized language representation is used. As described above, debiasing may be performed without the need for computationally intensive training or retraining of the model, and debiasing according to some embodiments may provide a lightweight extension to the vectorized language model.

[0116] A variety of techniques can be used to generate word lists. Examples include custom word lists generated by a person or a group of people. Existing word lists available from various sources can be used. In some embodiments, short word lists (e.g., a dozen or so words) generated by a person can be expanded using an automated process. For example, the average word vector in the initial (short) word list can be calculated (e.g., using equation (1) above), and words for the final word list can be selected based on similarity to the average vector, e.g., the closest 40, 60, or 100 words, or words within a certain threshold distance from the average vector. When using such techniques, the words in the initial list may or may not be included in the final word list.

[0117] The technology described herein can be implemented by suitable programming of a general-purpose computer. In some embodiments, the computer system includes a single computer device, wherein the subsystem can be a component of the computer device. The computer device can have various form factors, including, for example, a smart phone, a tablet computer, a laptop computer, a desktop computer, etc. In other embodiments, the computer system can include multiple computer devices, each of which is a subsystem with internal components. This debiasing technique described herein can improve the performance of various tasks using natural language models, such as by reducing the influence of stereotype associations in the training corpus that may cause unwanted stereotyped behavior in the natural language processing system.

[0118] A computer system may include, for example, a plurality of components or subsystems connected together by an external interface or by an internal interface. In some embodiments, a computer system, subsystem or device may communicate via a network. In such cases, one computer may be considered a client and another computer may be considered a server, wherein each computer may be part of the same computer system. The client and the server may each include a plurality of systems, subsystems or components.

[0119] It should be understood that any embodiment of the present invention can be implemented in the form of control logic using hardware (e.g., an application specific integrated circuit or a field programmable gate array) and / or using computer software, wherein a general-purpose programmable processor is modular or integrated. As used herein, a processor includes a single-core processor, a multi-core processor on the same integrated chip, or multiple processing units on a single circuit board or networked. Based on the disclosure and the teachings provided herein, a person of ordinary skill in the art will know and understand other ways and / or methods of implementing embodiments of the present invention using hardware and combinations of hardware and software.

[0120] Any software component or functionality described in this application may be implemented as software code executed by a processor using any suitable computer language such as Java, C, C++, C#, Objective-C, Rust, Golang, Swift, or a scripting language such as Perl or Python using, for example, conventional or object-oriented techniques. The software code may be stored as a series of instructions or commands on a computer-readable storage medium; suitable media include random access memory (RAM), read-only memory (ROM), magnetic media such as a hard drive or floppy disk, or optical media such as a compact disk (CD) or digital versatile disk (DVD), flash memory, and the like. A computer-readable storage medium may be any combination of such storage devices or other storage devices capable of retaining stored data.

[0121] Such programs can also be encoded and transmitted using carrier signals suitable for transmitting via wired networks, optical networks and / or wireless networks that meet multiple protocols including the Internet. Therefore, the computer-readable transmission medium according to an embodiment of the present invention can be created using data signals encoded with such programs. The computer-readable medium encoded with program code can be encapsulated with compatible devices or provided separately from other devices (e.g., downloaded via the Internet). Any such computer-readable medium can reside on or in a single computer product (e.g., hard drive, CD or entire computer system), and can be present on or in different computer products in a system or network. A computer system may include a monitor, printer or other suitable display for providing any result mentioned herein to a user.

[0122] Any method described herein may be performed completely or in part with a computer system comprising one or more processors that may be configured to perform these steps. Therefore, embodiments may relate to a computer system configured to perform the steps of any method described herein, which may have different components that perform corresponding steps or corresponding step groups. Although presented in numbered steps, the method steps herein may also be performed simultaneously or in different orders. In addition, the parts of these steps may be used together with the parts of other steps of other methods. In addition, all or part of the steps may be optional. In addition, any step of any method may be performed with a module, a circuit or other means for performing these steps.

[0123] Without departing from the spirit and scope of the embodiments of the invention, the specific details of the particular embodiments may be combined in any suitable manner. However, other embodiments of the invention may be directed to specific embodiments related to each individual aspect, or specific combinations of these individual aspects.

[0124] Unless expressly indicated to the contrary, the recitation of "a / an" or "the" is intended to mean "one or more." Unless expressly indicated to the contrary, the use of "or" is intended to mean an "inclusive or" rather than an "exclusive or."

[0125] All patents, patent applications, publications, and descriptions mentioned herein are incorporated by reference in their entirety for all purposes. No admission is made that they are prior art.

[0126] The above description is illustrative and not restrictive. After reading this disclosure, many variations of the present invention will become apparent to those skilled in the art. Therefore, the scope of patent protection should not be determined with reference to the above description, but should be determined with reference to the attached claims and their full scope or equivalent.

Claims

1. A computer-implemented method comprising: Obtaining vectorized language representations for a plurality of terms, wherein the vectorized language representations include a plurality of vectors in a vector space such that each term has an associated vector; Identify two pairs of concepts to be debiased; Obtain a list of representative words for each concept in each pair of concepts; For each concept, calculating a corresponding concept average value based on the vectors associated with the words in the representative word list for the concept; Calculating the center point of the corresponding concept averages; Compute the corresponding subspace directions for each pair of concept means; as well as For one or more vectors of the plurality of vectors in the vector space, calculating a debiasing vector, wherein calculating the debiasing vector comprises: Re-centering the vector on the center point; performing a correction operation on the vector with respect to the corresponding subspace direction; and Do not re-center the vector.

2. The method of claim 2, further comprising: Iteratively calculating the corresponding concept averages; calculating the center point; calculating the corresponding subspace direction; re-centering the vector; executing the correction operation; And the vector is not re-centered until a stopping criterion is met. The method of claim 2 , wherein the stopping criterion is a convergence criterion based on a change in a performance metric. The method of claim 2 , wherein the stopping criterion specifies a fixed number of iterations.

5. The method of claim 1 , wherein performing the correction operation on the vector comprises: determining a rotation angle to be applied to the vector; as well as Rotates the vector by the rotation angle.

6. The method of claim 5, wherein the rotation angle is based on the relative similarity of the vector to the corresponding subspace direction for each pair of concept means.

7. The method of claim 1, wherein the vectorized language representation is obtained from structured text.

8. The method of claim 1, further comprising: The one or more debiasing vectors are stored.

9. A computer system comprising: A memory for storing a vectorized language representation, the vectorized language representation comprising a plurality of vectors in a vector space such that each word has an associated vector; as well as a processor coupled to the memory and configured to: Identify two pairs of concepts to be debiased; Obtain a list of representative words for each concept in each pair of concepts; For each concept, calculating a corresponding concept average value based on the vectors associated with the words in the representative word list for the concept; Calculating the center point of the corresponding concept averages; computing the corresponding subspace directions for each pair of concept means; and For one or more vectors of the plurality of vectors in the vector space: Re-centering the vector on the center point; performing a correction operation on the vector with respect to the corresponding subspace direction; and Do not re-center the vector.

10. The computer system of claim 9, wherein the processor is further configured to iteratively calculate the corresponding concept average, calculate the center point, calculate the corresponding subspace direction, realign the vector, perform the correction operation, and unrealign the vector until a stopping criterion is met.

11. The computer system of claim 9, wherein the representative word list is generated by a human.

12. The computer system of claim 9, wherein the processor is further configured such that obtaining a representative word list for each concept in each pair of concepts comprises: For at least one of the concepts: Receiving an initial list of words generated by a human; determining an average of word vectors corresponding to words in the initial word list; as well as Words for the representative word list are selected based on vector similarity to an average of the word vectors.

13. The computer system of claim 9, wherein the processor is further configured such that performing the modification operation on the vector comprises: determining a rotation angle to be applied to the vector; as well as Rotates the vector by the rotation angle.

14. The computer system of claim 13, wherein the rotation angle is determined based at least in part on an angle between the vector and one of the subspace directions.

15. A computer-readable storage medium having program code instructions stored therein, the program code instructions, when executed by a processor in a computer system, causing the computer system to perform a method, the method comprising: Obtaining a vectorized language representation comprising a plurality of terms, wherein the vectorized language representation comprises a plurality of vectors in a vector space such that each term has an associated vector; identifying pairs of target concepts to be debiased; generating a list of representative words for each concept in each pair of target concepts; For each concept, calculating a corresponding concept average value based on the vectors associated with the words in the representative word list for the concept; Calculating the center point of the corresponding concept averages; Compute the corresponding subspace directions for each pair of concept means; Centering each vector in the vectorized language representation on the central point; performing a first correction of each vector relative to first two of the subspace directions; Projecting a third subspace direction among the subspace directions onto the span of the first two subspace directions; performing a second correction of each vector with respect to the third subspace direction and the projection; as well as Make each vector not centered.

16. The computer-readable storage medium of claim 15, further comprising: Iteratively calculating the corresponding concept averages; Calculating the center point; calculating the corresponding subspace direction; Center each vector; performing said first correction of each vector; Projecting the third of the subspace directions; performing the second correction of each vector; and uncentering each vector until a stopping criterion is met.

17. The computer-readable storage medium of claim 16, wherein the stopping criterion is a convergence criterion based on a change in a performance metric.

18. The computer-readable storage medium of claim 16, wherein the stopping criterion specifies a fixed number of iterations.

19. The computer-readable storage medium of claim 15, wherein performing each of the first modification and the second modification on each vector comprises: determining a rotation angle to apply to the vector based at least in part on a relative similarity of the vector to the corresponding subspace directions for a pair of concept averages; as well as Rotates the vector by the rotation angle.

20. The computer-readable storage medium of claim 15, wherein the first two of the subspace directions are a pair of the subspace directions that are most nearly orthogonal.

Images