Methods, programs, and computer systems (bias analysis and mitigation through protective attribute detection and evaluation classification)

Abstract

This invention provides a method and computer program for accurately detecting various protection attributes and classifying / balancing the evaluations for each attribute. [Solution] The method receives a training dataset to be assessed for unnecessary bias, matches training data samples with keywords in a protected attribute taxonomy to identify instances of keywords present in text clauses, performs semantic disambiguation on instances to distinguish protected attribute mentions from non-protected attribute mentions, inputs the resulting protected attribute mention instances into a trained evaluation classification engine, annotates them with evaluation annotations based on the evaluation classification, generates a distribution of evaluation classifications for training data samples in the training dataset and removes unnecessary bias, stores the balanced training dataset and provides it to the machine learning training logic.

Description

Technical Field

[0001] This application generally relates to data processing apparatuses and methods, and more specifically, to computing tools and computing tool operations / functions for bias analysis and mitigation through protected attribute detection and evaluation classification.

Background Art

[0002] Recent advances in artificial intelligence (AI) and machine learning computer models involve the use of language models (LM) or large language models (LLM), which are trained with large amounts of training data obtained from a variety of different sources.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Due to the diverse nature of the training data and its various sources, the training data often contains undesirable biases with respect to protected attributes such as gender, religion, nationality, ethnicity, etc. Training a machine learning computer model such as an LLM using such training data having unnecessary biases will result in perpetuating these unnecessary biases in the results generated by the machine learning computer model.

Means for Solving the Problems

[0004] The summary of this invention is provided to introduce, in a simplified form, a selected concept that will be further described in the mode for carrying out the invention in this specification. The summary of this invention is not intended to identify the important factors or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0005] In one exemplary embodiment, a method is provided in a data processing system for accurately detecting various protective attributes and classifying / equalizing evaluations for each attribute. The method includes the steps of obtaining a taxonomy of protective attributes having associated keywords, and performing keyword matching operations to detect instances of keywords associated with protective attributes in a first dataset. The method further includes performing evaluation classification on each instance of a keyword to classify evaluations for the corresponding protective attribute. The evaluation classification is one of positive, negative, or neutral evaluations. The method also includes annotating each instance with evaluation annotations corresponding to the evaluation classification for that instance to generate an evaluation-annotated dataset. In addition, the method includes performing bias analysis and evaluation classification distribution equilibration on a second dataset to mitigate undesirable biases based on the evaluation-annotated dataset.

[0006] In other exemplary embodiments, a computer program product is provided which comprises a computer-readable or computer-readable medium having a computer-readable program. When the computer-readable program is executed on a computing device, the computing device is caused to perform various operations and combinations thereof from among the operations outlined above with respect to exemplary embodiments of the method.

[0007] In yet another exemplary embodiment, a system / device is provided. The system / device may comprise one or more processors and memory coupled to the one or more processors. The memory may, when executed by the one or more processors, have instructions that cause the one or more processors to perform various operations and combinations thereof from among the operations outlined above with respect to exemplary embodiments of the method.

[0008] These and other features and advantages of the present invention will be described in the following detailed description of exemplary embodiments of the present invention, or will become apparent to those skilled in the art in consideration of such detailed description. [Brief explanation of the drawing]

[0009] The present invention, as well as preferred modes of use and further purposes and advantages, will be best understood by referring to the following detailed description of exemplary embodiments in conjunction with the accompanying drawings.

[0010] [Figure 1] This is an illustrative diagram of a distributed data processing system environment in which an exemplary embodiment may be implemented and in which at least a portion of the computer code involved in carrying out the method of the present invention may be executed.

[0011] [Figure 2] This is an exemplary block diagram showing the primary operating components of a bias analysis and mitigation system according to one exemplary embodiment.

[0012] [Figure 3] This is an illustrative diagram of a taxonomy of protection attributes and corresponding keywords according to one exemplary embodiment.

[0013] [Figure 4A] This is an illustrative figure illustrating a frequency bias analysis of keywords associated with exemplary protective attributes of ethnicity, according to one exemplary embodiment.

[0014] [Figure 4B] This is an exemplary figure illustrating a positive frequency bias analysis of keywords associated with exemplary protective attributes of ethnicity, according to one exemplary embodiment.

[0015] [Figure 4C] This is an exemplary figure illustrating a positive frequency bias analysis of keywords associated with exemplary protective attributes of ethnicity, according to one exemplary embodiment.

[0016] [Figure 5] An exemplary diagram of examples of positive, neutral, and negative evaluations regarding protection attributes according to one exemplary embodiment.

[0017] [Figure 6] A flowchart outlining exemplary operations for training an evaluation classification engine according to one exemplary embodiment.

[0018] [Figure 7] A flowchart outlining exemplary operations for performing bias analysis and mitigation according to one exemplary embodiment. **MODE FOR CARRYING OUT THE INVENTION**

[0019] Exemplary embodiments provide computing tools and computing tool operations / functions for bias analysis and mitigation through protection attribute detection and evaluation classification. Exemplary embodiments specifically aim to provide computer tools and computer tool operations and functions that solve the problem of unwanted bias in training data for training machine learning computer models such as large language models (LLMs). The computing tools and computing tool operations / functions detect mentions of protection attributes in training data, classify evaluations for each protection attribute into positive, neutral, or negative classifications, annotate instances of bias using the classification of the evaluations, and then implement a pipeline that includes a taxonomy of protection attributes used to equalize the evaluation distribution in the training data set based on the annotations. To equalize the evaluation distribution, the training data set is modified to include a balanced set of training data samples with various classifications of evaluations, which can be achieved through downsampling, or deletion of training data samples, and / or addition of training data samples for implementing the equalization, e.g., downsampling of sentences with negative evaluations.

[0020] Protected Attribute Detection In some exemplary embodiments, semantic disambiguation is implemented to accurately detect various attributes, such as "female," "black," "white," "Japanese," "American," "blind," etc., and to eliminate false positives, i.e., non-protected attribute mentions, where, for example, "My friends are Japanese" is a protected attribute mention, while "I don't speak Japanese" is a non-protected attribute mention. Semantic disambiguation may be used to generate keyword-gross pairs, where keyword is the term to be disambiguated and gross is a candidate semantic definition of the keyword, which is specialized for taxonomy-based protected attribute detection. In some exemplary embodiments, the LLM identifies these keyword-gross pairs based on the taxonomy and compiles them into training data having training data samples for negative, neutral, and positive evaluations. The training data is then used to train an evaluation classification computer model through machine learning training operations.

[0021] A taxonomy of protected attributes includes protected attributes (e.g., ethnicity, gender, age, nationality, etc.), corresponding keywords associated with these protected attributes, and definitions of these keywords. The taxonomy may be created manually, compiled from existing resources, or generated based on an LLM, prompting the LLM to generate such keywords and definitions for a given protected attribute. The taxonomy may be used to identify instances of keywords in text content, which are then subjected to semantic disambiguation to distinguish between instances targeting protected attribute references and instances targeting non-protected attribute references. Thus, when training an evaluation classification computer model, the taxonomy is used to identify instances of protected attribute keyword references, which are then rated by the LLM to determine the evaluation of the protected attribute for each instance and to annotate the corresponding portion of the text with evaluation annotations, e.g., positive, negative, or neutral. The annotated text may then be input into an evaluation classification computer model as training data for training the evaluation classification computer model via machine learning training. The evaluation classification computer model is a smaller, finely tuned language model (LM) that avoids the significant overhead of using an LLM to rate each instance of a protected attribute mention.

[0022] During runtime operation, after training an evaluation classification computer model, the taxonomy may be used again, along with its corresponding keywords and definitions, to identify instances of protected attribute mentions in text, such as the text of training data samples in the training data for training a computer model, e.g., a language model (LM) or a large language model (LLM). The trained evaluation classification computer model may then rate each instance of a protected attribute mention and classify the instances as representing a positive, negative, or neutral evaluation of the protected attribute. The instances may also be annotated with evaluation annotations. A distribution of evaluations for each protected attribute may then be generated for a particular training dataset. The distribution may then be used to modify the training dataset to mitigate undesirable biases or evaluations in the training dataset. To achieve equilibrium in evaluations for a particular protected attribute, modifications may be made, for example, by identifying evaluation annotations in the training dataset that correspond to unwanted biases and downsampling the portion of the training dataset that has such evaluation annotations, or by removing training data samples that have negative evaluations of the protected attribute. This may be done over a subset or all of such protected attributes.

[0023] While examples of downsampling are provided for negative ratings of protective attributes, it should be understood that in some cases, to achieve equilibrium, the exemplary embodiments may also need to downsample training data samples for positive and / or neutral ratings. Any modifications to the training dataset based on the rating distribution may be made to achieve equilibrium, for example, approximately the same number of training data samples for each type of rating of protective attributes across rating classifications within a given tolerance.

[0024] The balanced training dataset may then be provided to the machine learning training logic to train a machine learning computer model to perform the corresponding task. For example, this balanced training dataset may be used to train a language model (LM) or a large-scale language model (LLM) without introducing unnecessary bias into the training of the LM / LLM. It should be understood that the exemplary embodiments are not limited to training LM / LLMs and may be used to train any suitable machine learning computer model based on balanced training data with respect to protected attribute evaluation. Thus, the training of the machine learning computer model is improved by removing unnecessary bias in the training dataset used to train the machine learning computer model, resulting in a trained machine learning computer model with minimal or no unnecessary bias regarding protected attributes in its operation.

[0025] The following description provides examples of embodiments of the present disclosure, which may be modified and substituted in other embodiments. Herein, several examples are provided to further clarify the various aspects of the present disclosure.

[0026] Example 1: A method for accurately detecting various protective attributes and classifying / equalizing evaluations for each attribute. The method includes the steps of obtaining a taxonomy of protective attributes with associated keywords, and performing keyword matching operations to detect instances of keywords associated with protective attributes in a first dataset. The method further includes performing evaluation classification on each instance of a keyword to classify evaluations for the corresponding protective attribute, where the evaluation classification is one of positive, negative, or neutral evaluations. The method also includes annotating each instance with evaluation annotations corresponding to the evaluation classification for that instance to generate an evaluation-annotated dataset. In addition, the method includes performing bias analysis and evaluation classification distribution equivalence on the second dataset to mitigate undesirable biases based on the evaluation-annotated dataset. The above limitations are advantageous in that they allow for the equivalence of evaluation classifications in the dataset, which may be used to remove unwanted biases in the dataset with respect to protective attributes. This improves the training of any downstream computer processes operating on the equivalised second dataset, e.g., any machine learning computer models trained on the equivalised second dataset.

[0027] Example 2: A limitation of any of Examples 1 and 3-10, wherein the second dataset is a training dataset for training a machine learning computer model. The above limitation advantageously allows for the balancing of biases and evaluations of protective attributes found in the training dataset used to train the machine learning computer model. This allows for the removal of unnecessary biases from the training dataset before training the machine learning computer model, so that no unnecessary biases are introduced into the behavior of the resulting trained machine learning computer model.

[0028] Example 3: Any limitation of Examples 1-2 and and 4-10, wherein the machine learning computer model is a Large-Scale Language Model (LLM). The above limitation is advantageous in that it allows for the removal of unnecessary bias in the large training dataset used to train the LLM, thereby improving their behavior so that the LLMs are not biased toward / to certain protected attributes.

[0029] Example 4: Any limitation of Examples 1-3 and 5-10, wherein the step of performing the bias analysis and evaluation classification balancing includes a step of downsampling training data samples in the training dataset that have evaluation annotations indicating negative evaluations. The above limitation advantageously allows for the removal of unnecessary biased training data samples by removing them through downsampling.

[0030] Example 5: Any limitation of Examples 1-4 and 6-10, wherein the step of performing the bias analysis and evaluation classification equilibration includes a step of equilibrating the number of first training data samples in the training dataset that have evaluation annotations indicating positive evaluations and the number of second training data samples that have evaluation annotations indicating negative evaluations to be approximately the same number. The above limitation advantageously allows for the equilibration of positive and negative evaluation training data samples in the training dataset so that the training dataset does not represent training data samples that are more severely / less severely positive or negative evaluations.

[0031] Example 6: The method further comprises the step of performing semantic ambiguation on the instances of the keyword in the first dataset to distinguish between first instances in which the keyword refers to the protected attribute corresponding to the keyword and second instances in which the keyword does not refer to the protected attribute corresponding to the keyword, wherein the second instances are removed as instances relating to the evaluation classification and evaluation annotation, any limitation of Examples 1-5 and 7-10. The above limitation advantageously allows for a distinction between true positive references of a keyword that specifically indicates bias with respect to the protected attribute and false positive references of a keyword that does not indicate bias with respect to the protected attribute. Thus, the mechanism of the exemplary embodiment can focus on working with true positives and eliminate potentially confusing false positive instances.

[0032] Example 7: The method further comprises the step of performing machine learning training on an evaluation annotation classification machine learning computer model based on the annotated instances in the evaluation annotation dataset, thereby training the evaluation annotation classification machine learning computer model to classify input text to evaluation classification, where the evaluation classification is one of positive evaluation, negative evaluation, or neutral evaluation, as limited to any of Examples 1-6 and 8-10. The above limitation is advantageous in that it allows training of an evaluation classification model that can operate more efficiently to perform evaluation classification than using a large-scale language model (LLM) to perform such evaluation classification on keyword matching instances in the dataset. This reduces computational cost and resources.

[0033] Example 8: The evaluation classification performed for each instance of the keyword to classify evaluations for corresponding protection attributes is performed by prompting a Large Language Model (LLM) to specify an evaluation classification for an input text sentence, as in any of the limitations of Examples 1-7 and 9-10. The limitations above are advantageous in that they allow a pre-trained LLM to perform evaluation classifications for instances in a dataset, which can then be used to train an evaluation classification model to perform evaluation classifications. Thus, these limitations allow the LLM to be used to annotate a dataset that can then be used to train an evaluation classification model that can perform evaluation classifications for other input text sentences.

[0034] Example 9: Any limitation of Examples 1-8 and 10, wherein the step of obtaining the taxonomy includes the steps of obtaining an initial taxonomy of the protected attribute and keywords; assessing the frequency of bias for each keyword of each protected attribute in the initial taxonomy; and filtering the initial taxonomy based on the frequency of bias to generate the taxonomy of the protected attribute. The above limitation is advantageous in that it allows filtering of the taxonomy's keywords to identify those that occur most frequently in conjunction with the protected attribute.

[0035] Example 10: The method further comprises a step of evaluating the evaluation classification for each keyword of each protected attribute in the initial taxonomy, wherein the filtering is one of the limitations in Examples 1 to 9, based on an evaluation plus frequency bias index. The above limitations advantageously allow filtering of the taxonomy's keywords to identify those that occur most frequently in combination with a particular attribute and that are the most representative of the evaluations for the protected attribute.

[0036] Example 11: A system comprising one or more processors and one or more computer-readable storage media that collectively store program instructions configured, when executed by the one or more processors, to cause the one or more processors to perform the method described in any one of Examples 1 to 10. The above limitations are advantageous in enabling a system comprising one or more processors to implement and realize the advantages described with respect to Examples 1 to 10.

[0037] Example 12: A computer program product comprising one or more computer-readable storage media, and program instructions collectively stored on the one or more computer-readable storage media, the program instructions including instructions configured to cause one or more processors to perform the method described in any one of Examples 1 to 10. The above limitations are advantageous in that a computer program product configured with program instructions enables one or more processors to perform and realize the advantages described with respect to Examples 1 to 10.

[0038] Before continuing the discussion of exemplary embodiments and various aspects of the improved computer operation performed by such exemplary embodiments, it should first be understood that throughout this description, the term “mechanism” is used to refer to elements of the present invention that perform various operations, functions, and similar ones. Where the term is used herein, “mechanism” may be an implementation of a function or aspect of an exemplary embodiment in the form of an apparatus, procedure, or computer program product. In the case of a procedure, the procedure is implemented by one or more devices, apparatus, computer, data processing system, or similar. In the case of a computer program product, logic represented by computer code or instructions embodied within or on the computer program product is executed by one or more hardware devices to implement a function or perform an operation associated with a particular “mechanism.” Therefore, the mechanisms described herein may be implemented as special hardware, software executed on the hardware that configures the hardware to implement special functions of the present invention that the hardware would not otherwise be able to perform, software instructions stored on a medium that make the instructions easily executable by the hardware, thereby configuring the hardware to specifically perform the functions and specific computer operations described herein, or any combination thereof.

[0039] This specification and the claims may use the terms “one,” “at least one of,” and “one or more of” with respect to certain features and elements of exemplary embodiments. It should be understood that these terms and phrases are intended to state that at least one of certain features or elements present in a particular exemplary embodiment is present, but more than one may be present. That is, these terms / phrases are not intended to limit this specification or the claims to a single feature / element present, nor are they intended to require the presence of multiple such features / elements. On the contrary, these terms / phrases merely require at least a single feature / element, and multiple such features / elements may be present within the scope of this specification and the claims.

[0040] Furthermore, the use of the term “engine” as used herein in relation to the description of embodiments and features of the present invention is not intended to limit any particular technical implementation for achieving and / or performing any actions, stages, processes, etc. resulting from and / or performed by an engine. However, it should be understood that “engine” is limited in that even if it is implemented in computer technology and the engine can function in combination with manual input or provide output intended for manual or mental consumption, its actions, stages, processes, etc. are not performed as mental processes or through manual effort. An engine is implemented as one or more of software, dedicated hardware, and / or firmware, or any combination thereof, running on hardware and specifically configured to perform a specified function. Hardware may include, but is not limited to, the use of a processor loaded or stored in machine-readable memory and executed by a processor, thereby combining the processor with appropriate software that specifically configures the processor for a special purpose, including one or more of the functions of one or more embodiments of the present invention. Moreover, any names associated with a particular engine are for convenience of reference only and are not intended to limit to a particular implementation unless otherwise specified. In addition, any function belonging to a particular engine may be performed equally by multiple engines, incorporated into and / or combined with the functions of another engine of the same or different type, or distributed across one or more engines of various configurations.

[0041] In addition, it should be understood that the following description is intended to further illustrate the exemplary implementation of the exemplary embodiment and to aid in understanding the mechanism of the exemplary embodiment by using several different examples of various elements of the exemplary embodiment. These examples are intended to be non-limiting and do not exhaust all possible implementations of the mechanism of the exemplary embodiment. It will be apparent to those skilled in the art, in consideration of this specification, that there are many other alternative implementations of these various elements that can be used in addition to or instead of the examples provided herein without departing from the spirit and scope of the invention.

[0042] Various aspects of this disclosure are described by explanatory text, flowcharts, block diagrams of computer systems and / or block diagrams of machine logic included in computer program product (CPP) embodiments. With respect to any flowchart, operations may be performed in a different order than those shown in a given flowchart, depending on the technology involved. For example, again depending on the technology involved, two operations shown in consecutive flowchart blocks may be performed in reverse order, as a single integrated stage, simultaneously, or with at least partial time overlap.

[0043] Embodiments of a computer program product ("CPP Embodiment" or "CPP") are terms used in this disclosure to describe any set of one or more storage media (also called "mediums") that are collectively included in a set of one or more storage devices that collectively include machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A "storage device" is any tangible device that can hold and store instructions for use by a computer processor. Computer-readable storage media may be, but are not limited to, electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, mechanical storage media, or any preferred combination of those described above. Some known types of storage devices, including these media, include: diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices (such as pits / lands formed on the main surface of a punch card or disk), or any suitable combination of the foregoing. When the term "computer-readable storage medium" is used in this disclosure, it shall not be construed as storage in the form of transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides, optical pulses passing through optical fiber cables, electrical signals communicated through wires, and / or other transmitting media.As those skilled in the art will understand, data is typically moved at several intermittent points during the normal operation of a storage device, such as during access, defragmentation, or garbage collection; however, data is not transient while it is stored, and therefore, a storage device is not transient.

[0044] It should be understood that certain features of the present invention described in the context of separate embodiments for clarity may be provided in combination in a single embodiment. Conversely, various features of the present invention described in the context of a single embodiment for brevity may be provided separately or in any preferred subcombination.

[0045] The present invention may be a specially configured computing system comprising hardware and / or software that is specifically configured to implement certain mechanisms and functions described herein, a method implemented by the specially configured computing system, and / or a computer program product comprising software logic loaded into the computing system to specifically configure the computing system to implement the mechanisms and functions described herein. Whether described as a system, a method, or a computer program product, the exemplary embodiments described herein should be understood to particularly focus on improved computing tools and methodologies implemented by such improved computing tools. In particular, the improved computing tools of the exemplary embodiments specifically provide bias analysis and mitigation through protective attribute detection and evaluation classification. The improved computing tools implement a bias analysis and mitigation system that implements mechanisms and functions, such as computer natural language processing for protected attribute taxonomy, keyword detection, semantic disambiguation, large-scale language model (LLM)-based rating classification, machine learning training of rating classifiers, and training dataset modification to mitigate unwanted bias, which cannot be practically performed by humans outside of a technical environment, such as mental processes or similar, or with the assistance of such. The improved computing tools provide practical applications of the method in that, at the very least, the improved computing tools can mitigate unwanted bias in training data for machine learning training of machine learning computer models based on the identification of protected attribute mentions, the classification of ratings associated with protected attribute mentions, and the balancing of the distribution of ratings for protected attributes in the training dataset for balancing or removing unwanted bias in the training dataset.

[0046] Figure 1 is an exemplary diagram of a distributed data processing system environment in which an exemplary embodiment may be implemented and in which at least a portion of the computer code involved in carrying out the method of the present invention may be executed. That is, the computing environment 100 includes an example of an environment for executing at least a portion of the computer code associated with carrying out the method of the present invention, such as a bias analysis and mitigation system 200. In addition to the bias analysis and mitigation system 200, the computing environment 100 includes, for example, a computer 101, a wide area network (WAN) 102, an end user device (EUD) 103, a remote server 104, a public cloud 105, and a private cloud 106. In this embodiment, the computer 101 includes a processor set 110 (including processing circuits 120 and cache 121), a communication fabric 111, volatile memory 112, persistent storage 113 (including an operating system 122 and bias analysis and mitigation system 200 as identified above), a peripheral device set 114 (including a user interface (UI), a device set 123, storage 124, and an Internet of Things (IoT) sensor set 125), and a network module 115. The remote server 104 includes a remote database 130. The public cloud 105 includes a gateway 140, a cloud orchestration module 141, a host physical machine set 142, a virtual machine set 143, and a container set 144.

[0047] Computer 101 may take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch, or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device, currently known or to be developed in the future, capable of running programs, accessing networks, or querying databases such as remote database 130. As is well understood in the field of computer technology, and depending on the technology, the execution of the computer implementation method may be distributed among multiple computers and / or across multiple locations. On the other hand, in this presentation of computing environment 100, in order to keep the presentation as simple as possible, the detailed discussion focuses on a single computer, specifically computer 101. Computer 101 may be located in the cloud, although it is not shown in the cloud in Figure 1. On the other hand, computer 101 is not required to be located in the cloud, except to any extent that can be definitively shown.

[0048] The processor set 110 includes one or more computer processors of any type currently known or to be developed in the future. The processing circuitry 120 may be distributed across multiple packages, for example, multiple coordinated integrated circuit chips. The processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. The cache 121 is memory located within the processor chip package and is typically used for data or code that should be available for high-speed access by threads or cores running on the processor set 110. The cache memory is typically organized into multiple levels, depending on its relative proximity to the processing circuitry. Alternatively, some or all of the cache for the processor set may be located "off-chip". In some computing environments, the processor set 110 may operate using qubits and be designed to perform quantum computing.

[0049] Computer-readable program instructions are typically loaded onto computer 101, causing the processor set 110 of computer 101 to execute a series of operational steps, thereby enabling the computer implementation method. As a result, the instructions thus executed instantiate the method specified in the flowcharts and / or descriptions of the computer implementation method contained herein (collectively referred to as the "Method of the Invention"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by the processor set 110 to control and direct the execution of the Method of the Invention. In the computing environment 100, at least some of the instructions for implementing the Method of the Invention may be stored in the bias analysis and mitigation system 200 in persistent storage 113.

[0050] The communication fabric 111 is a signal conduction path that enables various components of the computer 101 to communicate with one another. Typically, this fabric is made up of switches and conductive paths, such as buses, bridges, physical input / output ports and similar components, and switches and conductive paths and similar components. Other types of signal communication paths, such as optical fiber communication paths and / or wireless communication paths, may be used.

[0051] The volatile memory 112 is any type of volatile memory currently known or to be developed in the future. Examples include dynamic type random access memory (RAM) or static type RAM. Typically, volatile memory is characterized by random access, but this is not required unless explicitly stated. In computer 101, the volatile memory 112 is located in a single package and resides inside computer 101, but alternatively or additionally, the volatile memory may be distributed across multiple packages and / or located externally to computer 101.

[0052] The persistent storage 113 is any form of non-volatile storage for a computer, currently known or to be developed in the future. The non-volatility of this storage means that the stored data is maintained regardless of whether power is supplied to the computer 101 and / or directly to the persistent storage 113. The persistent storage 113 may be read-only memory (ROM), but typically at least a portion of the persistent storage allows for writing, deleting, and rewriting of data. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. The operating system 122 can take multiple forms, such as various known proprietary operating systems or open-source portable operating system interface type operating systems employing a kernel. The code included in the bias analysis and mitigation system 200 specifically includes at least some of the computer code associated with implementing the method of the present invention.

[0053] The peripheral device set 114 includes a set of peripheral devices for the computer 101. Data communication connections between the computer 101's peripheral devices and other components can be implemented in various ways, including Bluetooth® connections, Near-Field Communication (NFC) connections, connections made by cables (such as universal serial bus (USB) type cables), insert-type connections (e.g., secure digital (SD) cards), connections made through local area communication networks, and even connections made through wide area networks such as the internet. In various embodiments, the UI device set 123 may include components such as a display screen, speaker, microphone, wearable devices (such as goggles and smartwatches), keyboard, mouse, printer, touchpad, game controller, and haptic device. Storage 124 is external storage such as an external hard drive, or insertable storage such as an SD card. Storage 124 may be persistent and / or volatile. In some embodiments, storage 124 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 101 is required to have a large amount of storage (for example, computer 101 locally stores and manages a large database), this storage may be provided by peripheral storage devices designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. The IoT sensor set 125 consists of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another may be a motion detector.

[0054] The network module 115 is a collection of computer software, hardware, and firmware that enables computer 101 to communicate with other computers via the WAN 102. The network module 115 may include hardware such as a modem or Wi-Fi® signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or web browser software for communicating data over the Internet. In some embodiments, the network control and network forwarding functions of the network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing Software-Defined Networking (SDN)), the control and forwarding functions of the network module 115 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for carrying out the method of the present invention can typically be downloaded to computer 101 from an external computer or external storage device via a network adapter card or network interface included in the network module 115.

[0055] WAN102 is any wide area network (e.g., the Internet) capable of communicating computer data over non-local distances using any currently known or future-developed technology for communicating computer data. In some embodiments, the WAN may be replaced and / or supplemented by a local area network (LAN), such as a Wi-Fi network, designed to communicate data between devices located in a local area. The WAN and / or LAN typically include computer hardware such as copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0056] An end-user device (EUD) 103 is any computer system used and controlled by an end-user (e.g., a customer of the company operating computer 101) and can take any of the forms discussed above in relation to computer 101. EUD 103 typically receives useful and valuable data from the operation of computer 101. For example, in a hypothetical case where computer 101 is designed to provide recommendations to an end-user, these recommendations would typically be transmitted from computer 101's network module 115 to EUD 103 via WAN 102. Thus, EUD 103 can display or otherwise present recommendations to the end-user. In some embodiments, EUD 103 may be a client device, such as a thin client, heavy client, mainframe computer, and desktop computer.

[0057] The remote server 104 is any computer system that provides at least some data and / or functionality to computer 101. The remote server 104 may be controlled and used by the same entity that operates computer 101. The remote server 104 represents a machine that collects and stores useful and valuable data for use by other computers, such as computer 101. For example, in a hypothetical case where computer 101 is designed and programmed to provide recommendations based on historical data, this historical data may be provided to computer 101 from the remote database 130 of the remote server 104.

[0058] The public cloud 105 is any computer system available for use by multiple entities, providing on-demand availability of computer system resources and / or other computing capabilities, particularly data storage (cloud storage) and computing power, without direct active management by the user. Cloud computing typically leverages resource sharing to achieve coherence and economies of scale. Direct and active management of the computing resources of the public cloud 105 is carried out by the computer hardware and / or software of the cloud orchestration module 141. The computing resources provided by the public cloud 105 are typically implemented by virtual computing environments running on various computers that make up the host physical machine set 142, which is a universe of physical computers within and / or available to the public cloud 105. The virtual computing environment (VCE) typically takes the form of virtual machines from the virtual machine set 143 and / or containers from the container set 144. These VCEs may be stored as images and are understood to be transferable either as images or after VCE instantiation, among and between various physical machine hosts. The cloud orchestration module 141 manages the transfer and storage of images, deploys new VCE instantiations, and manages the active instantiation of VCE deployments. The gateway 140 is a collection of computer software, hardware, and firmware that enables the public cloud 105 to communicate over the WAN 102.

[0059] Here, some further explanation of virtualized computing environments (VCEs) is provided. A VCE can be stored as an "image." A new active instance of a VCE can be instantiated from an image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows for the existence of multiple isolated user-space instances called containers. These isolated user-space instances typically behave like actual computers from the perspective of the programs running in them. Computer programs running on a normal operating system can utilize all the resources of that computer, including connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, programs running inside a container can only use the contents of the container and the devices allocated to the container; this feature is known as containerization.

[0060] The private cloud 106 is similar to the public cloud 105, except that its computing resources are available only for use by a single enterprise. While the private cloud 106 is shown as communicating with the WAN 102, in other embodiments, the private cloud may be completely isolated from the internet and accessible only via a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private, community, or public cloud types), often implemented by different vendors. Each of the multiple clouds remains a distinct and isolated entity, but the larger hybrid cloud architecture is bound together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the multiple configuration clouds. In this embodiment, both the public cloud 105 and the private cloud 106 are part of a larger hybrid cloud.

[0061] As shown in Figure 1, one or more computing devices, for example, a computer 101 or a remote server 104, may be specifically configured to implement the bias analysis and mitigation system 200. The configuration of the computing device may include providing application-specific hardware, firmware, or the like to facilitate the execution of the operations and generation of outputs described herein with respect to exemplary embodiments. The configuration of the computing device may similarly, or alternatively, include providing software applications stored in one or more storage devices and loaded into the memory of a computing device such as a computer 101 or a remote server 104, to cause one or more hardware processors of the computing device to run software applications that configure the processors to perform the operations and generate outputs described herein with respect to exemplary embodiments. Furthermore, any combination of application-specific hardware, firmware, software applications running on hardware, or the like may be used without departing from the spirit and scope of the exemplary embodiments.

[0062] When a computing device is configured in one of these ways, it should be understood that the computing device becomes a specialized computing device specifically configured to implement the mechanism of the exemplary embodiment, and not a general-purpose computing device. Furthermore, as will be described later, the implementation of the mechanism of the exemplary embodiment improves the functionality of the computing device and provides useful and clear results that facilitate bias analysis and the mitigation of unwanted bias with respect to protective attributes in training data for training machine learning computer models.

[0063] Figure 2 is an exemplary block diagram showing the primary operating components of a bias analysis and mitigation system according to one exemplary embodiment. The operating components shown in Figure 2 may be implemented as dedicated computer hardware components, computer software running on computer hardware configured to perform specific computer operations resulting from those components, or any combination of dedicated computer hardware and computer software-configured computer hardware. It should be understood that these operating components perform assigned operations automatically without human intervention, even if the input, such as a search query, may be provided by a human and the resulting output may assist a human. The present invention specifically concerns automatically operating computer components that improve the way in which machine learning training of machine learning computer models is performed, and provide specific solutions for implementing taxonomy-based bias analysis and mitigation, large-scale language model (LLM) evaluation classification, evaluation classification computer model training, evaluation annotation of text content associated with protected attribute mentions, modification of training datasets based on evaluation distributions for protected attributes, and machine learning training based on balanced training datasets, which cannot be practically performed by a human as a mental process and are not intended to organize any human activity.

[0064] As shown in Figure 2, the bias analysis and mitigation system 200 includes a taxonomy engine 210, a protected attribute detection and annotation engine 220, an evaluator classifier training engine 230, an evaluator classification engine 240, an evaluator distribution and balancing engine 250, and a bias mitigation engine 260. The bias analysis and mitigation system 200 operates in conjunction with a training data source computing system 270, a large-scale language model (LLM) source computing system 280, and a corpus 290 via one or more data networks 202. The corpus 290 includes multiple data sources, such as computing devices and network-attached storage, that provide natural language text documents of varying lengths and content, such as web pages and document repositories, which can be used to construct a taxonomy data structure 212 with respect to protected attributes and corresponding keywords and definitions (gross). The training data source computing system 270 provides one or more training datasets 272 for training one or more machine learning computer models, such as a Large-Scale Language Model (LLM) 282, which is available via the LLM source computing system 280. The training datasets 272 may be symmetric to bias analysis and mitigation operations performed by the bias analysis and mitigation system 200, as described below.

[0065] As previously discussed, the bias analysis and mitigation system 200 operates to implement a taxonomy of protected attributes and corresponding keywords, which is used to identify instances of such keywords in the training dataset 272. In generating the taxonomy data structure 212, manual, semi-automatic, and / or automatic generation of the taxonomy data structure 212 is implemented by the taxonomy engine 210. In some exemplary embodiments, the taxonomy engine 210 obtains an initial set of protected attributes and corresponding keywords and definitions (gross), which may be provided through manual input, an existing repository of taxonomy information, or through the operation of the LLM to generate a list and definitions of such protected attribute keywords. For example, in some exemplary embodiments, the LLM may be prompted to generate a taxonomy from a corpus 290 of keyword terms and definitions about a given protected attribute, e.g., ethnicity, nationality, gender, etc. This may be done for multiple different protected attributes. Therefore, for each protected attribute, the taxonomy engine 210 generates a corresponding list of keywords and definitions (gross). This initial taxonomy can then be evaluated by the taxonomy engine 210 to determine which keywords / definitions should be maintained in the taxonomy data structure 212.

[0066] Figure 3 is an exemplary diagram of a taxonomy of protective attributes and corresponding keywords in one exemplary embodiment. As shown in Figure 3, protective attributes may include sex, religion, race / ethnicity, disability, place of residence, reproductive status, dietary habits, physical characteristics, socioeconomic status, etc. These are merely examples of protective attributes that may be used in the exemplary embodiment, and it should be understood that many other protective attributes may be used in addition to or instead of one or more of those shown in Figure 3. Each protective attribute has a set of keywords associated with it, which may include keywords and synonyms or related terms. For example, for the protective attribute "religion," the keyword may be "Islam" and the synonym or related term may be "Muslim." Similarly, for the protective attribute "nationality," the keyword may be "China" and the related term or synonym may be "Chinese." The terms in square brackets in Figure 3 are a list of keywords used for surface-level keyword matching, which may include terms that may potentially indicate a protective attribute. In the example, only one term is shown for each protection attribute, but this can be easily expanded, for example, 'female':['female'] → 'female':['female','woman','women','she','her'], etc.

[0067] Each protected attribute further includes a corresponding definition or gross. The definition or gross may include a description of the protected attribute and / or examples of how the protected attribute keyword and / or related terms are used in a sentence or part of text. In some exemplary embodiments, the gross is a simple description of the protected human attribute (e.g., 'vegan' → a person who is vegan, or 'poor' → a person who is economically disadvantaged). The goal of the gross is to help practically distinguish whether a term indicates a protected attribute as a human characteristic, for example, to distinguish between the use of 'vegan woman' and 'vegan food' or 'poor dogs' and 'poor people'. For this purpose, a simplified gross may suffice, but a more specific and detailed gross may also be desirable. The gross may be used to prompt the LLM to generate training data for disambiguation. During training, the gross may also be used to facilitate training and generalization, as it can serve as an explicit criterion for disambiguation.

[0068] Therefore, in some exemplary embodiments, the taxonomy engine 210 may retrieve documents from the corpus 290 and evaluate them via LLM to extract keywords associated with protected attributes. Word sense disambiguation (WSD) may be implemented for keywords in the corpus 290 to determine the meaning of these keywords in terms of whether they represent protected attributes because they describe a protected entity. For example, as described above, in some cases a term may be used to refer to a protected attribute, while in other cases the same term may be used to refer to an unprotected attribute. As an example, the term "vegan" can be used in a sentence such as "I started my journey as a vegan after discovering that...", which refers to a protected attribute of an entity. However, the sentence "Arepas are 100% gluten-free, vegan, and handmade" uses the term "vegan" in the sense of an unprotected attribute. Therefore, WSD may be used to determine and distinguish between the protected and unprotected use of a given term. Such a WSD can be used to collect examples of keyword definitions in a taxonomy by selecting portions of text that target protected attribute references.

[0069] In addition to using LLM to generate the initial taxonomy, in some exemplary embodiments the initial taxonomy may be generated manually. In some exemplary embodiments a combination of manual and automated generation of the initial taxonomy may be implemented. In some exemplary embodiments a taxonomy repository may be used to select a portion of the taxonomies that target protected attributes, which may be retrieved and used for the initial taxonomy.

[0070] Regardless of the method used to generate the initial taxonomy, the taxonomy engine 210 may rate the keywords in the initial taxonomy to determine which keywords to retain in the final protected attribute taxonomy data structure 212. The rate may include a rate of bias and / or evaluation frequency. The term “evaluation” refers to whether the bias is positive, negative, or neutral, and describes how a particular entity mentioned is “evaluated” in the corresponding text. The rate of bias frequency involves generating a frequency bias score for each keyword in the initial taxonomy, where the frequency bias score is the ratio of the probability of occurrence of a word in combination with an attribute to the mean of these probabilities across all attributes, i.e., p(w|a) / E a [p(w|a)], where A represents an attribute class (e.g., gender), a∈A is an attribute (e.g., female), w is a word in the keyword vocabulary V in the initial taxonomy, e.g., a vocabulary of common words with rare words removed, for example, p(w|a) is the probability that word w appears in a sentence containing attribute a, E a [p(w|a)] is the average of this probability for all a in A. Therefore, p(w|a) / E a A higher score for [p(w|a)] indicates that w is more likely to co-occur with a compared to other protected attributes in A. To calculate these scores from the corpus, the mechanism of the exemplary embodiment can randomly select (or use all, if possible) sentences containing attribute a in the corpus by applying the protected attribute detection mechanism of the exemplary embodiment.

[0071] The taxonomy engine 210 obtains a list of keywords having frequency bias scores equal to or higher than a given threshold. Figure 4A is an exemplary figure showing a frequency bias analysis of keywords associated with exemplary protected attributes of ethnicity, according to one exemplary embodiment. As shown in Figure 4A, various attributes of a given attribute class (race / ethnicity) (e.g., white, black, Asian, Hispanic) are shown along with their corresponding keywords and their frequency bias scores, arranged in descending order. Thus, the initial taxonomy may have more keywords referring to protected attributes that can be scored using frequency bias scores, and these keywords shown in Figure 4A may be selected as the highest-ranking keyword terms to be included in the protected attribute taxonomy data structure 212.

[0072] In some exemplary embodiments, once the evaluation classification engine 240 is trained, it can be used to determine the evaluation classification of keyword usage in an initial taxonomy. This evaluation classification can be used to calculate an evaluation plus frequency bias score for each keyword, where the evaluation plus frequency bias score is a combination of the frequency bias score and the evaluation bias score for evaluation (positive, negative, neutral). Similar to when using only the frequency bias score, a list of keywords with high evaluation plus frequency bias scores, i.e., scores above a given threshold, can be generated for each attribute. Furthermore, such lists can be generated for each evaluation classification, e.g., positive, negative, and neutral. For example, in addition to frequency bias, the evaluation plus frequency bias score is of type min(p(w|a) / E a [p(w|a)], p(r|w,a) / E r [p(r|w,a)]) can be such that r∈R={positive, negative, neutral} represents the evaluation of attribute a in a sentence, and p(r|w,a) is the probability of evaluation r when word w and attribute a appear simultaneously, E r[p(r|w,a)] is the average of this probability over all r in R. Therefore, p(r|w,a) / E r A higher score for [p(r|w,a)]) indicates that the word w and attribute a tend to co-occur with the evaluation r. To calculate these scores from the corpus, the exemplary embodiment can apply the exemplary embodiment's evaluation classification to obtain the evaluation label or annotation r in each sentence in which the word w and attribute a co-occur. While the use of frequency bias-based analysis is often insufficient, evaluation plus frequency bias can more intuitively illustrate positive / negative stereotypes.

[0073] Figure 4B is an exemplary diagram of a rating plus frequency bias analysis for positive ratings and keywords associated with an exemplary ethnic protective attribute, according to one exemplary embodiment. Figure 4C is an exemplary diagram of a rating plus frequency bias analysis for negative ratings and keywords associated with an exemplary ethnic protective attribute, according to one exemplary embodiment. These lists, like the list in Figure 4A, can be used to determine which keywords to retain in the protective attribute taxonomy data structure 212. Whether using frequency bias scores alone or rating plus frequency bias scores, the taxonomy engine 210 generates a protective attribute taxonomy data structure 212 containing keywords and definitions (gross) that specifically target the entity's protective attribute and have the highest ranking score, meaning they best represent the text that is likely to be biased with respect to the protective attribute.

[0074] The protected attribute taxonomy data structure 212 may be provided to the protected attribute detection and annotation engine 220, which may use the protected attribute taxonomy data structure 212 to perform taxonomy matching on the content of one or more corpora 290 by the taxonomy matching engine 222. This matching will result in a set of text portions, such as sentences, that have instances of keywords in the taxonomy. These instances may then be evaluated by the word ambiguity resolution (WSD) engine 224 to determine whether the instances actually refer to an entity's protected attribute or whether they are unprotected attribute references. In other words, with respect to protected attribute detection, the semantic ambiguation engine 224 is implemented to accurately detect various attributes, such as "female," "black," "white," "Japanese," "American," and "blind," and to remove false positives, i.e., non-protected attribute references. For example, "Disabled drivers make up about 6% of all licensed drivers in the UK" is a protected attribute reference of the term "disabled," while "Poll Options are limited because JavaScript is disabled in the browser" is a non-protected attribute reference of the term "disabled." WSD may be used to generate keyword-gross pairs, where keyword is the term to be deambiguated, obtained from the protected attribute taxonomy data structure 212, and gross is the deambiguated text in the case of a protected attribute reference identified by WSD, such as a candidate semantic definition of the keyword.

[0075] In some exemplary embodiments, the LLM performs this WSD and operates to identify these keyword-gross pairs based on the protected attribute taxonomy 212. For example, the following type of prompt may be entered into the LLM: instruction: Does the [attribute_keyword] in the input text represent *person* or *people*{attribute_gloss}? Focus on the term "{attribute_gloss}". Briefly explain its usage in the text, then answer with yes, no, or unknown. - Answer yes if there is a clear reference to a person or human(*humans*){attribute_gloss}. - If the reference is to a *non-human*, such as an animal, object, or event, answer no. -Otherwise, for example, if the reference is unclear, indirect, or implied, answer "unknown". Please conclude your response with "Therefore, the answer is...". input: {passage} response: This prompt, or other similar types of prompts, may be implemented for each protected attribute reference identified in one or more corpora 290 identified by the taxonomy matching engine 222 based on the protected attribute taxonomy data structure 212, where {passage} is the portion of the text in which the instance of the protected attribute reference was identified. Thus, for each identified instance, the LLM will return a response indicating whether the instance refers to a person, does not refer to a person, or cannot be determined. In the latter case, depending on the desired implementation, these instances may be considered either a reference to a person or a reference to a non-person, depending on whether the implementation should be more sensitive to protected attribute references in natural language text.

[0076] As a result, instances of keywords from the protected attribute taxonomy data structure 212 in corpus 290 that specifically refer to protected attributes can be identified, in contrast to instances that refer to non-protected attributes, i.e., instances that refer to entities other than people. Instances that refer to protected attributes may be compiled into training data, which is then rated by an LLM-based rating classification engine 226 to determine an evaluation classification, e.g., negative, neutral, or positive evaluation for each instance. Similar to the prompt above, the LLM-based rating classification engine 226 may be provided with a prompt that specifies that the LLM should determine the evaluation classification (positive, negative, or neutral) of a given instance of a protected attribute keyword reference in a text sample from corpus 290. The LLM may return an instance-by-instance classification of whether it represents a positive, negative, or neutral evaluation of a particular protected attribute of an entity, e.g., a person / human. The results of the LLM may be used by a rating annotation engine 228 to annotate instances in text.

[0077] Annotated instances may be compiled into training data for training the evaluation classifier engine 240. That is, the training data includes instances of keywords from the protected attribute taxonomy data structure 212 in corpus 290, which have been determined by the semantic ambiguation disambiguation engine 224 to refer to a person or human being, and have been annotated with evaluation annotations specifying whether the reference to the protected attribute by the keyword instance is an affirmative, negative, or neutral evaluation of the protected attribute of a person or human being. This training data is provided to the evaluation classifier training engine 230 to train the evaluation classification engine 240 to perform evaluation classification on subsequent texts processed by the trained evaluation classification engine 240.

[0078] The evaluation classifier training engine 230 performs machine learning training on the evaluation classification engine 240 using training data generated by the protection attribute detection and annotation engine 240. This evaluation classification engine 240 is a smaller, finely tuned machine learning computer model, in contrast to the large-scale LLM used for the LLM-based evaluation classification engine 226. Therefore, the trained classification engine 240 requires fewer resources and computation time than the large-scale LLM to perform evaluation classification on subsequently received text. The training data trains the evaluation classification engine 240 by causing it to appropriately classify training data samples so that the evaluation classification corresponds to the evaluation annotations generated by the evaluation annotation engine 228. The machine learning training involves multiple iterations or epochs of inputting training data into the classification computer model of the evaluation classification engine 240, generating outputs, comparing the outputs with the evaluation annotations associated with the inputs, and determining the loss. Subsequently, machine learning logic is used to make adjustments to the classification computer model to reduce this loss between the output of the classification computer model and the evaluation annotations. This process is repeated until a convergence criterion is reached, for example, when a predetermined number of iterations occur or when a predetermined threshold level of accuracy is achieved.

[0079] Once trained in this manner, the evaluation classification engine 240 can be used in place of the LLM-based evaluation classification engine 226 to perform evaluation classification on subsequent text inputs, such as training data samples from the training dataset 272. That is, during runtime operation, after the evaluation classification engine 240 has been trained, the protected attribute taxonomy data structure 212, along with its corresponding keywords and definitions, can be used again to identify instances of keywords in text, such as the text of the training data samples in the training data 272 for training a computer model, for example, a language model (LM) or a large-scale language model (LLM) 282 in the LLM source computing system 280. Keyword instances can again be processed using the WSD and WSD engine 224 to distinguish between instances that are protected attribute references and instances that are not protected attribute references. For protected attribute references, the trained evaluation classification engine 240 can then evaluate each protected attribute reference instance to classify whether it represents a positive, negative, or neutral evaluation of the protected attribute. Instances can also be annotated with evaluation annotations by the evaluation annotation engine 228. Thus, each reference to a protected attribute in the training dataset 272 can be identified and annotated with its evaluation of the entity's protected attribute, e.g., positive, negative, or neutral.

[0080] After annotating the training dataset 272 with evaluation annotations, the evaluation distribution and equilibration engine 250 may generate a distribution of evaluations for each protected attribute in the protected attribute taxonomy data structure 212. This distribution can then be input to and used by the evaluation distribution and equilibration engine 250 to determine any modifications or changes that should be made to the training dataset to mitigate undesirable biases or evaluations in the training dataset with respect to the protected attributes. To achieve equilibrium in evaluations for a particular protected attribute, modifications may be implemented, for example, by identifying evaluation annotations in the training dataset that correspond to unwanted biases and downsampling the portion of the training dataset that has such evaluation annotations, or by removing training data samples that have negative evaluations for the protected attribute. This may be done over a subset or all of such protected attributes. Actual downsampling or other modifications to the training dataset may be performed by a bias mitigation engine 260 that implements the modifications and generates an equilibrated training dataset 262.

[0081] The balanced training dataset 262 may be stored in the training data source computing system 270 as an alternative or additional training dataset that can be used to train a machine learning computer model such as LLM282. In addition, the balanced training dataset 262 may be provided to the machine learning training logic of the LLM source computing system 280 (not shown) to train a machine learning computer model, e.g., LLM282, to perform the corresponding task. For example, this balanced training dataset 262 may be used to train LLM282 without introducing unnecessary bias into the trained LLM282. Thus, training of a machine learning computer model, e.g., LLM282, is improved by removing unnecessary bias in the training dataset 272, which generates the balanced training dataset 262 used to train the machine learning computer model, resulting in a trained machine learning computer model with minimized or removed unnecessary bias in its behavior with respect to protected attributes.

[0082] As discussed above, the exemplary embodiment has instances of keywords from the Protected Attribute Taxonomy Data Structure 212 and annotates text that, through the WSD, is determined to refer to an entity's protected attribute. The annotation is an evaluation annotation indicating whether the instance has a positive, negative, or neutral evaluation of the protected attribute. This results in different sets of text for each of the different evaluation classifications, i.e., positive, negative, or neutral. Figure 5 is an exemplary diagram of examples of positive, neutral, and negative evaluations of a protected attribute by one exemplary embodiment. As can be seen in Figure 5, the examples include portions of text that can be compiled into a training dataset for training the evaluation classification engine 240. The first portion 510 includes sentences in which the protected attribute keyword "female" exists and the evaluation annotation is positive. The second portion 520 includes sentences in which the protected attribute keyword "female" exists and the evaluation annotation is neutral. The third portion 530 includes sentences in which the protected attribute keyword "female" exists and the evaluation annotation is negative. By training the evaluation classification engine 240 using such examples, the evaluation classification engine 240 is trained to distinguish between positive, neutral, and negative evaluation sentences, and this learning is applied to new texts such as training dataset samples. The distribution of such evaluation classifications is then determined for the training dataset and any modifications made to balance the training dataset, and negative evaluations in the training dataset are minimized by using downsampling, or unnecessary biases in the training dataset are otherwise minimized or removed.

[0083] Figures 6 and 7 are flowcharts outlining exemplary operations for training an evaluation classifier engine and performing bias analysis and mitigation, according to one or more exemplary embodiments. It should be understood that the operations outlined in Figures 6 and 7 are specifically performed automatically by improved computer tools of the exemplary embodiments, and that they are not intended, and cannot be practically performed, by humans, either as a mental process or by organizing human activities. In contrast, humans may, in some cases, initiate the execution of the operations described in Figures 6 and 7, and in some cases utilize the results generated as a result of the operations described in Figures 6 and 7, but the operations in Figures 6 and 7 themselves are specifically performed in an automated manner by improved computing tools.

[0084] As shown in Figure 6, the operation begins in the stage of obtaining or constructing a protective attribute taxonomy (stage 610). The protective attribute taxonomy includes keywords and definitions for one or more protective attributes. The protective attribute taxonomy may be a refinement of the initial taxonomy by calculating and filtering the initial taxonomy based on, for example, the frequency of bias indicators, the evaluation of bias indicators plus frequency bias, etc.

[0085] A protected attribute taxonomy is used to perform taxonomic matching of the text clauses with one or more corpora to identify instances of keywords present in the text clauses (step 620). Semantic ambiguity is performed on the instances to distinguish protected attribute references from unprotected attribute references (step 630). The resulting protected attribute reference instances are input into an LLM-based evaluation classification engine to determine the corresponding evaluation classification for each instance of a protected attribute reference, e.g., positive, negative, or neutral (step 640). The instances are annotated with evaluation annotations based on the evaluation classification and compiled into a training dataset for training the evaluation classification engine, i.e., a specifically trained classifier computer model for performing evaluation classification of the input text sentences (step 650).

[0086] Subsequently, the evaluation classifier computer model of the evaluation classification engine is trained using machine learning based on the training dataset (step 660). The resulting trained evaluation classifier computer model is then deployed for runtime operation on the training dataset to perform evaluation classification without unnecessary bias (step 670). The operation then terminates.

[0087] With respect to Figure 7, the operation begins by receiving a training dataset to be rated for unnecessary bias (step 710). The training data sample is matched with keywords in the protected attribute taxonomy to identify instances of keywords present in text clauses (step 720). Semantic ambiguity is performed on the instances to distinguish protected attribute references from non-protected attribute references (step 730). The resulting protected attribute reference instances are input into a trained rating classification engine to determine the corresponding rating classification for each instance of a protected attribute reference, e.g., positive, negative, or neutral (step 740). The instances are annotated with rating annotations based on the rating classification (step 750).

[0088] A distribution of evaluation classifications for training data samples in the training dataset is generated (step 760). Then, unnecessary biases are removed from the training dataset (step 770), and equilibration and bias reduction logic is performed based on the generated distribution to achieve the desired equilibrium of positive, negative, and neutral instances of protected attribute mentions in the training dataset. In some cases, this may involve performing downsampling of instances of negative evaluations in the training dataset. The resulting equilibrated training dataset is then stored for use in training a machine learning computer model and / or provided to the machine learning training logic to train the machine learning computer model (step 780). The operation then ends.

[0089] The description of the present invention is presented for illustrative and explanatory purposes and is not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications and variations will become apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The embodiments have been selected and described to best illustrate the principles and practical applications of the invention and to enable those skilled in the art to understand the invention in various embodiments with various modifications to suit specific uses intended. The terminology used herein has been selected to best illustrate the principles, practical applications, or technological improvements over existing technologies, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for accurately detecting various protective attributes and classifying / balancing the evaluations for each attribute, wherein the method is: The step of obtaining the taxonomy of protected attributes that have associated keywords; A step of performing keyword matching operations to find instances of the keyword associated with the protection attribute in the first dataset; The step involves performing an evaluation classification on each instance of the aforementioned keyword to classify the evaluation for the corresponding protection attribute, where the evaluation classification is one of positive evaluation, negative evaluation, or neutral evaluation; The step of generating an annotated dataset by annotating each instance with an evaluation annotation corresponding to the aforementioned evaluation classification for that instance; and A second step involves performing bias analysis and balancing the evaluation classification distribution of the dataset to mitigate undesirable biases based on the evaluation-annotated dataset. A method that includes [a certain feature].

2. The method according to claim 1, wherein the second dataset is a training dataset for training a machine learning computer model.

3. The method according to claim 2, wherein the machine learning computer model is a large-scale language model (LLM).

4. The method according to claim 2 or 3, wherein the step of performing the bias analysis and evaluation classification balancing includes the step of downsampling training data samples in the training dataset that have evaluation annotations indicating a negative evaluation.

5. The method according to claim 2 or 3, wherein the step of performing the bias analysis and evaluation classification equilibration includes the step of equilibrating the training dataset so that the number of first training data samples having evaluation annotations indicating a positive evaluation and the number of second training data samples having evaluation annotations indicating a negative evaluation are approximately the same.

6. The method according to any one of claims 1 to 3, further comprising the step of performing semantic ambiguity resolution on the instances of the keyword in the first dataset to distinguish between first instances in which the keyword refers to the protection attribute corresponding to the keyword and second instances in which the keyword does not refer to the protection attribute corresponding to the keyword, wherein the second instances are deleted as instances relating to the evaluation classification and evaluation annotation.

7. The method according to any one of claims 1 to 3, further comprising the step of performing machine learning training on an evaluation annotation classification machine learning computer model based on the annotated instances in the evaluation annotation dataset, thereby training the evaluation annotation classification machine learning computer model to classify input text to evaluation classification, wherein the evaluation classification is one of positive evaluation, negative evaluation, or neutral evaluation.

8. The method according to any one of claims 1 to 3, wherein the evaluation classification performed for each instance of the keyword in order to classify the evaluation for the corresponding protection attribute is performed by prompting a Large Language Model (LLM) to specify an evaluation classification for the input text sentence.

9. The steps involved in obtaining the aforementioned taxonomy are: The stage of obtaining the initial taxonomy of protection attributes and keywords; The step of assessing the frequency of bias with respect to each keyword of each protective attribute in the aforementioned initial taxonomy; and A step of filtering the initial taxonomy based on the frequency of the bias to generate the taxonomy of the protected attribute. The method according to any one of claims 1 to 3, including the method described in any one of claims 1 to 3.

10. The method according to claim 9, further comprising a step of evaluating the evaluation classification for each keyword of each protective attribute in the initial taxonomy, wherein the filtering is based on an evaluation plus frequency bias index.

11. One or more computer-readable storage media; and Procedure for obtaining the taxonomy of a protected attribute that has associated keywords; A procedure for performing keyword matching operations to detect instances of the keyword associated with a protection attribute in a first dataset; A procedure for performing an evaluation classification on each instance of the aforementioned keyword to classify the evaluation for the corresponding protection attribute, wherein the evaluation classification is one of positive evaluation, negative evaluation, or neutral evaluation; The procedure for generating an annotated dataset by annotating each instance with an evaluation annotation corresponding to the evaluation classification for that instance; and A procedure to mitigate undesirable biases based on the annotated evaluation dataset by performing bias analysis and balancing of the evaluation classification distribution of the second dataset. Program instructions stored on one or more computer-readable storage media to perform operations including the above A computer program that includes [a specific feature / feature].

12. The computer program according to claim 11, wherein the second dataset is a training dataset for training a machine learning computer model.

13. The computer program according to claim 12, wherein the machine learning computer model is a large-scale language model (LLM).

14. The computer program according to claim 12 or 13, wherein the procedure for performing the bias analysis and evaluation classification equilibration includes a procedure for downsampling training data samples in the training dataset that have evaluation annotations indicating a negative evaluation.

15. The computer program according to claim 12 or 13, wherein the procedure for performing the bias analysis and evaluation classification equilibration includes a procedure for equilibrating the training dataset such that the number of first training data samples having evaluation annotations indicating a positive evaluation and the number of second training data samples having evaluation annotations indicating a negative evaluation are approximately equal.

16. The computer program according to any one of claims 11 to 13, further comprising a step of performing semantic ambiguity on the instances of the keyword in the first dataset to distinguish between a first instance in which the keyword refers to the protection attribute corresponding to the keyword and a second instance in which the keyword does not refer to the protection attribute corresponding to the keyword, wherein the second instance is deleted as an instance relating to the evaluation classification and evaluation annotation.

17. The computer program according to any one of claims 11 to 13, further comprising the steps of performing machine learning training on an evaluation annotation classification machine learning computer model based on the annotated instances in the evaluation annotation dataset, thereby training the evaluation annotation classification machine learning computer model to classify input text to evaluation classification, wherein the evaluation classification is one of positive evaluation, negative evaluation, or neutral evaluation.

18. The computer program according to any one of claims 11 to 13, wherein the evaluation classification performed for each instance of the keyword in order to classify evaluations for corresponding protection attributes is performed by prompting a Large Language Model (LLM) to specify an evaluation classification for an input text statement.

19. The procedure for obtaining the aforementioned taxonomy is: Procedure for obtaining the initial taxonomy of protection attributes and keywords; A procedure for assessing the frequency of bias for each keyword of each protective attribute in the aforementioned initial taxonomy; and A procedure for filtering the initial taxonomy based on the frequency of the bias to generate the taxonomy of the protected attribute. A computer program according to any one of claims 11 to 13, including the computer program described in any one of claims 11 to 13.

20. Processor set; One or more computer-readable storage media; and Procedure for obtaining the taxonomy of a protected attribute that has associated keywords; A procedure for performing keyword matching operations to detect instances of the keyword associated with a protection attribute in a first dataset; A procedure for performing an evaluation classification on each instance of the aforementioned keyword to classify the evaluation for the corresponding protection attribute, wherein the evaluation classification is one of positive evaluation, negative evaluation, or neutral evaluation; The procedure for generating an annotated dataset by annotating each instance with an evaluation annotation corresponding to the evaluation classification for that instance; and A procedure to mitigate undesirable biases based on the annotated evaluation dataset by performing bias analysis and balancing of the evaluation classification distribution of the second dataset. Program instructions stored on the one or more computer-readable storage media to cause the processor set to perform operations including the above A computer system equipped with the following features.

Images