Automatic identification of quality deviations in machine learning models

Abstract

A method for identifying quality deviations in machine learning applications, the method being implemented by a corresponding system and computer readable medium, may include training (205) a first machine learning application (MLA) (212) based on a world truth normal form (202) to generate a world truth prediction model (216); tagging (209) the intended use training data (224) and the field data (228) via a first MLA using a world truth prediction model; training (211) a second machine learning application (MLA) (240) using the marked intended use training data to generate an intended use model (246); generating (219) at least one prediction (256) having an associated confidence level via the trained second MLA using the tagged field data and the expected use model; and determining at least one quality deviation of the expected use model from the world truth normal form (258), or determining at least one quality deviation of the at least one predicted from the world truth normal form.

Description

Technical Field

[0001] This disclosure generally relates to methods and systems for training machine learning applications and for identifying quality biases in machine learning models. Background Technology

[0002] Machine learning models are developed, trained, and used on biased data, which may not represent the true distribution of the characteristics the data represents. Biased data may unintentionally include noise or may contain overrepresented or underrepresented characteristics that the machine learning model is being trained to predict. Furthermore, datasets may be characterized with too many unknowns, meaning it's difficult to know which data points are missing. Training machine learning models with such data can cause them to exhibit unintended deviations or quality biases. Machine learning models require manual validation to determine the quality bias between what any given model predicts, is expected, and what was used in the training data. Because validating data and machine learning models requires significant human and time investment, improvements to the models are delayed, resulting in lower predictive accuracy. Improvements to methods for validating data and machine learning models are desired. Summary of the Invention

[0003] The disclosed embodiments include a method for automatically identifying quality biases in machine learning models. A first machine learning application (MLA) is trained based on a world truth paradigm to generate a world truth prediction model. The world truth prediction model is used to label intended-use training data and field data via the first MLA. A second machine learning application is trained using the labeled intended-use training data to generate an intended-use model. The method includes generating at least one prediction with an associated confidence level using the labeled field data and the intended-use model in the second MLA. The method includes determining at least one quality bias between the intended-use model and the world truth paradigm, or determining at least one quality bias between at least one prediction and the world truth paradigm.

[0004] In various implementations, the method includes defining at least one property that defines a world truth paradigm, wherein the at least one property defines a distribution of multiple property values.

[0005] In various implementations, the method includes labeling world-truth training data and world-truth test data based on a world-truth paradigm.

[0006] In various implementations, the method includes generating a world ground truth prediction model based on world ground truth training data.

[0007] In various implementations, the method includes generating a first plurality of key performance indicators (KPIs) associated with the generated world truth prediction model.

[0008] In various implementations, the method includes testing a world truth prediction model using world truth test data via a first MLA.

[0009] In various implementations, the method includes generating a first test result and generating a second set of multiple key performance indicators (KPIs) associated with the test world truth prediction model.

[0010] In various implementations, the method includes comparing a first plurality of KPIs with a second plurality of KPIs to determine whether there is a statistical difference between a first test result and the world truth paradigm.

[0011] In various implementations, the method includes retraining the world truth prediction model based on second world truth training data when there is a statistical difference between the first test result and the world truth paradigm.

[0012] In various implementations, the method includes labeling intended-use test data via a first MLA based on a world truth paradigm.

[0013] In various implementations, the method includes generating a third plurality of KPIs associated with labeled intended-use test data and intended-use training data.

[0014] In various implementations, the method includes generating a fourth plurality of KPIs associated with training a second MLA and the intended use model.

[0015] In various embodiments, the method includes testing the intended use model using labeled intended use test data via a second MLA.

[0016] In various implementations, the method includes generating a second test result and a fifth plurality of KPIs associated with the model intended for the test.

[0017] In various implementations, determining at least one quality deviation includes determining whether the training data violates a world truth threshold.

[0018] In various implementations, determining at least one quality deviation includes determining whether at least one of a first plurality of KPIs, a second plurality of KPIs, a third plurality of KPIs, a fourth plurality of KPIs, and a fifth plurality of KPIs violates a world truth threshold.

[0019] In various embodiments, the method includes determining at least one quality deviation by comparing at least one prediction against a third, fourth, and fifth plurality of KPIs to identify a difference between at least one prediction and the expected outcome represented by at least one of the third, fourth, or fifth plurality of KPIs.

[0020] In various embodiments, the method includes generating a deviation notification based on a determination of whether a world truth threshold has been violated, the deviation notification being configured to be displayed on a display. The method may include the step of reusing new training or testing data to train or test a first MLA or a second MLA when the world truth threshold is violated.

[0021] In various embodiments, the method includes determining whether the confidence level of at least one prediction exceeds a confidence level threshold. The method may include generating a notification on a display device, the notification including at least one prediction and whether it exceeds the confidence level threshold.

[0022] In various embodiments, the method includes determining whether the confidence level of at least one prediction exceeds a confidence level threshold, and generating a notification on a display device. The notification includes at least one prediction, at least one quality deviation, whether the confidence level threshold is exceeded, and whether the world truth threshold is exceeded.

[0023] In various embodiments, the method includes causing the automatic intended use system to perform or stop performing intended use operations.

[0024] The foregoing has provided a fairly broad overview of the features and technical advantages of this disclosure, enabling those skilled in the art to better understand the following detailed description. Additional features and advantages of this disclosure will be described below, forming the subject matter of the claims. Those skilled in the art will understand that they can readily use the disclosed concepts and specific embodiments as the basis for modifying or designing other structures for practicing the same purposes of this disclosure. Those skilled in the art will also recognize that such equivalent constructions do not depart from the spirit and scope of the broadest form of this disclosure.

[0025] Before proceeding with the detailed description below, it may be advantageous to set forth the definitions of certain words or phrases used throughout this patent document: the terms “comprising” and “including” and their derivatives mean including but not limited to; the term “or” is inclusive, meaning and / or; the phrases “associated with” and “associated with” and their derivatives may mean including, being included, interconnected with, containing, being contained, connected to or connected to, coupled to or coupled to, communicable with, cooperating with, interleaving, juxtaposing, proximate, bound to or bound to, having, having the properties of, etc.; and the term “controller” means any means, system or part thereof that controls at least one operation, whether such means is implemented in hardware, firmware, software or some combination of at least two of them. It should be noted that the functionality associated with any particular controller can be centralized or distributed, whether local or remote. Definitions of certain words and phrases are provided throughout this patent document, and those skilled in the art will understand that such definitions apply, in many instances (if not most instances), to the prior and future use of such defined words and phrases. Although some terms may include a wide variety of implementations, the appended claims may expressly limit these terms to the specific implementations. Attached Figure Description

[0026] To gain a more complete understanding of this disclosure and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, wherein the same numerals designate the same objects, and in the drawings: Figure 1 The diagram illustrates a block diagram of a data processing system that can implement the proposed implementation. Figures 2A to 2C The diagram illustrates the process of a method for identifying quality biases in machine learning models; and Figure 3 The diagram illustrates a flowchart of another method for identifying quality biases in machine learning models. Detailed Implementation

[0027] The following discussion Figures 1 to 3 The various embodiments used to describe the principles of this disclosure in this patent document are for illustrative purposes only and should not be construed as limiting the scope of this disclosure in any way. Those skilled in the art will understand that the principles of this disclosure can be implemented in any suitably arranged apparatus. Numerous innovative teachings of this application will be described with reference to exemplary, non-limiting embodiments.

[0028] To address the need for efficient and rapid identification of quality biases in machine learning models, this disclosure provides a method and system for determining quality biases in machine learning models using the world truth paradigm.

[0029] Today, model administrators manually assess data quality against a data reference framework to identify quality biases in the data or machine learning models. Model administrators need to run tests on specific datasets to determine if any given number of problems exist in the machine learning model, the data used to train the model, or the data used to test the model. Problems include data deviation, data with excessive noise, or pattern drift. Data deviation occurs when certain data attributes are underrepresented, overrepresented, or not represented at all. Data with excessive noise can unintentionally affect the training or testing of a machine learning model. Pattern drift occurs when new data processed by the machine learning model differs from the data used to train the model. When a problem is identified, the model is retrained to address or resolve the identified problem.

[0030] This disclosure enables data processing system or model administrators to quickly and efficiently identify quality deviations by providing a reference framework carried over through the training, testing, and real-world implementation of a machine learning model. The reference framework is a world truth paradigm. The method includes: receiving a world truth paradigm; training a first machine learning application to generate a world truth paradigm prediction model that can label data according to the world truth paradigm; and using the world truth prediction model to label additional data for training, testing, and implementing a second machine learning application according to the world truth paradigm.

[0031] The method disclosed herein enables model administrators and machine learning applications to more quickly diagnose and correct unexpected deviations and biases in machine learning models without human intervention by automatically and instantly detecting unexpected deviations in training, testing, and field data. By carrying data labeled according to the world truth paradigm and comparing the labeled data with the world truth paradigm, data used to generate or test machine learning models can be selected more carefully for more accurate modeling. Furthermore, the speed of detecting deviations and their causes is significantly enhanced because such diagnosis and detection do not need to be performed solely by the model administrator. More specifically, by automatically labeling data based on the world truth paradigm as described herein, previously unknown data can be identified quickly and efficiently, and more known unknown data can be selected to retrain the machine learning model to eliminate or mitigate biases learned in the machine learning model. The disclosed process offers significant technical advantages, drastically reducing the time required to identify biased data (e.g., overrepresented or underrepresented data) and the time required to identify, request, or collect the training and testing data needed to overcome unexpected deviations. Simultaneously, this further reduces the time required to retrain machine learning applications to generate more accurate machine learning models, which can result in faster and more accurate application of machine learning models.

[0032] As used in this paper, the phrase “world truth paradigm” describes a set of properties and parameters and their distributions that include a set of metadata specifying expectations for the quality and quantity of data. In other words, the “world truth paradigm” is defined as reflecting the properties and parameters and their distributions expected to be found in any given dataset (e.g., training data, test data, or field data).

[0033] "World Truth Prediction" is a prediction of the characteristics, parameters, and distributions of a dataset based on the "World Truth Paradigm" and compared to it, using a model generated by a machine learning application. In other words, "World Truth Prediction" is a dataset labeled based on the "World Truth Paradigm." For example, the "World Truth Paradigm" might specify a distribution of 25% squares, 25% circles, 25% rhombuses, and 25% triangles, and the "World Truth Prediction" for the data in the dataset could be 23% squares, 27% circles, 10% rhombuses, and 35% triangles.

[0034] As used herein, the phrase “intended use” describes the relationship to a machine learning model developed for a specific intended use, such as detecting anomalies in data, detecting potential medical abnormalities (e.g., detecting cancer in an image of a body part), or determining the shape or face in an image or video. Thus, for example, “intended use training data” refers to training data representing the field data that a machine learning model will process when making predictions for any intended use. Similarly, for example, “intended use test data” refers to data representing field data used to test a model generated by a machine learning application. In another example, the phrase “intended use field data” refers to field or real-world data processed by a machine learning model to make predictions for an intended use (e.g., predicting the presence of shape or defects when manufacturing a product).

[0035] Figure 1 A block diagram of a data processing system 100 in which embodiments can be implemented is illustrated, for example, the data processing system being implemented as a computer system particularly configured by software or otherwise to execute processes as described herein, and particularly as each of a plurality of interconnect and communication systems as described herein. The depicted data processing system 100 includes a processor 102 connected to a secondary cache / bridge 104, which in turn is connected to a local system bus 106. The local system bus 106 may be, for example, a peripheral component interconnect (PCI) architecture bus. In the depicted example, main memory 108 and a graphics adapter 110 are also connected to the local system bus 106. The graphics adapter 110 may be connected to a display 111.

[0036] Other peripheral devices, such as LAN / WAN / wireless (e.g., WiFi) adapter 112, can also be connected to the local system bus 106. An expansion bus interface 114 connects the local system bus 106 to the input / output (I / O) bus 116. The I / O bus 116 connects to a keyboard / mouse adapter 118, a disk controller 120, and an I / O adapter 122. The disk controller 120 can be connected to a storage device 126, which can be any suitable machine-usable or machine-readable storage medium, including but not limited to: non-volatile, hard-coded media (such as read-only memory (ROM) or erasable electrically programmable read-only memory (EEPROM), magnetic tape storage devices), and user-recordable media (such as floppy disks, hard disk drives, and optical disc read-only memory (CD-ROM) or digital versatile discs (DVDs)), as well as other known optical, electrical, or magnetic storage devices.

[0037] In the example shown, audio adapter 124 is also connected to I / O bus 116, and a speaker (not shown) can be connected to the audio adapter to play sound. Keyboard / mouse adapter 118 provides connectivity for pointing devices (not shown) such as mice, trackballs, trackpointers, touchscreens, etc.

[0038] Those skilled in the art will understand that Figure 1 The hardware depicted may vary depending on the specific implementation. For example, other peripheral devices, such as optical disc drives, may be used in addition to or instead of the depicted hardware. The examples depicted are provided for illustrative purposes only and are not intended to impose architectural limitations on this disclosure.

[0039] The data processing system 100 according to embodiments of this disclosure includes an operating system employing a graphical user interface. The operating system allows multiple display windows to be simultaneously presented in the graphical user interface, wherein each display window provides an interface to a different application or a different instance of the same application. A cursor in the graphical user interface can be manipulated by a user via a clicking device. The cursor position can be changed and / or events (such as clicking a mouse button) can be generated to initiate a desired response.

[0040] With appropriate modifications, one of various commercial operating systems may be used, such as a version of Microsoft Windows™, a product of Microsoft Corporation, located in Redmond, Washington. The operating system may be modified or created based on this disclosure as described.

[0041] The LAN / WAN / wireless adapter 112 can connect to network 130 (not part of data processing system 100), which can be any public or private data processing system network or combination of networks known to those skilled in the art, including the Internet. Data processing system 100 can communicate with server system 140 via network 130, which is also not part of data processing system 100, but can be implemented as, for example, a separate data processing system 100.

[0042] In various embodiments, storage 126 may store any data and / or code necessary and / or available for performing the processes described herein. For example, storage 126 may store elements such as computer programs (e.g., machine learning application 150A, machine learning application 150b, method 200, or method 300), data 152 (e.g., training data 152A, test data 152B, or field data 152C, second training data 152d, second test data 152e, second field data 152f), and computer program models (e.g., machine learning generative models 154A or 154b). Data 152 may include software architecture, any elements thereof, or any other data, programs, code, tables, data lake repositories, key performance indicators, or other information or data used to implement any of the methods discussed below in this disclosure. In each aspect, machine learning application 150A or 150B, data 152A, 152B or 152C, method 200, method 300, or machine learning generative model 154A or 154B may each be stored separately or in other combinations on another storage device of data processing system 100 or remote data processing system (not shown). Machine learning application 150A or 150B may be machine learning application 212 or 240 as described below. Data 152 may include any data described below with respect to method 200 or 300.

[0043] refer to Figures 2A to 2C This disclosure provides a process diagram of a method 200 for identifying quality deviations in a machine learning model according to various aspects of the present disclosure. Parts of method 200 may be implemented by one or more data processing systems 100 (collectively referred to herein as "systems"), and each process described herein is executed by the system. Method 200 may be stored as executable instructions on a non-transitory computer-readable medium.

[0044] Method 200 includes a process 201 of receiving a World Truth Paradigm (WTP) 202. As used herein, “receive” includes, but is not limited to, loading from machine-readable storage, receiving from another computing system or process, or receiving via interaction with a user (e.g., as input via a keyboard, mouse, or touchscreen). The World Truth Paradigm (WTP) 202 includes one or more WTP attributes 204, each WTP attribute 204 defining a distribution of multiple attribute values. The WTP 202 may be predefined by a model administrator (e.g., someone training a machine learning application to generate a predictive model) or predefined by a system configured to define or improve the WTP based on a predefined WTP.

[0045] WTP attribute 204 is configured to be used as metadata to tag any given data (e.g., training data, test data, or field data) and to specify expectations for the quality and quantity of data used to train machine learning models. For example, for each data item in the dataset, at least one attribute is assigned a respective attribute value. Thus, WTP 202 expresses the frequency that each attribute value, or a combination of attributes and associated attribute values, should represent by the dataset.

[0046] For example, WTP attribute 204 might be "shape," and attribute values ​​could include circle, triangle, square, rhombus, or rectangle. In another example, WTP attribute 204 could be "color," and attribute values ​​could be yellow, red, and blue. WTP attribute 204 can be determined based on the desired purpose of the machine learning model associated with the data to be labeled. WTP attribute 204 can include, but is not limited to: object category, including color, shape, and type; modality (e.g., image or video); quality (e.g., black or white, color depth 1, color depth 2, or excessive noise); perspective (e.g., top, bottom, left, right, or isometric); race (e.g., African American, Caucasian, Asian, or Latino); and direction (e.g., north, south, east, west). For example, if a machine learning model is being trained to predict faces in an image, the attributes could include eyes, ears, nose, and mouth, and values ​​could be defined differently for each attribute. In another example, if a machine learning model is being trained to predict cans of various beverages on a conveyor belt, the attributes could be the type of beverage (e.g., juice, water, soda), brand name, or volume.

[0047] WTP attribute values ​​can be expressed using a binary mapping distribution scheme. For example, Table 1 illustrates an example of WTP attributes expressed using binary values.

[0048]

[0049] Table 1. Example binary mapping of example WTP attributes

[0050] Therefore, Table 1 represents the WTP status, with squares in red, triangles in yellow, and circles in blue.

[0051] In each aspect, WTP attribute values ​​can be expressed using the probability of occurrence (in percentage). For example, Table 2 illustrates an example of WTP attributes expressed using binary values.

[0052]

[0053] Table 2. Percentage of Example Attribute Value Distribution

[0054] Therefore, Table 2 expresses the WTP state, where 40% of the shapes targeted for recognition should be circles, 30% should be triangles, and 30% should be squares. While Tables 1 and 2 describe the attributes of shape and the attribute values ​​as color, those skilled in the art will understand that attributes and attribute values ​​can be extended beyond basic shape and color, and can include modality, viewpoint, quality, or other attributes that the machine learning model is trained to recognize. The defined world truth paradigm and its attributes are used in each subsequent step of method 200 to track whether the machine learning model is deviating from or moving away from any specific WTP attribute 204.

[0055] Method 200 includes a process 203 of labeling data using WTP attribute 204, resulting in WTP training data 208 and WTP test data 210. For each data item in the dataset (e.g., WTP training data 208 or WTP test data 210), a minimum or maximum number of data items can be assigned to a WTP attribute 204 or a combination of WTP attributes 204. By labeling the training and test data using WTP 202, system 100 or a model administrator can determine the correlation between the labeled data and the results or predictions of a machine learning model trained on that data, in order to identify quality biases.

[0056] Then, in process 205, labeled WTP training data 208 is used to train a first machine learning application (MLA) 212 (e.g., a module for world truth prediction), thereby generating a WTP prediction model 216 and training key performance indicators (KPIs) 214. The WTP prediction model 216 is configured to automatically predict WTP attributes 204 of unlabeled data via the first MLA 212, and then label the data with the WTP attributes 204. KPIs 214 may include metrics known to those skilled in the art, such as, but not limited to, accuracy, precision, recall, F1 score, mean squared error, area under the curve (AUC), etc.

[0057] During process 207, the WTP prediction model 216 is tested using WTP test data 210 to determine whether the WTP prediction model correctly predicts and labels the WTP attributes of the data according to WTP 202. Process 207 generates test results 220 and test key performance indicators (KPIs) 222. Test results 220 are the predicted WTP attributes 204 of the WTP test data 210. Test KPIs 222 provide an indication of whether the WTP prediction model 216 accurately predicts the WTP test data 210. The system or model administrator compares the test KPIs 222 against the training KPIs 214 to determine quality deviations in the WTP prediction model 216. When quality deviations are identified, the system or model administrator can determine which data is missing or which data is inaccurately represented compared to WTP 202 in the training or test data. The system or model administrator can then repeat processes 203, 205, and 207.

[0058] Method 200 includes process 209, in which WTP prediction model 216 is used via a first MLA 212 to label intended-use data (including training data 224 and test data 226), thereby generating labeled training data 234 and labeled test data 236. Process 209 also generates WTP labeled KPIs 232. WTP labeled KPIs 232 are configured to indicate the performance of WTP prediction model 216 in predicting WTP attributes 204 of training data 224, test data 226, and field data 228. WTP labeled KPIs 232 measure the accuracy of training data 224 and test data 226 in conforming to WTP 202.

[0059] During process 211, a second machine learning model 240 is trained using labeled training data 234 to generate a intended-use prediction model 246 and a WTP KPI 242. The intended-use prediction model 246 is configured to make predictions for an intended purpose using field data (e.g., detecting anomalies on a conveyor belt, detecting quality issues during manufacturing, or determining a medical diagnosis from medical images). WTP KPI 242 is a measure of the accuracy of the training data in conforming to WTP 202. WTP KPI 242 differs from WTP KPI 232 in that it can be a measure of all training data used to train the second machine learning model 240. For example, multiple batches of labeled training data 234 can be used as input in process 211, and WTP KPI 242 reflects the overall accuracy of all used labeled training data in conforming to WTP 202.

[0060] During process 213, the intended use predictive model 246 is tested via a second MLA 248 using labeled test data 236. Process 213 generates test results 252 and WTP KPI 250. Test results 252 are at least one prediction made by the intended use predictive model 246. WTP KPI 250 provides an indication of the presence of quality bias in the test data based on a comparison with WTP 202. WTP KPI 250 is a measure of the accuracy of the test data in conforming to WTP 202. WTP KPI 250 differs from WTP KPI 232 in that WTP KPI 250 can be a measure of all test data used to test the second machine learning model 240. For example, multiple batches of labeled test data 236 can be used as input in process 213, and WTP KPI 250 reflects the overall accuracy of all used labeled training data in conforming to WTP 202.

[0061] During process 215, WTP 202, WTP KPI 232, WTP KPI 242, and WTP KPI 250 are compared to each other to determine quality biases or data deviations in the data, thereby identifying quality biases or data deviations in the intended use prediction model 246. For example, the test WTP KPI 250 is compared to the training KPI 242 to determine which WTP attributes are expected and which are predicted, thus indicating quality biases in the intended use prediction model 246. Process 215 may also include comparing a world truth threshold 202a to the KPIs. The world truth threshold 202a is a predefined tolerance regarding the degree to which any data in the data (e.g., training data 224 or test data 226) might deviate from WTP 202. When data deviations are found to violate the world truth threshold 202a during process 215, a quality bias is determined.

[0062] When a quality deviation is detected (e.g., a deviation is detected in any data in the training or testing data), the data processing system 100 generates a notification 260 identifying the deviation data. The notification 260 may be a signal or display on the user device or on the display 111 of the data processing system 100.

[0063] Process 215 may include identifying missing or inaccurately represented data in the training or test data compared to WTP 202, and selecting new data from storage device 126 to correct, counteract, or mitigate deviations when a world truth threshold is violated. In some embodiments, the notification includes a request for another data processing system to provide missing training or test data. In some embodiments, the request may be displayed on display 111 and may identify which WTP attributes in the data are overrepresented or underrepresented. Notification 260 may include an instruction for the data processing system to repeat one or more of processes 209, 211, 213, and 215. In some embodiments, a model administrator may instruct the data processing system to repeat processes 209, 211, 213, and 215. In some embodiments, process 215 may include generating a notification that no deviation data has been identified or that the world truth threshold 202a has not been violated.

[0064] During process 217, field data 228 is labeled using WTP prediction model 216 via first MLA 212, resulting in labeled field data 238. In some embodiments, process 217 is not executed until process 215 determines that no WTP threshold 202a has been violated or no deviations or quality biases have been identified.

[0065] After the intended use prediction model 246 is trained and tested, the intended use prediction model 246 processes the labeled field data 238 via a second MLA 240 during process 219. In some embodiments, process 219 is not executed until process 215 determines that the WTP threshold 202a has not been violated or that no off-data or quality deviation has been identified.

[0066] The intended use prediction model 246 generates at least one prediction 256 via a second MLA 240, the prediction including a confidence level. For example, field data of 10 beverages on a conveyor belt can be processed by the intended use prediction model 246, and for each beverage in the field data, 10 predictions and confidence levels are generated, such as the first beverage being regular cola with a confidence level of 66%, the second beverage being orange soda with a confidence level of 33%, and so on. At least one identified quality defect 258 is also generated during process 219. The at least one quality defect 258 reflects the tracking of WTP attributes of the field data based on the predictions reflected by the control WTP 202. A quality defect is identified when WTP 202 differs from the prediction with a confidence level. The system or model administrator can then determine which data is missing from the training or test data and, if necessary, retrain the WTP prediction model 216 or the intended use prediction model 246. By carrying WTP202 throughout Method 200, quality biases can be automatically identified by observing the similarity between training data, test data, and results (including test results 220, 252, and prediction 256).

[0067] In process 221, at least one prediction 256 with an associated confidence level is compared to at least one confidence level threshold 262. Process 221 may include comparing at least one identified quality defect 258 against at least one quality defect threshold 264 of the intended use model 246. Process 221 may include generating a notification 266 indicating whether the confidence level or the number of identified quality defects 258 exceeds or does not exceed the confidence level threshold or the identified quality defect threshold 264, respectively.

[0068] Similar to deviation notification 266, notification 260 may be a signal or display on a user device or on a display 111 of the data processing system 100. Similar to deviation notification 260, in some embodiments, notification 266 may include a request for another data processing system to provide more training or test data. In some embodiments, request 266 may be displayed on display 111 and may identify which WTP attributes in the field data are overrepresented or underrepresented, or which predictions have unacceptable confidence levels. Notification 266 may be human-readable and / or machine-readable (e.g., additional instructions). Notification 260 may include instructions for the data processing system to repeat one or more of processes 201, 203, 205, 207, 209, 211, 213, 215, 217, or 219 using additional training and test data (e.g., for the first machine learning application 212 or for the second machine learning application 240).

[0069] In all aspects, Method 200 can be categorized into three phases: a first phase comprising processes 201, 203, 205, and 207; a second phase comprising processes 209, 211, and 213; and a third phase comprising processes 219 and 221. At any phase, the machine learning administrator can review and compare the KPI performance of the machine learning application or generated model as reflected by WTP 202 (since WTP 202 is carried through each phase via labeled data) to ensure the model performs as expected without deviation. In cases where KPIs (measured against WTP) indicate quality deviations or undesirable results, the machine learning administrator can modify WTP 202 or the data used to generate or test the model to mitigate learning deviations or address quality biases.

[0070] The first machine learning application 212 and the second machine learning application 240 can be classical machine learning applications (e.g., supervised learning applications or reinforcement learning applications), or they can be deep learning MLAs (e.g., convolutional neural networks or recurrent neural networks). The first machine learning application 212 and the second machine learning application 240 can be intended to be used, for example, in computer vision or natural language processing. Those skilled in the art will understand that the methods described herein can be applied to a wide variety of machine learning applications with various use cases.

[0071] By implementing method 200, the intended use model can be updated to eliminate or address deviations present in the intended use model, such as when detecting defects in parts or products during sorting and identification of items. Therefore, method 200 may include additional steps such as retraining the intended use model or implementing the model in an intended use facility or operation (e.g., a manufacturing plant, fulfillment facility, as part of a medical imaging device for diagnosing medical conditions, or as part of a facial recognition device). Method 200 may include transmitting or sending predictions with associated confidence levels to an automated intended use system (e.g., a manufacturing plant, fulfillment facility, or medical device) and causing the automated intended use system to perform intended use operations (e.g., removing defective items from an assembly line, sorting items into packages for transport, or removing tissue for sampling). For example, method 200 may include transmitting or sending at least one prediction with associated confidence levels for each beverage on a conveyor belt to an automated drive system, and causing the picking machine of the automated drive system to sort the beverages based on the predictions. In one implementation, notification 266 may include instructions to cause the automated intended-use system to perform or stop performing intended-use operations (e.g., stop sorting items on a conveyor belt).

[0072] In another example, method 200 may include enabling the automatic anticipation system to alert the user of the automatic anticipation system based on a prediction with an associated confidence level (e.g., by operating a signal, light, or sound). In such an example, the prediction might be about machine malfunction, a defective product, a potential medical abnormality (e.g., cancer), etc., and the automatic anticipation system alerts the user to the prediction by transmitting the prediction and alert to the user's device (e.g., another data processing system, such as a smartphone, laptop computer, etc.).

[0073] refer to Figure 3 This paper illustrates another method 300 for automatically identifying quality biases in machine learning models. Method 300 can be implemented by a system as disclosed herein. Method 300 can be stored as executable instructions on a non-transitory computer-readable medium.

[0074] In operation 310, the system implementing method 300 (e.g., system 100) trains a first machine learning application (e.g., 212) based on a world truth paradigm (e.g., 202) to generate a world truth prediction model (e.g., 216). Operation 310 may also include receiving at least one attribute of the world truth paradigm (e.g., 204), the at least one attribute having a distribution of multiple attribute values ​​(e.g., as shown in Table 1 or Table 2 above). In various embodiments, operation 310 may include receiving the world truth paradigm as input from a model administrator, input from another system, or input from a computer-readable storage device. In some embodiments, operation 310 includes receiving labeled world truth training data (e.g., 208) and labeled world truth test data (e.g., 210) based on the world truth paradigm, and generating a world truth prediction model based on the world truth training data. The system generates a first plurality of key performance indicators (KPIs) (e.g., 214) associated with generating the world truth prediction model in operation 310. In another implementation, operation 310 includes: testing a world-truth prediction model using world-truth test data via a first MLA; generating a first test result (e.g., 220); and generating a second plurality of key performance indicators (KPIs) associated with testing the world-truth prediction model (e.g., 222). In further implementations, operation 310 includes: comparing the first plurality of KPIs with the second plurality of KPIs to determine whether there is a statistical difference between the first test result and the world-truth paradigm; and when there is a statistical difference between the first test result and the world-truth paradigm, retraining the world-truth prediction model based on second world-truth training data.

[0075] In operation 320, the system uses a world truth prediction model via a first MLA (e.g., 212) to label intended-use training data (e.g., 224) and field data (e.g., 228). Operation 320 may include labeling intended-use test data based on a world truth paradigm (e.g., 226). In operation 330, the system uses the labeled intended-use training data to train a second machine learning application (e.g., 240) to generate an intended-use model (e.g., 246). Operation 330 may include at least one of the following: generating a third plurality of KPIs associated with the labeled intended-use test data and intended-use training data (e.g., 232); generating a fourth plurality of KPIs associated with training the second MLA and the intended-use model (e.g., 242); testing the intended-use model via the second MLA using the labeled intended-use test data; generating a second test result and a fifth plurality of KPIs associated with testing the intended-use model (e.g., 250).

[0076] In operation 340, the system uses labeled field data and intended use model via a second MLA to generate at least one prediction with an associated confidence level.

[0077] At operation 350, the system determines at least one quality deviation between the intended use model and the world truth paradigm, or determines at least one quality deviation between at least one prediction and the world truth paradigm. Operation 350 may include comparing at least one prediction against a third, fourth, and fifth set of KPIs to identify a difference between at least one prediction and the expected outcome represented by at least one of the third, fourth, or fifth set of KPIs. Operation 350 may include displaying a notification (e.g., 260) on display device 111, the notification including information about the at least one quality deviation when it is detected.

[0078] Of course, those skilled in the art will recognize that, unless the order of operations is specifically indicated or required, certain steps in the above process may be omitted, performed simultaneously or sequentially, or performed in a different order.

[0079] Those skilled in the art will recognize that, for simplicity and clarity, this document does not depict or describe the complete structure and operation of all data processing systems suitable for use with this disclosure. Instead, only portions of data processing systems specific to or necessary for understanding this disclosure are depicted and described. The remainder of the construction and operation of data processing system 100 may conform to any of the various current implementations and practices known in the art.

[0080] It is important to note that although this disclosure is described within the context of a fully functional system, those skilled in the art will understand that at least a portion of the mechanisms of this disclosure can be distributed in the form of instructions (instructions contained in a machine-usable, computer-usable, or computer-readable medium in any of a variety of forms), and this disclosure applies equally to any type of instruction or signal-bearing medium or storage medium used for distribution in practice. Examples of machine-usable / readable or computer-usable / readable media include: non-volatile, hard-coded media such as read-only memory (ROM) or erasable electrically programmable read-only memory (EEPROM) and user-recordable media such as floppy disks, hard disk drives, and optical disc read-only memory (CD-ROM) or digital universal disc (DVD) .

[0081] Although exemplary embodiments of the present disclosure have been described in detail, those skilled in the art will understand that various changes, substitutions, variations and modifications disclosed herein may be made without departing from the spirit and scope of the broadest form of the disclosure.

[0082] Nothing described in this application should be construed as implying that any particular element, step, or function is an essential element that must be included within the scope of the claims: the scope of patent protection is limited only by the granted claims. Furthermore, none of these claims are intended to invoke 35 USC §112(f) unless the exact phrase “means for…” is followed by a participle. When terms such as (but not limited to) “mechanism,” “module,” “device,” “unit,” “component,” “element,” “building block,” “device,” “machine,” “system,” “processor,” or “controller” are used within the claims, they are understood and intended to refer to structures known to those skilled in the art, structures that are further modified or enhanced by the features of the claims themselves, and are not intended to invoke 35 USC §112(f).

Claims

1. A method for identifying quality biases in a machine learning model, said method being performed by at least one data processing system, comprising: The first machine learning application (MLA) (212) is trained (205) based on the world truth paradigm (202) to generate a world truth prediction model (216). The intended use training data (224) and field data (228) are labeled (209, 217) by using the first MLA of the world truth prediction model. The second machine learning application (MLA) (240) is trained (211) using labeled intended-use training data to generate an intended-use model (246). Using labeled field data and the intended use model, at least one prediction with an associated confidence level (256) is generated (219) via a trained second MLA; as well as Determine (219) at least one quality deviation (258) between the intended use model and the world truth paradigm, or determine at least one quality deviation between the at least one prediction and the world truth paradigm.

2. The method according to claim 1, wherein, Training the first MLA includes receiving at least one attribute (204) of the world truth paradigm, the at least one attribute defining the distribution of multiple attribute values.

3. The method according to claim 1, wherein, Training the first MLA also includes: Receive labeled world truth training data (208) and labeled world truth test data (210), the labeled world truth training data and the labeled world truth test data being labeled based on the world truth paradigm; The world ground truth prediction model is generated based on the world ground truth training data; and Generate a first plurality of key performance indicators (KPIs) associated with the generation of the world truth prediction model (214).

4. The method according to claim 3, wherein, Training the first MLA also includes: The world truth prediction model is tested (207) using the world truth test data via the first MLA; Generate the first test result (220); and Generate and test a second set of multiple key performance indicators (KPIs) associated with the world truth prediction model (222).

5. The method according to claim 4, wherein, Training the first MLA also includes: The first set of KPIs is compared with the second set of KPIs (215) to determine whether there is a statistical difference between the first test results and the world truth paradigm; and When there is a statistical difference between the first test result and the world truth paradigm, the world truth prediction model is retrained based on the second world truth training data (152d).

6. The method according to claim 1, wherein, Training the second MLA also includes: The intended use test data is labeled based on the world truth paradigm via the first MLA (226). Generate a third set of KPIs (232) that are associated with the labeled intended use test data and the intended use training data. Generate a fourth set of KPIs (242) that are associated with training the second MLA and the intended use model. The intended use model (213) is tested via the second MLA using labeled intended use test data; Generate a second test result (252) and a fifth set of multiple KPIs (250) associated with the intended use model described in the test.

7. The method according to claim 6, wherein, Determining at least one quality deviation includes comparing the at least one prediction against the third plurality of KPIs, the fourth plurality of KPIs, and the fifth plurality of KPIs to identify the difference between the at least one prediction and the expected result represented by at least one of the third plurality of KPIs, the fourth plurality of KPIs, or the fifth plurality of KPIs.

8. The method according to claim 7, wherein, Determining at least one quality deviation includes determining whether at least one of a first plurality of KPIs, a second plurality of KPIs, a third plurality of KPIs, a fourth plurality of KPIs, and a fifth plurality of KPIs violates a world truth threshold.

9. The method according to claim 8, further comprising: A deviation notification (260) is generated based on a determination of whether the world truth threshold is violated, and the deviation notification is configured to be displayed on a display (111); as well as When the world truth threshold is violated, the step of using new training or testing data to train or test the first MLA or the second MLA is repeated.

10. The method of claim 1, further comprising: Determine whether the confidence level of the at least one prediction exceeds a confidence level threshold; as well as A notification (266) is generated on the display device (111), the notification including the at least one prediction and whether the confidence level threshold is exceeded.

11. The method according to claim 1, wherein, Determining at least one quality bias includes determining whether the training data violates the world truth threshold (202a).

12. The method of claim 11, further comprising: Determine whether the confidence level of the at least one prediction exceeds the confidence level threshold (262); as well as A notification (266) is generated on the display device, the notification including: The at least one prediction; The at least one quality deviation; Whether the confidence level threshold is exceeded; and Whether it exceeds the stated world truth threshold.

13. The method of claim 12, further comprising: To enable or disable the automatic intended use system from performing or stopping the intended use operation.

14. A data processing system (100), comprising: Processor (102); as well as Accessible memory (108), the data processing system is particularly configured to perform the method according to any one of claims 1 to 13.

15. A non-transitory computer-readable medium (126) encoded with executable instructions that, when executed, cause one or more data processing systems to perform the method according to any one of claims 1 to 13.

Images