Automated identification of quality deviations in machine learning models

Abstract

A method for identifying quality deviations in a machine learning application, implementable by corresponding systems and computer-readable mediums, may include training (205), based on a world-truth paradigm (202), a first machine learning application (MLA) (212) to generate a world-truth prediction model (216); labeling (209), via the first MLA using the world-truth prediction model, intended-use training data (224) and field data (228); training (211) a second machine learning application (MLA) (240) using the labeled intended-use training data to generate an intended-use model (246); generating (219), via the trained second MLA, using the labeled field data and the intended-use model, at least one prediction (256) having an associated confidence level; and determining at least one quality deviation (258) of the intended-use model from the world-truth paradigm or of the at least one prediction and from the world-truth paradigm.

Description

AUTOMATED IDENTIFICATION OF QUALITY DEVIATIONS IN MACHINE LEARNING MODELSTECHNICAL FIELD

[0001] The present disclosure is directed, in general, to methods and systems for training machine learning applications and identifying quality deviations in machine learning models.BACKGROUND OF THE DISCLOSURE

[0002] Machine learning models are developed, trained, and used based on biased data that may not represent a proper distribution of the characteristics represented by the data. The biased data may inadvertently include noise or may include overrepresented or underrepresented characteristics the machine learning model is being trained to predict. Further, a data set can be characterized as having too many unknown unknowns, that is, it is difficult to know which data is missing. Training the machine learning model using such data causes the machine learning model to develop unintended biases or quality deviations. Machine learning models require manual-human validation to determine any quality deviations between what any given machine learning model predicts, what is expected, and what the data is trained on. Since validation of the data and the machine learning model requires significant human effort and time, improvement of the machine learning model is delayed resulting in lower accuracy predictions. Improved methods for validating data and machine learning models are desired.SUMMARY OF THE DISCLOSURE

[0003] Various disclosed embodiments include a method for automatically identifying quality deviations in a machine learning model. A first machine learning application (MLA) is trained based on a world truth paradigm to generate a world-truth prediction model. Intended-use training data and field data are labeled via the first MLA using the world-truth prediction model. A second machine learning application is trained using the labeled intended-use training data to generate an intended-use model. The method includes generating, using the labeled field data and the intended-use model in the second MLA, at least one prediction having an associated confidence level. The method includes determining at least one quality deviation of the intended-use model from the world-truth paradigm, or at least one quality deviation of the at least one prediction from the worldtruth paradigm.

[0004] In various embodiments, the method includes defining at least one attribute of the world-truth paradigm, the at least one attribute defining a distribution of a plurality of attribute values.

[0005] In various embodiments, the method includes labeling world-truth training data and world-truth test data based on the world-truth paradigm.

[0006] In various embodiments, the method includes generating the world-truth prediction model based on the world-truth training data.

[0007] In various embodiments, the method includes generating a first plurality of key performance indicators (KPIs) associated with generating the world-truth prediction mode.

[0008] In various embodiments, the method includes testing the world-truth prediction model via the first MLA using the world-truth test data.

[0009] In various embodiments, the method includes generating a first test result and generating a second plurality of key performance indicators (KPIs) associated with testing the world- truth prediction model.

[0010] In various embodiments, the method includes comparing the first plurality of KPIs and the second plurality of KPIs to determine whether there is a statistical difference between the first test result and the world-truth paradigm.

[0011] In various embodiments, the method includes retraining, when there is a statistical difference between the first test result and the world-truth paradigm, the world-truth prediction model based on second world-truth training data.

[0012] In various embodiments, the method includes labeling, via the first MLA, intendeduse test data based on the world-truth paradigm.

[0013] In various embodiments, the method includes generating a third plurality of KPIs associated with labeling the intended-use test data and the intended-use training data.

[0014] In various embodiments, the method includes generating a fourth plurality of KPIs associated with training the second MLA and the intended-use model.

[0015] In various embodiments, the method includes testing the intended-use model via the second MLA using the labeled intended-use test data.

[0016] In various embodiments, the method includes generating a second test result and a fifth plurality of KPIs associated with testing the intended-use model.

[0017] In various embodiments, determining at least one quality deviation includes determining whether a world truth threshold was violated by the training data.

[0018] In various embodiments, determining at least one quality deviation includes determining whether at least one of the first, second, third, fourth, and fifth pluralities of KPIs violates a world truth threshold.

[0019] In various embodiments, the method includes determining at least one quality deviation includes comparing the at least one prediction against the third, fourth, and fifth pluralities of KPIs to identify a difference between the at least one prediction and an expected result represented by at least one of the third, fourth, or fifth pluralities of KPIs.

[0020] In various embodiments, the method includes generating a bias notification based on the determination of whether the world truth threshold was violated, the bias notification configured to be displayed on a display. The method may include repeating, when the world truth threshold was violated, the steps of training or testing the first or second MLAs using new training data or testing data.

[0021] In various embodiments, the method includes determining whether the confidence level of the at least one prediction exceeds a confidence level threshold. The method may include generating a notification on a display device, the notification including the at least one prediction and whether the confidence level threshold was exceeded.

[0022] In various embodiments, the method includes determining whether the confidence level of the at least one prediction exceeds a confidence level threshold and generating a notification on a display device. The notification includes the at least one prediction, the at least one quality deviation, whether the confidence level threshold was exceeded, and whether the world truth threshold was exceeded.

[0023] In various embodiments, the method includes causing an automated intended-use system to perform or cease to perform an intended-use operation.

[0024] The foregoing has outlined rather broadly the features and technical advantages of the present disclosure so that those skilled in the art may better understand the detailed description that follows. Additional features and advantages of the disclosure will be described hereinafter that form the subject of the claims. Those skilled in the art will appreciate that they may readily use the conception and the specific embodiment disclosed as a basis for modifying or designing other structures for carrying out the same purposes of the present disclosure. Those skilled in the art will also realize that such equivalent constructions do not depart from the spirit and scope of the disclosure in its broadest form.

[0025] Before undertaking the DETAILED DESCRIPTION below, it may be advantageous to set forth definitions of certain words or phrases used throughout this patent document: the terms “include” and “comprise,” as well as derivatives thereof, mean inclusion without limitation; the term “or” is inclusive, meaning and / or; the phrases“associated with” and “associated therewith,” as well as derivatives thereof, may mean to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, or the like; and the term “controller” means any device, system or part thereof that controls at least one operation, whether such a device is implemented in hardware, firmware, software or some combination of at least two of the same. It should be noted that the functionality associated with any particular controller may be centralized or distributed, whether locally or remotely. Definitions for certain words and phrases are provided throughout this patent document, and those of ordinary skill in the art will understand that such definitions apply in many, if not most, instances to prior as well as future uses of such defined words and phrases. While some terms may include a wide variety of embodiments, the appended claims may expressly limit these terms to specific embodiments.BRIEF DESCRIPTION OF THE DRAWINGS

[0026] For a more complete understanding of the present disclosure, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, wherein like numbers designate like objects, and in which:

[0027] Figure 1 illustrates a block diagram of a data processing system in which an embodiment can be implemented;

[0028] Figures 2A-C illustrate a process-diagram of a method for identifying quality deviations of a machine learning model; and

[0029] Figure 3 illustrates a flow chart of another method for identifying quality deviations of a machine learning model.DETAILED DESCRIPTION

[0030] FIGURES 1 through 3, discussed below, and the various embodiments used to describe the principles of the present disclosure in this patent document are by way of illustration only and should not be construed in any way to limit the scope of the disclosure. Those skilled in the art will understand that the principles of the present disclosure may be implemented in any suitably arranged device. The numerous innovative teachings of the present application will be described with reference to exemplary non-limiting embodiments.

[0031] To address the need to identify quality deviations of machine learning models efficiently and quickly, this disclosure provides a method and systems thereof for determining quality deviations in a machine learning model using a world-truth paradigm.

[0032] Today, a model administrator needs to evaluate the quality of data manually against a data reference framework to find quality deviations in data or a machine learning model. Model administrators need to run tests with specific data sets to determine whether one of any given number of problems exist with a machine learning model, the data that trained the machine learning model, or the data used to test the machine learning model. Problems include biased data, data with excess noise, or mode drift. Biased data occurs when some data attributes are underrepresented, overrepresented, or not represented at all. Data with excess noise can unintentionally influence the training of a machine learning model or the testing of a machine learning model. Mode drift occurs when new data processed by the machine learning model is different than the data used to train the machine learning model. When a problem is identified, the model is retrained to address or resolve the identified problem.

[0033] This disclosure enables data processing systems or model administrators to quickly and efficiently identify quality deviations by providing a reference framework that is carried through the training, testing, and real- world implementation of a machine learning model. The reference framework is a world-truth paradigm. The method includes receiving the world-truth paradigm, training a first machine learning application to generate a worldtruth paradigm prediction model that can label data according to the world-truth paradigm,and using the world-truth prediction model to label, according to the world-truth paradigm, additional data used to train, test, and implement a second machine learning application.

[0034] The methods of this disclosure enable model administrators and machine learning applications to more quickly diagnose and correct unintentional biases and preferences in a machine learning model by automatic and immediate detection of unintended biases in training data, test data, and field data without human involvement. By carrying data labeled according to the world-truth paradigm, and comparing the labeled data to the world-truth paradigm, data used to generate or test a machine learning model can be more carefully selected for more accurate modeling. Further, the speed at which biases and the cause of the biases can be detected is greatly enhanced since such diagnosis and detection does not need to be performed exclusively by a model administrator. Rather, by automatically tagging data based on a world-truth paradigm as described herein, what was previously unknown unknown-data can be determined quickly and efficiently, and more known unknown-data can be selected to retrain the machine learning model to remove or mitigate learned biases in the machine learning model. Disclosed processes provide a distinct technical advantage in greatly shortening the time required to identify the biased data (e.g., data that is overrepresented or underrepresented) and the time required to identify, request, or gather training and testing data needed to overcome the unintended bias. Together, this further reduces the time to retrain a machine learning application to generate a more accurate machine learning model, which can lead to faster and more accurate applications of a machine learning model.

[0035] As used herein, the phrase “world truth paradigm” describes a set of characteristics and parameters, and distributions thereof, that comprise a set of metadata that specify an expectation for the quality and quantity data. In other words, the “world-truth paradigm” is defined to reflect the characteristics and parameters, and distributions thereof expected to be found in any given data set (e.g., training data, test data, or field data).

[0036] A “world truth prediction” is a prediction, made using a model generated by a machine learning application, of the characteristics and parameters, and distributions thereof, of a data set based on and compared to the “world truth paradigm.” In other words,a “world truth prediction” is a set of data labeled based on the “world truth paradigm.” For example, a “world truth paradigm” may specify a distribution of 25% squares, 25% circles, 25% diamonds, and 25% triangles, and a "world truth prediction” of data in a data set may be 23% squares 27% circles, 10% diamonds, and 35% triangles.

[0037] As used herein, the phrase “intended-use” describes a relation to a machine learning model developed for a particular intended-use, (e.g., to detect anomalies in data, to detect a potential medical anomaly (e.g., detect cancer in an image of a body part), or to determine shapes or faces in an image or video). Thus, for example, “intended-use training data” refers to training data that is representative of field data a machine learning model will process to make a prediction for any intended-use. Similarly, for example, “intended-use test data” refers to data that is representative of field data that is used to test a model generated by a machine learning application. In another example, the phrase, “intendeduse field data” refers to field data or real- world data processed by a machine learning model to make a prediction for an intended use (e.g., predicting the presence of a shape or a defect in a manufactured product).

[0038] Figure 1 illustrates a block diagram of a data processing system 100 in which an embodiment can be implemented, for example as a computer system particularly configured by software or otherwise to perform the processes as described herein, and in particular as each one of a plurality of interconnected and communicating systems as described herein. The data processing system 100 depicted includes a processor 102 connected to a level two cache / bridge 104, which is connected in turn to a local system bus 106. Local system bus 106 may be, for example, a peripheral component interconnect (PCI) architecture bus. Also connected to the local system bus 106 in the depicted example arc a main memory 108 and a graphics adapter 110. The graphics adapter 110 may be connected to display 111.

[0039] Other peripherals, such as local area network (LAN) / Wide Area Network I Wireless (e.g. WiFi) adapter 112, may also be connected to local system bus 106. Expansion bus interface 114 connects local system bus 106 to input / output (I / O) bus 116. I / O bus 116 is connected to keyboard / mouse adapter 118, disk controller 120, and I / Oadapter 122. Disk controller 120 can be connected to a storage 126, which can be any suitable machine usable or machine readable storage medium, including but not limited to nonvolatile, hard-coded type mediums such as read only memories (ROMs) or erasable, electrically programmable read only memories (EEPROMs), magnetic tape storage, and user-recordable type mediums such as floppy disks, hard disk drives and compact disk read only memories (CD-ROMs) or digital versatile disks (DVDs), and other known optical, electrical, or magnetic storage devices.

[0040] Also connected to I / O bus 116 in the example shown is audio adapter 124, to which speakers (not shown) may be connected for playing sounds. Keyboard / mouse adapter 118 provides a connection for a pointing device (not shown), such as a mouse, trackball, trackpointer, touchscreen, etc.

[0041] Those of ordinary skill in the art will appreciate that the hardware depicted in Figure 1 may vary for particular implementations. For example, other peripheral devices, such as an optical disk drive and the like, also may be used in addition or in place of the hardware depicted. The depicted example is provided for the purpose of explanation only and is not meant to imply architectural limitations with respect to the present disclosure.

[0042] A data processing system 100 in accordance with an embodiment of the present disclosure includes an operating system employing a graphical user interface. The operating system permits multiple display windows to be presented in the graphical user interface simultaneously, with each display window providing an interface to a different application or to a different instance of the same application. A cursor in the graphical user interface may be manipulated by a user through the pointing device. The position of the cursor may be changed and / or an event, such as clicking a mouse button, generated to actuate a desired response.

[0043] One of various commercial operating systems, such as a version of Microsoft Windows™, a product of Microsoft Corporation located in Redmond, Wash, may be employed if suitably modified. The operating system is modified or created in accordance with the present disclosure as described.

[0044] LAN / WAN / Wireless adapter 112 can be connected to a network 130 (not a part of data processing system 100), which can be any public or private data processing system network or combination of networks, as known to those of skill in the art, including the Internet. Data processing system 100 can communicate over network 130 with server system 140, which is also not part of data processing system 100, but can be implemented, for example, as a separate data processing system 100.

[0045] Storage 126 can store, in various embodiments, any data or code necessary or useful for performing processes as described herein. For example, storage 126 can store elements such as computer programs (e.g., machine learning application 150A, machine learning application 150b, methods 200, or method 300), data 152 (e.g., training data 152A, test data 152B, or field data 152C, second training data 152d, second test data 152e, second field data 152f), computer program models (e.g., machine learning generated models 154A or 154b). Data 152 may include software architecture, any of its elements, or any other data, programs, code, tables, data lake depositories, key performance indicators, or other information or data used in implementing any of the methods of this disclosure discussed below. In aspects, the machine learning applications 150A or 150B, the data 152A, 152B, or 152C, the method 200, the method 300, or the machine learning generated models 154A or 154B may be each stored separately or in other combinations on another storage device of the data processing system 100 or of a remote data processing system (not shown). Machine learning applications 150A or 150B may be the machine learning applications 212 or 240 described below. Data 152 may include any of the data described below with respect to methods 200 or 300.

[0046] With reference to Figures 2A-C, a process diagram of a method 200 for identifying quality deviations of machine learning models in accordance with aspects of this disclosure is provided. Portions of the method 200 may be implemented by one or more data processing systems 100, referred to generically as the “system” herein, and each of the processes described are performed by the system. The method 200 may be stored as executable instructions on a non-transitory computer readable medium.

[0047] The method 200 includes process 201 of receiving a world-truth paradigm (WTP) 202. As used herein, “receiving” includes, but is not limited to, loading from machine readable storage, receiving from another computing system or process, or receiving via an interaction with a user (e.g., as an input via a keyboard, a mouse, or a touchscreen). The world-truth paradigm (WTP) 202 includes one or more WTP attributes 204, each WTP attribute 204 defining a distribution of a plurality of attribute values. The WTP 202 may be pre-defined by a model administrator (e.g., a person training a machine learning application to generate a predictive model) or by a system configured to define or refine a WTP based a pre-defined WTP.

[0048] The WTP attributes 204 are configured to be used as meta-data to label any given data (e.g., training data, test data, or field data) and to specify an expectation for the quality and quantity of data used to train a machine learning model. For example, for each data item of a data set, an attribute value per at least one attribute is assigned. The WTP 202 thus expresses how often each attribute value, or a combination of attributes and associated attribute values, should be represented by the data set.

[0049] For example, a WTP attribute 204 might be “form” and attribute values may include circle, triangle, square, rhombus, or rectangle. In another example, a WTP attribute 204 may be “color” and attribute values may be yellow, red, and blue. The WTP attribute 204 may be determined based on the desired purpose of a machine learning model the data to be labeled is related to. The WTP attributes 204 may include, but are not limited to: object category including color, shape, type; modality (e.g., image or video); quality (e.g., black or white, color depth 1, color depth 2, or excess noise); perspective (e.g., top, bottom, left side, right side, or isometric); race (e.g., African-American, Caucasian, Asian, or Latin- American); or direction (e.g., north, south, east, west). For example, if a machine learning model for predicting faces in a picture is to be trained, attributes may include eyes, ears, nose, and mouth, and values might be variously defined for each attribute. In another example, if a machine learning model is being trained to predict cans of various drinks on a conveyor belt, the attributes might be types of a drink (e.g., juice, water, soda), brand names, or volume size.

[0050] The WTP attribute values may be expressed using a binary mapping distribution scheme. For example, Table 1 illustrates one example of WTP attributes expressed using binary values.Table 1. Example Binary Mapping of Example WTP AttributesTable 1 thus expresses that the WTP states that squares are red, triangles are yellow, and circles are blue.

[0051] In aspects, the WTP attribute values may be expressed using a likelihood of occurrence as a percentage. For example, Table 2 illustrates one example of WTP attributes expressed using binary values.Table 2. Example Attribute Value Distribution as a PercentageTable 2 thus expresses that the WTP states that of shapes targeted for identification, 40% should be circles, 30% should be triangles, and 30% should be squares. While Tables 1 and 2 describe attributes of shapes and attribute values that are colors, one of ordinary skill in the art would appreciate that attributes and attribute values may extend beyond basic shapes and colors, and may include a modality, a perspective, a quality, or other attributes a machine learning model is trained to identify. The defined world-truth paradigm and itsattributes are used in each subsequent step of the method 200 to track whether a machine learning model is biased toward or away from any particular WTP attribute 204.

[0052] The method 200 includes a process 203 of labeling data using WTP attributes 204 producing WTP training data 208 and WTP test data 210. For each data item of a data set (e.g., WTP training data 208 or WTP test data 210) a minimum or maximum number of data items that are assigned to a WTP attribute 204 or combinations of WTP attributes 204 can be determined. By labeling training data and test data using the WTP 202, the system 100 or a model administrator is able to determine how the labeled data correlates to a result or prediction of a machine learning model trained on the data to identify quality deviations.

[0053] The labeled WTP training data 208 is then used in a process 205 to train a first machine learning application (MLA) 212, (e.g., a module for world-truth prediction), to generate a WTP prediction model 216 and training key performance indicators (KPIs) 214. The WTP prediction model 216 is configured to automatically predict, via the first MLA 212 implementing the WTP prediction model 216, WTP attributes 204 of unlabeled data and then label the data with the WTP attributes 204. The KPIs 214 may include metrics such as, but not limited to, accuracy, precision, recall, F-l score, mean squared error, area- under-the-curve (AUC), and the like known by those of ordinary skill in the art.

[0054] The WTP prediction model 216 is tested during a process 207 using the WTP test data 210 to determine whether the WTP prediction model properly predicts and labels WTP attributes of data according to the WTP 202. Process 207 yields a test result 220 and test key performance indicators (KPIs) 222. The test result 220 are the predicted WTP attributes 204 of the WTP test data 210. The test KPIs 222 provide an indication as to whether the WTP prediction model 216 accurately predicted the WTP test data 210. The test KPIs 222 are compared, either by the system or the model administrator, against the training KPIs 214 to determine a quality deviation in the WTP prediction model 216. When a quality deviation is identified, the system or the model administrator can determine which data is missing or inaccurately represented in the training or test data as compared to the WTP 202. The system or the model administrator may then repeat, processes 203, 205, and 207.

[0055] The method 200 includes a process 209 wherein intended-use data, including training data 224 and test data 226is labeled using the WTP prediction model 216, via the first MLA 212, yielding labeled training data 234 and labeled test data 236. Process 209 also generates WTP labeling KPIs 232. WTP labeling KPIs 232 are configured to indicate the performance of the WTP prediction model 216 to predict the WTP attributes 204 of the training data 224, test data 226, and field data 228. The WTP labeling KPIs 232 measure how accurately the training data 224 and test data 226 comply with the WTP 202.

[0056] During a process 211, a second machine learning model 240 is trained using the labeled training data 234 to generate an intended-use prediction model 246 and WTP KPIs 242. The intended-use prediction model 246 is configured to make a prediction using field data for an intended purpose (e.g., detecting anomalies on a conveyor belt, detecting quality issues during a manufacturing process, or determining a medical diagnosis from medical images). The WTP KPIs 242 are a measure of how accurately the training data complies with the WTP 202. The WTP KPIs 242 differ from the WTP KPIs 232 in that the WTP KPIs 242 may be a measure of all training data used to train the second machine learning model 240. For example, multiple batches of labeled training data 234 may be used as an input in process 211, and the WTP KPIs 242 reflects a total of how accurately all labeled training data used complies with the WTP 202.

[0057] During a process 213, the intended-use prediction model 246 is tested, via the second MLA 248, using the labeled test data 236. The process 213 generates a test result 252 and WTP KPIs 250. The test result 252 is at least one prediction made by the intendeduse prediction model 246. The WTP KPIs 250 provide an indication as to whether there is a quality deviation in the testing data based on a comparison to the WTP 202. The WTP KPIs 250 are a measure of how accurately the test data complies with the WTP 202. The WTP KPIs 250 differ from the WTP KPIs 232 in that the WTP KPIs 250 may be a measure of all test data used to test the second machine learning model 240. For example, multiple batches of labeled test data 236 may be used as an input in process 213, and the WTP KPIs 250 reflects a total of how accurately all labeled training data used complies with the WTP 202.

[0058] During a process 215 the WTP 202, the WTP KPIs 232, the WTP KPIs 242, and the WTP KPIs 250, are compared with each other to o determine a quality deviation or data bias in the data, and thus in the intended-use prediction model 246. For example, the test WTP KPIs 250 are compared to the training KPIs 242 to determine which WTP attributes are expected versus which were predicted, indicating a quality deviation in the intendeduse prediction model 246. The process 215 may include also comparing world truth thresholds 202a against the KPIs. The world truth threshold 202a is a predefined tolerance of how far from the WTP 202 any of the data (e.g., training data 224 or test data 226) may venture. A quality deviation is determined when, during process 215, biased data is found to violate the world truth threshold 202a.

[0059] When a quality deviation is identified, (e.g., a bias in any of the training or test data are identified), the data processing system 100 generates a notification 260 identifying the biased data. The notification 260 may be a signal or a display on a user device or the display 111 of the data processing system 100.

[0060] Process 215 may include determining which data is missing or inaccurately represented in the training or test data as compared to the WTP 202 and selecting new data from the storage device 126 to cure, counter, or mitigate the biased data when the world truth threshold his violated. In some embodiments, the notification includes a request to another data processing system to provide missing training or test data. In some embodiments, the request may be displayed on the display 111 and may identify which WTP attributes were overrepresented or underrepresented in the data. The notification 260 may include instructions to the data processing system to repeat one or more of processes 209, 211, 213, and 215. In some embodiments, a model administrator may cause the data processing system to repeat processes 209, 211, 213, and 215. In some embodiments, process 215 may include generating a notification that no biased data was determined or that the world truth threshold 202a was not violated.

[0061] During a process 217, field data 228 is labeled using the WTP prediction model 216, via the first MLA 212, yielding labeled field data 238. In some embodiments, process217 is not executed until after process 215 determines that the WTP threshold 202a was not violated or that no biased data or quality deviations were identified.

[0062] After the intended-use prediction model 246 is trained and tested, the intended-use prediction model 246, via the second MLA 240, processes the labeled field data 238 during process 219. In some embodiments, process 219 is not executed until after process 215 determines that the WTP threshold 202a was not violated or that no biased data or quality deviations were identified.

[0063] The intended-use prediction model 246, via the second MLA 240, generates at least one prediction 256 which includes a confidence level of the prediction. For example, field data of 10 drinks on a conveyor belt may be processed by the intended-use prediction model 246 and generate 10 predictions and a confidence level, such as a first drink is a regular cola with a confidence level of 66%, a second drink is an orange soda with a confidence level of 33%, etc., for each drink of the field data. Also generated during process 219 is at least one identified quality deficiency 258. The at least one quality deficiency 258 reflects the tracking of the WTP attributes of the field data reflected based on the predictions against the WTP 202. Where the WTP 202 and the predictions with confidence level differ, a quality deficiency is identified. The system or the model administrator is then able to determine which data is missing from the training or test data and retrain the WTP prediction model 216 or the intended-use prediction model 246 as required. By carrying the WTP 202 throughout the method 200, quality deviations can be identified automatically by seeing how closely training data, test data, and results (including the test results 220, 252, and the predictions 256).

[0064] At process 221, the at least one prediction 256 having an associated confidence level is compared to at least one confidence level threshold 262. Process 221 may include comparing the at least one identified quality deficiency 258 against at least one quality deficiency threshold 264 for the intended-use model 246. Process 221 may include generating a notification 266 whether the confidence level or number of identified quality deficiencies 258 exceeds, or does not exceed, the confidence level threshold or identified quality deficiencies threshold 264, respectively.

[0065] Like bias notification 266, notification 260 may be a signal or a display on a user device or the display 111 of the data processing system 100. Similarly to bias notification 260, in some embodiments, the notification 266 may include a request to another data processing system to provide more training or test data. In some embodiments, the request 266 may be displayed on the display 111 and may identify which WTP attributes were overrepresented or underrepresented in the field data, or which predictions had an unacceptable confidence level. The notification 266 can be human-readable and / or machine-readable (e.g., further instructions). The notification 260 may include instructions to the data processing system to repeat one or more of processes 201, 203, 205, 207, 209, 211, 213, 215, 217, or 219 using additional training and test data (e.g., for the first machine learning application 212 or for the second machine learning application 240).

[0066] In aspects, the method 200 may be categorized into 3 phases, a first phase including processes 201, 203, 205, and 207; a second phase including processes 209, 211, and 213; and a third phase including processes 219 and 221. At any phase, a machine learning administrator may review and compare KPIs performance of a machine learning application or a generated model, reflected by the use of the WTP 202 as the WTP 202 is carried through each phase via labeled data, to ensure a model is behaving as desired without biases. Where quality deviation or undesired result is indicated by the KPIs (measured against the WTP), the WTP 202 or the data used to generate or test a model may be modified by a machine learning administrator to mitigate the learned bias or address the quality deviation.

[0067] The first and second machine learning applications 212, 240, respectively, may be classical machine learning applications (e.g., supervised learning applications or reinforcement learning applications) or they may be deep learning MLAs (e.g., convolutional neural networks or recurrent neural networks). The first and second machine learning applications 212, 240, may be intended to be used as, for example, computer vision or natural language processing. One of ordinary skill in the art will understand that the methods described herein may apply to a wide variety of machine learning applications having various use cases.

[0068] By implementing the method 200, the intended-use model can be updated to eliminate or address biases the intended-use model has, for example, when detecting defects in parts or products when sorting and identifying items. The method 200 may thus include further steps of retraining the intended-use model, or implementing the model in an intended use facility or operation (e.g., a manufacturing plant, a fulfillment facility, as part of a medical imaging device for diagnosing a medical condition, or as part of a facial recognition device). The method 200 may include transmitting or sending the prediction with an associated confidence to an automated intended-use system (e.g., manufacturing plant, fulfillment facility, or a medical device) and causing the automated intended-use system to perform an intended-use operation (e.g., remove a defective item from an assembly line, sort an item into a package for shipping, or resect a tissue for sampling). For example, the method 200 may include transmitting or sending to an automated actuation system the at least one prediction with an associated confidence level of each drink on a conveyor belt and cause a picking machine of the automated actuation system to sort a drink based on the prediction. In embodiments, the notification 266 may include instructions to the automated intended-use system to perform or cease to perform an intended-use operation (e.g., stop sorting items on a conveyor belt).

[0069] In another example, the method 200 may include causing an automated intendeduse system, based on the prediction having an associated confidence level, to alert a user of the automated intended-use system (e.g., by operating a signal, a light, or a sound). In such an example, the prediction may be of a machine operating in error, of a defective product, of a potential medical anomaly (e.g., cancer), and the like, the automated intendeduse system alerts a user of the prediction by transmitting the prediction and an alert to a user device (e.g., another data processing systems such as a smartphone, laptop, and the like).

[0070] With reference to Figure 3, another method 300 for automatically identifying quality deviations in a machine learning model is shown. The method 300 may be implemented by a system as disclosed herein. The method 300 may be stored as executable instructions on a non-transitory computer readable medium.

[0071] At operation 310, a system (e.g., system 100) implementing the method 300 trains a first machine learning application (e.g., 212) based on a world truth paradigm (e.g., 202) to generate a world-truth prediction model (e.g., 216). Operation 310 may further include receiving at least one attribute (e.g., 204) of the world-truth paradigm, the at least one attribute having a distribution of a plurality of attribute values (e.g., as shown in Tables 1 or 2 above). In various embodiments, operation 310 may include receiving the world-truth paradigm as an input from a model administrator, from another system, or a computer readable storage device. In some embodiments, operation 310 includes receiving labeled world-truth training data (e.g., 208) and labeled world-truth test data (e.g., 210) based on the world-truth paradigm and generating the world-truth prediction model based on the world- truth training data. The system, at operation 310, generates a first plurality of key performance indicators (KPIs) (e.g., 214) associated with generating the world-truth prediction model. In additional embodiments, operation 310 includes: testing the worldtruth prediction model via the first MLA using the world-truth test data; generating a first test result (e.g., 220); and generating a second plurality of key performance indicators (KPIs) (e.g., 222) associated with testing the world-truth prediction model. In more embodiments, operation 310 includes comparing the first plurality of KPIs and the second plurality of KPIs to determine whether there is a statistical difference between the first test result and the world-truth paradigm; and retraining, when there is a statistical difference between the first test result and the world-truth paradigm, the world-truth prediction model based on second world-truth training data.

[0072] At operation 320, the system labels intended-use training data (e.g., 224) and field data (e.g., 228) via the first MLA (e.g., 212) using the world-truth prediction model. Operation 320 may include labeling intended-use test data (e.g., 226) based on the world truth paradigm. At operation 330, the system trains a second machine learning application (e.g., 240) using the labeled intended-use training data to generate an intended-use model (e.g., 246). Operation 330 may include at least one of: generating a third plurality of KPIs (e.g., 232) associated with labeling the intended-use test data and the intended-use training data; generating a fourth plurality of KPIs (e.g., 242) associated with training the second MLA and the intended-use model; testing the intended-use model via the second MLAusing the labeled intended-use test data; generating a second test result and a fifth plurality of KPIs (e.g., 250) associated with testing the intended-use model.

[0073] At operation 340 the system generates, the second MLA, using the labeled field data and the intended-use model, at least one prediction having an associated confidence level.

[0074] At operation 350, the system determines at least one quality deviation of the intended-use model from the world-truth paradigm, or at least one quality deviation of the at least one prediction from the world-truth paradigm. Operation 350 may include comparing the at least one prediction against the third, fourth, and fifth pluralities of KPIs to identify a difference between the at least one prediction and an expected result represented by at least one of the third, fourth, or fifth pluralities of KPIs. Operation 350 may include displaying a notification (e.g., 260) on a display device 111, the notification including information regarding the at least one quality deviation if at least one quality deviation is detected.

[0075] Of course, those of skill in the art will recognize that, unless specifically indicated or required by the sequence of operations, certain steps in the processes described above may be omitted, performed concurrently or sequentially, or performed in a different order.

[0076] Those skilled in the art will recognize that, for simplicity and clarity, the full structure and operation of all data processing systems suitable for use with the present disclosure is not being depicted or described herein. Instead, only so much of a data processing system as is unique to the present disclosure or necessary for an understanding of the present disclosure is depicted and described. The remainder of the construction and operation of data processing system 100 may conform to any of the various current implementations and practices known in the art.

[0077] It is important to note that while the disclosure includes a description in the context of a fully functional system, those skilled in the art will appreciate that at least portions of the mechanism of the present disclosure are capable of being distributed in the form of instructions contained within a machine-usable, computer-usable, or computer-readablemedium in any of a variety of forms, and that the present disclosure applies equally regardless of the particular type of instruction or signal bearing medium or storage medium utilized to actually carry out the distribution. Examples of machine usable / readable or computer usable / readable mediums include: nonvolatile, hard-coded type mediums such as read only memories (ROMs) or erasable, electrically programmable read only memories (EEPROMs), and user-recordable type mediums such as floppy disks, hard disk drives and compact disk read only memories (CD-ROMs) or digital versatile disks (DVDs).

[0078] Although an exemplary embodiment of the present disclosure has been described in detail, those skilled in the art will understand that various changes, substitutions, variations, and improvements disclosed herein may be made without departing from the spirit and scope of the disclosure in its broadest form.

[0079] None of the description in the present application should be read as implying that any particular element, step, or function is an essential element which must be included in the claim scope: the scope of patented subject matter is defined only by the allowed claims. Moreover, none of these claims are intended to invoke 35 USC § 112(f) unless the exact words "means for" are followed by a participle. The use of terms such as (but not limited to) “mechanism,” “module,” “device,” “unit,” “component,” “element,” “member,” “apparatus,” “machine,” “system,” “processor,” or “controller,” within a claim is understood and intended to refer to structures known to those skilled in the relevant art, as further modified or enhanced by the features of the claims themselves, and is not intended to invoke 35 U.S.C. § 112(f).

Claims

WHAT IS CLAIMED IS:

1. A method for identifying quality deviations in a machine learning model, the method performed by at least one data processing system, comprising: training (205), based on a world-truth paradigm (202), a first machine learning application (MLA) (212) to generate a world-truth prediction model (216); labeling (209, 217), via the first MLA using the world-truth prediction model, intended-use training data (224) and field data (228); training (211) a second machine learning application (MLA) (240) using the labeled intended-use training data to generate an intended-use model (246); generating (219), via the trained second MLA, using the labeled field data and the intended-use model, at least one prediction having an associated confidence level (256); and determining (219) at least one quality deviation (258) of the intended-use model from the world-truth paradigm, or at least one quality deviation of the at least one prediction from the world-truth paradigm.

2. The method of claim 1, wherein training the first MLA includes receiving at least one attribute (204) of the world-truth paradigm, the at least one attribute defining a distribution of a plurality of attribute values.

3. The method of claim 1, wherein training the first MLA further includes: receiving labeled world-truth training data (208) and labeled world-truth test data (210), the labeled world-truth training and test data labeled based on the world-truth paradigm; generating the world-truth prediction model based on the world-truth training data; and generating a first plurality of key performance indicators (KPIs) (214) associated with generating the world-truth prediction model.

4. The method of claim 3, wherein training the first MLA further includes: testing (207) the world-truth prediction model via the first MLA using the worldtruth test data; generating a first test result (220); and generating a second plurality of key performance indicators (KPIs) (222) associated with testing the world-truth prediction model.

5. The method of claim 4, wherein training the first MLA further includes: comparing (215) the first plurality of KPIs and the second plurality of KPIs to determine whether there is a statistical difference between the first test result and the worldtruth paradigm; and retraining, when there is a statistical difference between the first test result and the world-truth paradigm, the world-truth prediction model based on second world-truth training data (152d).

6. The method of claim 1, wherein training the second MLA further includes: labeling, via the first MLA, intended-use test data (226) based on the world-truth paradigm; generating a third plurality of KPIs (232) associated with labeling the intended-use test data and the intended-use training data; generating a fourth plurality of KPIs (242) associated with training the second MLA and the intended-use model; testing (213) the intended-use model via the second MLA using the labeled intended-use test data; generating a second test result (252) and a fifth plurality of KPIs (250) associated with testing the intended-use model.

7. The method of claim 6, wherein determining at least one quality deviation includes comparing the at least one prediction against the third, fourth, and fifth pluralities of KPIs to identify a difference between the at least one prediction and an expected result represented by at least one of the third, fourth, or fifth pluralities of KPIs.

8. The method of claim 7, wherein determining at least one quality deviation includes determining whether at least one of the first, second, third, fourth, and fifth pluralities of KPIs violates a world truth threshold.

9. The method of claim 8, further comprising: generating a bias notification (260) based on the determination of whether the world truth threshold was violated, the bias notification configured to be displayed on a display (111); and repeating, when the world truth threshold was violated, the steps of training or testing the first or second MLAs using new training data or testing data.

10. The method of claim 1, further including: determining whether the confidence level of the at least one prediction exceeds a confidence level threshold; and generating a notification (266) on a display device (111), the notification including the at least one prediction and whether the confidence level threshold was exceeded.

11. The method of claim 1 , wherein determining at least one quality deviation includes determining whether a world truth threshold (202a) was violated by the training data.

12. The method of claim 11, further including: determining whether the confidence level of the at least one prediction exceeds a confidence level threshold (262); and generating a notification (266) on a display device, the notification including: the at least one prediction; the at least one quality deviation; whether the confidence level threshold was exceeded; and whether the world truth threshold was exceeded.

13. The method of claim 12, further including: causing an automated intended-use system to perform or cease to perform an intended-use operation.

14. A data processing system (100) comprising: a processor (102); and an accessible memory (108), the data processing system particularly configured to perform a method as in any of claims 1-13.

15. A non-transitory computer-readable medium (126) encoded with executable instructions that, when executed, cause one or more data processing systems to perform a method as in any of claims 1-13.

Images