Method and system for enabling user feedback and summarizing return of investment for ML systems
By receiving user feedback and retraining the machine learning model based on feedback, the problems of feedback explanation in the prior art are solved, and the model performance improvement and user-friendly feedback mechanism are achieved.
Patent Information
- Application Number
- CN202280094916.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-29
- Publication Date
- 2024-12-03
AI Technical Summary
The lack of appropriate interactive explanations in the prior art that allow users to provide compatible feedback, it is difficult to understand the mechanism used in the feedback process and its impact on the AI model, and the input types or formats for a specific AI model are incompatible, making it difficult to upgrade the model. There is a lack of tools to state the aspects of inference of the ML model.
Provides a method and system that can receive user feedback by training data sets and initial machine learning models, enrich data sets based on feedback, retrain the model, calculate return on input using interactive components and feedback outline views, allowing users to provide about interest feedback of the region and retrain the model through an interactive interpretation mechanism.
The user-friendly feedback mechanism is implemented, the performance of the AI model is improved, the model upgrade process is simplified, and the performance improvement is provided.
Smart Images

Figure CN119072700A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a method and system for enabling user feedback for a machine learning system and summarizing the return on investment. Background Art
[0002] The general context of this disclosure is that interactive machine learning (ML), e.g., active learning, explanatory learning, or visual interactive labeling is a good approach for obtaining labels for supervised machine learning models.
[0003] Artificial intelligence models are becoming more and more popular and are being used more and more frequently in industrial applications. In order to archive the expected machine learning model performance, the model should be continuously updated to maintain the expected performance over a longer time span.
[0004] Continuous updates are required as the dynamic conditions of industrial environments and machines are constantly modified to meet customer requirements and business goals.
[0005] As conditions change, the capabilities and performance of the model may degrade. To avoid degradation in model performance, intuitive tools and streamlined processes (and interfaces) need to be provided to make it easy for end users to provide inputs to AI models.
[0006] The following challenges may arise in the process of providing and assimilating inputs to AI models and should be addressed: lack of suitable interactive explanations that allow users to provide compatible feedback, lack of understanding of the mechanisms used in the feedback process and their impact on the upgraded AI models, incompatible input types or formats for specific AI models that make assimilation and upgrading cumbersome, and lack of tools that allow users to state interesting aspects of the reasoning of ML models. Summary of the invention
[0007] In one aspect of the present invention, a method for enabling user feedback for a machine learning system and summarizing the return on investment is provided, the method comprising: providing a training data set and an initial machine learning model; providing results of the initial machine learning model; receiving feedback from users about the results of the initial machine learning model, and enriching the training data set based on the feedback to obtain an enriched data set; and retraining the initial machine learning model into a retrained machine learning model based on the enriched data set.
[0008] In one embodiment of a method for enabling user feedback for a machine learning system and summarizing return on investment, the step of receiving feedback from a user regarding results of an initial machine learning model is based on a query regarding an area of interest of reasoning of the initial machine learning model.
[0009] In one embodiment of a method for enabling user feedback for a machine learning system and summarizing return on investment, the step of receiving feedback from a user regarding results of an initial machine learning model is based on an interactive explanation that allows the user to provide feedback regarding areas of interest.
[0010] In one embodiment of a method for enabling user feedback for a machine learning system and summarizing return on investment, the step of receiving feedback from a user regarding results of an initial machine learning model includes a feedback summary view.
[0011] In one embodiment of a method for enabling user feedback for a machine learning system and summarizing return on investment, the step of receiving feedback from a user regarding the results of an initial machine learning model includes a return on investment calculation that is configured to illustrate the benefits obtained regarding the overall initial machine learning model performance.
[0012] In one embodiment of the method for enabling user feedback and summarizing return on investment for a machine learning system, the method further includes the step of integrating an interactive component into the interpretation, the interactive component allowing a user to provide feedback on the area of interest.
[0013] In one embodiment of the method for enabling user feedback for a machine learning system and summarizing return on investment, the method also includes the step of calculating the return on investment for the feedback process based on the retrained machine learning model.
[0014] In one embodiment of a method for enabling user feedback for a machine learning system and summarizing return on investment, the method also includes a step of summarizing mechanisms used during the step of receiving feedback regarding results of an initial machine learning model.
[0015] In one aspect of the present invention, a system for enabling user feedback for a machine learning system and summarizing return on investment is provided, the system comprising a processor for executing the method according to the first aspect.
[0016] Any disclosure and embodiment described herein is related to the above-mentioned method and system, and vice versa. Advantageously, the benefits provided by any embodiment and example are also applicable to all other embodiments and examples, and vice versa.
[0017] As used herein, "determining" also includes "initiating or causing determination", "generating" also includes "initiating or causing generation", and "providing" also includes "initiating or causing determination, generation, selection, sending or receiving". "Initiating or causing performance of an action" includes any processing signal that triggers a computing device to perform a corresponding action. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Hereinafter, the present disclosure will be further described with reference to the accompanying drawings:
[0019] Figure 1 Illustrated is a system that enables a user to provide feedback and view the return on investment of the provided feedback in a feedback session summary view;
[0020] Figure 2 illustrates a force plot with interactive intersection points representing the features that contribute to the prediction. A slider component has been integrated into the plot to allow the user to change the position of the intersection points and thereby decide how much each feature contributes to the local or global prediction;
[0021] Figure 3 illustrates an example of how to display the decision boundary of an ML model and use an interactive line component to allow the user to provide input about the decision boundary, thereby illustrating which predictions may be incorrectly predicted; DETAILED DESCRIPTION
[0022] The following embodiments are merely illustrative of the methods and systems disclosed herein and should not be considered limiting.
[0023] Figure 1 Illustrated is a system that enables a user to provide feedback and view the return on investment for the provided feedback in a feedback session summary view.
[0024] During the execution of an ML model, situations may arise where the end user detects unexpected system behaviors that make the end user suspicious about the ML model performance and reasoning. At this stage, the user may not understand what aspects led to the unexpected results, for example, an increase in the number of alarms in a distributed control system or a relatively high number of stroke predictions in a medical system. To explore unexpected system behaviors, the user may decide to investigate the behavior and reasoning of the ML model in the used technical system.
[0025] To allow users to understand which aspects of an ML model may lead to desired or undesired behavior, we propose solutions that allow users to provide queries they think they are interested in. This input is fed to an analyzer that determines which aspects may be worth explaining or representing to answer the query.
[0026] Feedback is used to find the most appropriate explanations, which are also equipped with interactive components to allow users to provide feedback on the explanations.
[0027] The feedback is used to retrain the ML model where the user can get an overview of the different mechanisms used during the feedback session.
[0028] These solutions disclose an input mechanism that allows users to provide input about interesting aspects of the ML model in a variety of ways. The input mechanism is independent of the device and interaction modality, which means that any interaction technology and input modality can be used to obtain input from the user.
[0029] For example, a text input field can be displayed in the human-machine interface to allow the user to type in queries such as "Which features contribute most to the prediction?" or use voice input to state "Which pixels did the ML discard when predicting the guitar on image 4?" or "What was the accuracy of the last 3 predictions?".
[0030] Techniques such as natural language processing or flexible search grammar identification can be used to process and analyze the provided input to identify keywords or sentences that can be used to find the most appropriate explanation class / type to answer the user's query. The analyzed input is matched with an "interpreter classification database" that contains information about the properties of the ML model and the types / classes of explanations that can be derived based on these properties.
[0031] For example, ML model attributes represent the algorithm(s) used to train the model (linear regression, KNN, random forest, neural network), the data format used during ML model training (image, table, time series, binary, text, etc.), and applicable explanation methods (SHAP, LIME) compatible with the ML model. For example, the different explanation classes contained in the "Explainer Classification Database" are features, counterfactuals, confidence scores, performance, etc.
[0032] If there is not any relevant match between the user input and the "Interpreter Classification Database", the database will return a low match score, wherein the user will be informed via any modal communication channel that the above query cannot be answered or represented.
[0033] The search for suitable explanations can also be triggered by a system that tracks and has trigger thresholds for different criteria, such as: ML model accuracy target or number of tasks executed within a duration. The analyzer uses this input to calculate and match areas or aspects of interest that the user should express to identify the cause of the behavior.
[0034] The stage where exploration is generated but may not have any interactive components, which prohibits users from providing feedback, is where the "interactive component visitor and suggester" comes into play.
[0035] The ability of the user to properly provide feedback on the interpretation comes from the ‘Interactive Technology Visitor’. The ‘Interactive Technology Visitor’ includes the following components to be able to evaluate and suggest the correct type or interactive components to be combined with the interpretation;
[0036] (i) compatible data types (which can be used for ML model training);
[0037] (ii) type of equipment;
[0038] (iii) an interactive component database, and
[0039] (iv) Interactive Technology Rules
[0040] (i) Since the feedback provided by the user should be compatible with the data type used to train the ML model, the data format needs to be stored and specified. The data format here can be, for example, binary, image (even pixel coordinates, color and quantity), table or time series data.
[0041] (ii) Device type describes the device and the interactive capabilities it has. Since various devices have different interactive capabilities that can be used to provide output or retrieve input, it is considered best to choose interactive components that are compatible with the input and output capabilities of the device. For example, a tablet computer typically has a screen and audio capabilities that can be used to provide output that the user can perceive, but rarely a taste (smell) modality. It is also important to track the interactive capabilities to provide input to the devices in the tablet computer (e.g., touch screen, microphone voice, stylus, etc.).
[0042] (iii) The interactive competency database includes any interactive elements that can be used to build a human-computer interface. Examples of interactive comments include sliders, buttons, text input fields, drop-down menus, etc. Interactive components have defined (multiple) interactive behaviors and states, such as a button, which can have pressed, held, and released states to make it interactively different from other components. Interactive components can also be interacted with by using various interaction techniques, for example, a button can be interacted with by pressing it on a touch screen, or by clicking a mouse when the cursor is hovering over the button.
[0043] (iv) Interaction Technology Rules define what type of interactive components should be applied to allow the user to provide appropriate feedback by interacting with the presented interpretation. Rules are the glue and intelligence that takes into account the parts included in the “Interactive Component Visitors and Suggestors” and integrates interactive components in otherwise static interpretations.
[0044] The technical rules evaluate the interactive features or interactive components, as well as the interpretation type (drawing or chart type, etc.), the device type, and integrate the appropriate interactive components into the interpretation type to allow it to be interactive.
[0045] Interactivity allows providing feedback on the query of interest. The logic of the interaction technique rule can be exemplified in words accordingly: "If the tablet has touch capabilities and is intended to be used as an explanatory visualization, use a slider component to allow the position of the intersection of the represented feature(s) to be varied". Since the slider component has predefined interactive behavior that it will not slide out of its boundaries, this allows the user to provide feedback on how much any feature should contribute to the prediction within the given interactive boundaries, as this is the area of interest to the user.
[0046] Another rule example is "If a two-dimensional plot is displayed on a computer monitor, and the decision boundary is the region of interest, then if a mouse is available, select Interactive Line to allow the location of the decision boundary to be modified by mouse cursor input."
[0047] Rules can be stored in decision trees or other formats that the system reads more efficiently.
[0048] The description mechanism produces an interactive explanation that is displayed through any output capabilities on the device being used. At this point, the user can interact with the explained statement and provide feedback.
[0049] The feedback provided from the user via interactive explanations is further used to retrain the ML model to improve its performance. In the process of providing queries and feedback through interactive explanations, the user and the system perform various tasks that can be considered as efforts.
[0050] Effort includes (i) user effort, and (ii) system effort performed during the process of explanation generation and providing feedback.
[0051] (i) User effort includes, for example, the number of interactions performed by the user and the duration used to perform the interaction. Examples or interactions include the number of mouse clicks, the number of touch inputs, the number of drag interactions, the number of keyboard presses, or the time spent providing voice input. The system stores the number of interactions in a database, which can later be used to calculate the return on investment. The duration of an interaction is, for example, the time between 2 consecutive mouse clicks or a press and release on any type of touch input.
[0052] (ii) System effort includes the amount of computing resources used in processes such as retrieving user input, analyzing input, generating explanations, and other processes involved starting from when a query is provided by a user to retraining the model and calculating the return on investment. The process starts when a user performs the first interaction providing input to the input mechanism and ends when the "return on investment" index (score) is calculated. This includes resources also used in processes such as deriving a new dataset for retraining the ML model and retraining, which can sometimes be lengthy and resource intensive.
[0053] The effort is normalized to an arithmetic value and summed up to obtain a score for the effort. The various values used in the process (e.g., percentages and durations) are normalized so that a score can be calculated. Percentages can be converted to arithmetic values, such as 67% converting to 0.67, while durations can be converted to points, which means that each millisecond corresponds to 0.01, and as the duration increases, the 0.01 is added up. Also to allow the system or user to balance the degree to which each measured aspect contributes to the effort, weights are applied to each measurement value, which can be manipulated and adjusted by the user, or provided equal weights if initially unmodified.
[0054] Examples of hypothetical formulas expressed in words:
[0055] Effort = sum (weight A*aspect ratio A+weight B*aspect ratio B+…weight N*aspect ratio N)
[0056] Examples using numbers:
[0057] Effort = Total (0.2*0.67+0.9*23+0.3*48+……)
[0058] The upgraded ML model is retrained using enriched set points, which may have a positive or negative impact on the performance. To calculate the impact of the feedback on the retrained ML model, an overall performance score is calculated based on various values. Examples of these values are F1 score, precision, accuracy, AUC, etc. These values are summed up to get an overall performance score for the ML model.
[0059] Similarly, calculate the overall performance score of the initial ML model. Then, subtract the overall performance score from the overall performance score of the initial ML model.
[0060] The ROI score is calculated using the following formula:
[0061] Return on investment = ((overall performance score of the upgraded model) - (overall performance score of the initial model) / effort
[0062] By obtaining ROI (return on investment) calculations, users will understand how the feedback provided and the effort invested affect the overall performance of the upgraded ML model. This understanding can play a role in how users choose to provide feedback and how much effort to spend on the goal of improving performance.
[0063] Additionally, the system stores a history of various data types and aspects used in the process of providing feedback to the ML model. The goal is to create an idea of what and how much was used in the process of providing feedback. Each aspect is represented in the Feedback Summary view. Examples of data and aspects stored by the system:
[0064] (i) an image with a link to an explanation of what is being used,
[0065] (ii) Graphical representations of which interactive components are incorporated into the instructions. These links take the user to a library of graphical components.
[0066] (iii) Name and graphical representation of the original model and the retrained model. Interactive links that allow operators to deploy any ML model.
[0067] (iv) Return on investment scores and values used in the calculations.
[0068] The system also stores a history of each feedback process, where the model was interpreted and further retrained based on the usage feedback. With an overview of different feedback sessions, users can draw conclusions about which exploratory techniques and feedback amounts provided the most favorable or unfavorable effects on various ML models.
[0069] Figure 2 A force plot is illustrated with interactive intersection points representing features that contribute to the prediction. A slider component has been integrated into the plot to allow the user to change the position of the intersection points and thereby decide how much each feature contributes to the local or global prediction.
[0070] Figure 3 An example is shown of how to display the decision boundary of an ML model and use an interactive line component to allow the user to provide input about the decision boundary, thereby illustrating which predictions may be incorrectly predicted.
[0071] In the claims and the description, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality of elements or steps. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
Claims
1. A method for enabling user feedback for a machine learning system and summarizing return on investment, the method comprising: Provide training datasets and initial machine learning models; providing results of the initial machine learning model; receiving feedback from a user regarding the results of the initial machine learning model, and enriching the training dataset based on the feedback to obtain an enriched dataset; The initial machine learning model is retrained into a retrained machine learning model based on the enriched data set.
2. The method according to claim 1, Wherein the step of receiving feedback from a user regarding the results of the initial machine learning model is based on a query related to an area of interest regarding the reasoning of the initial machine learning model.
3. The method according to claim 2, Wherein the step of receiving feedback from a user regarding said results of said initial machine learning model is based on an interactive interpretation allowing a user to provide feedback regarding said region of interest.
4. The method according to any one of claims 1 to 3, Wherein the step of receiving feedback from a user regarding the results of the initial machine learning model includes a feedback summary view.
5. The method according to any one of claims 1 to 4, The step of receiving feedback from a user regarding the results of the initial machine learning model includes a return on investment calculation, which is configured to illustrate the benefits obtained regarding the overall initial machine learning model performance.
6. The method according to any one of claims 1 to 5, Also included is the step of integrating an interactive component into the interpretation, the interactive component allowing a user to provide feedback regarding the region of interest.
7. The method according to any one of claims 1 to 6, Also included is a step of calculating a return on investment for the feedback process based on the retrained machine learning model.
8. The method according to any one of claims 1 to 7, Also included is the step of summarizing the mechanisms used during the step of receiving feedback regarding the results of the initial machine learning model.
9. A system for enabling user feedback for a machine learning system and summarizing return on investment, the system comprising a processor for executing the method according to claims 1 to 8.
