Display device, display method, and display program

Abstract

This system helps to output appropriate explanatory information even when the prediction results of a machine learning model differ from intuitive results. [Solution] The display device generates a learning model by training it with training data that associates information about multiple subjects being evaluated with multiple explanatory variables and a target variable, calculates a first SHAP value for each explanatory variable, extracts interacting features based on the first SHAP value for each explanatory variable, calculates the prediction probability and a second SHAP value based on the learning model and inference data, and generates and outputs explanatory information that shows the basis for calculating the second SHAP value based on the combination of explanatory variables and features that interact with the second SHAP value.

Description

[Technical Field]

[0001] This disclosure relates to a display device, a display method, and a display program. [Background technology]

[0002] Patent Document 1 describes an explanatory information output program that causes a computer to perform a process in which it calculates the contribution of each of the multiple data to the prediction result, selects a specific variable from among the multiple variables, determines a specific data from among the multiple data based on the value of the specific variable for each of the multiple data and the contribution, and outputs the specific data as explanatory information for the prediction result. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-134563 [Overview of the project] [Problems that the invention aims to solve]

[0004] This disclosure was devised in view of the aforementioned conventional circumstances and aims to provide a display device, display method, and display program that can output appropriate explanatory information even when the prediction results of a learning model differ from intuitive results. [Means for solving the problem]

[0005] This disclosure provides a display device comprising: a learning unit that generates a learning model by training each of several evaluated persons with information about the evaluated person and training data that associates a plurality of explanatory variables with a target variable; a first SHAP value calculation unit that calculates a first SHAP value for each of the evaluated persons with a first SHAP value for each of the evaluated persons from the learning model and the training data; an interaction target extraction unit that extracts interacting features based on the first SHAP value for each of the explanatory variables; a second SHAP value calculation unit that calculates a prediction probability and a second SHAP value based on the learning model and inference data; and an output unit that generates and outputs explanatory information indicating the basis for calculating the second SHAP value based on the combination of explanatory variables that interact with the second SHAP value and the features.

[0006] Furthermore, this disclosure provides a display method performed by at least one processor, which generates a learning model by training each of a plurality of subjects with information about the subject and training data that associates a plurality of explanatory variables with a target variable; calculates a first SHAP value for each of the subject from the learning model and the training data; extracts interacting features based on the first SHAP value for each of the explanatory variables; calculates a prediction probability and a second SHAP value based on the learning model and inference data; and generates and outputs explanatory information showing the basis for calculating the second SHAP value based on the combination of explanatory variables that interact with the second SHAP value and the features.

[0007] The present disclosure also provides a display program executed by at least one processor, which causes the processor to implement the steps of: generating a trained model by training, for each of a plurality of evaluatees, training data in which the evaluatee's information and a plurality of explanatory variables are associated with an objective variable; calculating a first SHAP value for each of the explanatory variables for each of the evaluatees from the trained model and the training data; extracting feature values having interaction based on the first SHAP value for each of the explanatory variables; calculating a prediction probability and a second SHAP value based on the trained model and inference data; and generating and outputting explanatory information indicating a basis for calculation of the second SHAP value based on a combination of explanatory variables that interact with the second SHAP value and the feature values.

[0008] The present disclosure also provides a display device, comprising: a training unit that generates a trained model by training, for each of a plurality of evaluatees, training data in which the evaluatee's information and a plurality of explanatory variables are associated with an objective variable; a SHAP value calculation unit that calculates a SHAP value for each of the explanatory variables based on the trained model and data for score / SHAP value calculation; a prediction probability calculation unit that calculates a prediction probability for each evaluatee based on the data for score / SHAP value calculation; and an output unit that determines at least one explanatory variable for making the prediction probability close to or match a target prediction probability among the plurality of explanatory variables and a change amount of the explanatory variable based on the data for score / SHAP value calculation, the SHAP value, the prediction probability and settings configured for each evaluatee, and generates and outputs recommendation information including information about the determined explanatory variable and the change amount of the value of the explanatory variable.

[0009] Further, the present disclosure provides a display method performed by at least one processor, the method comprising: for each of a plurality of evaluatees, generating a trained model by training with learning data in which information of the evaluatee, a plurality of explanatory variables and an objective variable are associated with each other; calculating a SHAP value for each of the explanatory variables based on the trained model and score-SHAP value calculation data; calculating a prediction probability for each of the evaluatees based on the score-SHAP value calculation data; based on the score-SHAP value calculation data, the SHAP values, the prediction probabilities, and a target prediction probability set for each of the evaluatees, determining at least one explanatory variable among the plurality of explanatory variables for bringing the prediction probability close to or matching the target prediction probability, and a change amount of the explanatory variable; generating and outputting recommendation information including information of the determined explanatory variable and the change amount of the value of the explanatory variable.

[0010] Further, the present disclosure provides a display program executed by at least one processor, which causes the processor to implement the steps of: for each of a plurality of evaluatees, generating a trained model by training with learning data in which information of the evaluatee, a plurality of explanatory variables and an objective variable are associated with each other; calculating a SHAP value for each of the explanatory variables based on the trained model and score-SHAP value calculation data; calculating a prediction probability for each of the evaluatees based on the score-SHAP value calculation data; based on the score-SHAP value calculation data, the SHAP values, the prediction probabilities, and a target prediction probability set for each of the evaluatees, determining at least one explanatory variable among the plurality of explanatory variables for bringing the prediction probability close to or matching the target prediction probability, and a change amount of the explanatory variable; and generating and outputting recommendation information including information of the determined explanatory variable and the change amount of the value of the explanatory variable. Effects of the Invention

[0011] According to the present disclosure, appropriate explanatory information can be output corresponding to a case where a result different from intuition is output as a prediction result of a machine learning model. [Brief explanation of the drawing]

[0012] [Figure 1] Block diagram showing an example of the internal configuration of the display device according to Embodiment 1. [Figure 2] Block diagram showing an example of the internal configuration of the processor in Embodiment 1. [Figure 3] Flowchart showing an example of the operation procedure of the model-related processing unit in Embodiment 1. [Figure 4] Flowchart showing an example of the procedure for calculating the degree of deviation of the interaction target extraction unit in Embodiment 1. [Figure 5] Flowchart showing an example of the explanation text generation process procedure for the explanation text output unit in Embodiment 1. [Figure 6] A diagram showing an example of training data and SHAP value data in Embodiment 1. [Figure 7] Figure showing an example of SHAP interaction value data and interaction description target interval data in Embodiment 1. [Figure 8] A diagram showing an example of explanatory text data in Embodiment 1. [Figure 9] Figure showing an example of prediction results in Embodiment 1. [Figure 10] Figure showing an example of the display of the explanatory text in Embodiment 1. [Figure 11] Block diagram showing an example of the internal configuration of the processor in Embodiment 2. [Figure 12] Flowchart showing an example of the procedure for generating activity probability data and SHAP value data of the model-related processing unit in Embodiment 2. [Figure 13] Flowchart showing example 1 of the recommendation action generation procedure of the recommendation action generation unit in Embodiment 2. [Figure 14] This figure shows an example of training data and score / SHAP value calculation data in Embodiment 2. [Figure 15] Figure showing an example of SHAP value data and activity probability data in Embodiment 2. [Figure 16]Flowchart showing example 2 of the recommendation action generation procedure of the recommendation action generation unit in Embodiment 2. [Figure 17] Diagram showing examples of use cases for conventional display devices. [Figure 18] A diagram showing an example of SHAP values ​​and explanatory information. [Modes for carrying out the invention]

[0013] (Background leading to Embodiment 1) In recent years, there has been a demand for visualization of the process leading to prediction or estimation results obtained by learning models. Examples of conventional prediction results and explanations of prediction results using learning models will be explained with reference to Figures 17 and 18, respectively. Figure 17 is a diagram showing an example of a use case for a conventional display device. Figure 18 is a diagram showing an example of SHapley Additive exPlanation (hereinafter referred to as "SHAP") values ​​and explanatory information.

[0014] For example, a system could be conceivable in which, when learning data 2 is input to a learning model 1, which includes user information of the person being evaluated as explanatory variables such as "gender: male", "age: 30 years old", sleep information "sleep: 7.5 hours", diet information "meals: 2600kcal, nutritionally balanced", exercise information "exercise: moderate", and body composition information "body composition: medium build", the system would output a score 3 as a predicted probability, indicating the user's health status as a predicted result. In the example shown in Figure 17, score 3 would be output as, for example, "health score: 75".

[0015] In such cases, users want to know the basis for their predicted health score so they can understand what they need to improve to raise their score. Therefore, conventionally, a score screen SC1 that visualizes SHAP values ​​was sometimes output. SHAP, in this context, is a method for calculating the contribution of each feature (explanatory variable) to the predicted value (score), and is based on the concept of the Shapley value. In the examples shown in Figures 17 and 18, the SHAP values ​​are calculated values ​​that show how each feature (user information, sleep information, diet information, exercise information, and body composition information) included in the input training data 2 contributes to the output health score. The score screen SC1 shown in Figure 18 is a screen that includes score 3A, which visualizes the contribution of each explanatory variable's SHAP value to the health score shown in Figure 17. For example, since the user gets enough sleep, the SHAP value corresponding to sleep time is "+0.75". Also, since the user gets less exercise, the SHAP value corresponding to exercise is "-0.2".

[0016] However, it was difficult for the average user to interpret how the SHAP values ​​of each explanatory variable were calculated simply by looking at the SHAP values ​​shown in score 3A. Therefore, in this embodiment 1, explanatory information At11 is displayed to explain why the SHAP values ​​are what they are. The explanatory information At11 shown in Figure 18 is an explanatory text that explains why the SHAP value of the explanatory variable "sleep time" is "+0.75", and displays "It has a positive effect if sleep time is 7 hours or more".

[0017] However, SHAP values ​​are not always intuitively understandable to the user, such as the fact that a health score increases when sleep duration is 7 hours or more, and decreases when sleep duration is less than 7 hours. SHAP values ​​can sometimes take values ​​that contradict the user's common intuition, such as calorie intake. The appropriate daily calorie intake for a 30-year-old male is 2400-2800 kcal. Therefore, the SHAP value for calorie intake for a male who consumes 2600 kcal per day should be positive. However, in the score screen SC1, the actual SHAP value is negative. In this disclosure, cases where an item that should have a positive SHAP value actually has a negative value, or an item that should have a negative SHAP value actually has a positive value, are referred to as "counterintuitive." When counterintuitive results occur, there may be an interaction between this explanatory variable and one or more other explanatory variables.

[0018] Therefore, in the following Embodiment 1, a display device, display method, and display program will be described for displaying explanatory information, including the interactions between each explanatory variable, in the explanation of scores and SHAP values ​​obtained using multiple different explanatory variables.

[0019] Hereinafter, with reference to the drawings as appropriate, each embodiment specifically disclosing the display device, display method, and display program relating to this disclosure will be described in detail. However, unnecessarily detailed explanations may be omitted. For example, detailed explanations of already well-known matters or redundant explanations of substantially identical configurations may be omitted. This is to avoid unnecessarily redundancy in the following explanation and to facilitate understanding by those skilled in the art. The accompanying drawings and the following explanation are provided to enable those skilled in the art to fully understand this disclosure and are not intended to limit the subject matter described in the claims.

[0020] (Embodiment 1) Next, with reference to Figures 1 and 2, a display device P1 that displays a predicted performance score indicating the performance level of the person being evaluated will be described. Figure 1 is a block diagram showing an example of the internal configuration of the display device P1 according to Embodiment 1. Figure 2 is a block diagram showing an example of the internal configuration of the processor 11 in Embodiment 1.

[0021] The display device P1 includes a communication unit 10, a processor 11, and a memory 12. The display device P1 receives input training data 20 and predicts the evaluation of the person being evaluated by executing the processes described later in the processor 11. The prediction results are displayed by the display unit 13 and made visible to the user. The display unit 13 may be integrated with the display device P1 or it may be a separate unit. The training data 20 may be sent to the display device P1 from a training database or it may be stored in the memory 12 in advance.

[0022] The display device P1 in this disclosure may be implemented by, for example, a personal computer (hereinafter referred to as "PC"), a notebook PC, a tablet device, or a smartphone, or by, for example, an on-premise server or a cloud server.

[0023] The communication unit 10 is configured using a communication interface circuit for sending and receiving data or information to and from a learning database (not shown) that stores the learning data 20. The communication unit 10 is capable of receiving input of the learning data 20. The acquisition of the learning data 20 may be achieved by reading learning data stored on an external storage medium, or by receiving learning data 20 transmitted from an external device connected for data communication. The communication unit 10 outputs the learning data 20 to the processor 11.

[0024] The communications performed by the communications unit 10 are either wired communications or wireless communications. Wireless communications are those provided in accordance with wireless communication standards such as wireless Local Area Network (LAN), wireless Wide Area Network (WAN), 4G (fourth-generation mobile communication system), 5G (fifth-generation mobile communication system), or Wi-Fi (registered trademark).

[0025] The processor 11 is configured using, for example, a Central Processing Unit (CPU) or a Field Programmable Gate Array (FPGA), and works in cooperation with the memory 12 to perform various processing and control operations. Specifically, the processor 11 refers to the programs and data held in the memory 12 and executes those programs to realize various functions of the display device P1.

[0026] Memory 12 includes, for example, Random Access Memory (RAM) used as work memory when executing each process of the processor 11, and Read Only Memory (ROM) which stores programs and data that define the operation of the processor 11. Memory 12 may also include a storage device such as a Solid State Drive (SSD) or a Hard Disk Drive (HDD). Data or information generated or acquired by the processor 11 is temporarily stored in the RAM. The ROM contains programs that define the operation of the processor 11.

[0027] The display unit 13 is configured using, for example, a Liquid Crystal Display (LCD) or an organic electroluminescence (EL). The display unit 13 displays the score screen SC2 (see Figure 10) and other output information generated by the processor 11. The display unit 13 may be implemented by an external terminal (for example, a PC, notebook PC, tablet, or smartphone) that is communicatively connected to the display device P1.

[0028] Here, we will describe the processor 11 in detail. The processor 11 includes a model-related processing unit 111, an interaction target extraction unit 112, and an explanatory text output unit 113.

[0029] The model-related processing unit 111 acquires training data 20 from an external source or memory 12 and performs various processes using the trained model. The model-related processing unit 111 includes a training unit 1111 and a first SHAP value calculation unit 1112.

[0030] The learning unit 1111 receives training data containing multiple evaluation information for multiple individuals being evaluated (in this case, employees). The learning unit 1111 uses the multiple evaluation information as explanatory variables and generates an AI model by training it with a dataset of scores for at least one evaluation item (e.g., performance flag or performance probability) which is the target variable. The evaluation information referred to here is training data used to evaluate the individuals being evaluated, and may include, for example, self-development reviews, employment categories, performance evaluations for the past two years, job history, overseas work experience, or years spent at a particular rank.

[0031] The first SHAP value calculation unit 1112 calculates the first SHAP value for each explanatory variable of the person being evaluated based on the input AI model and training data 20. The person being evaluated is, for example, the subject of evaluation (prediction) of the performance flag, which is a prediction probability, and in this disclosure, is an employee. The explanatory variables (evaluation items) are, for example, the highest evaluation in the last two years, sociability, leadership, years of experience in the job, cooperativeness, stability, years of work experience, years of staying at the same grade, overseas work experience, job history, or ingenuity. It goes without saying that the explanatory variables mentioned above are just examples and are not limited thereto. Years of experience in the job, years of work experience, highest evaluation in the last two years, years of staying at the same grade, overseas work experience, and job history can be said to be attribute information of the person being evaluated, while sociability, leadership, cooperativeness, stability, or ingenuity can be said to be evaluation data of the person being evaluated.

[0032] The interaction target extraction unit 112, in calculating the first SHAP value corresponding to one explanatory variable from among the first SHAP values ​​of multiple explanatory variables, determines the first SHAP value that is determined to have a counterintuitive tendency from among the multiple first SHAP values ​​to be the explanatory variable for which explanatory text will be generated. The interaction target extraction unit 112 includes a data splitting unit 1121, a data trend calculation unit 1122, and an explanatory text target extraction unit 1123.

[0033] The data splitting unit 1121 divides the scores (features) of multiple evaluators for each explanatory variable into four sets of data by dividing them into quartiles. Of course, other division numbers such as octiles can also be used. Dividing by quartiles means arranging the features in order of magnitude and dividing them into four equal parts.

[0034] The data trend calculation unit 1122 compares the positive sample rate for each divided data set with the first SHAP value. The data trend calculation unit 1122 determines whether the relationship between the positive sample rate and the SHAP value is counterintuitive.

[0035] The explanatory text target extraction unit 1123 determines the first SHAP value that is determined to have a counterintuitive tendency among the multiple first SHAP values ​​as the explanatory variable for which the explanatory text will be generated.

[0036] The explanatory text output unit 113 generates and outputs explanatory information for the first SHAP values ​​corresponding to the explanatory variables determined to be interacting, based on the calculated combinations of the first SHAP values ​​with interactions and the information on the intervals in which the interactions occur. The explanatory information here refers to information that shows the basis (i.e., explainability) for the prediction of the first SHAP values ​​by the learning model, and is an explanatory text that explains how each explanatory variable influences the prediction of the first SHAP values. The explanatory text output unit 113 includes a SHAP value trend analysis unit 1131, a SHAP interaction value trend analysis unit 1132, and an explanatory text generation unit 1133.

[0037] The SHAP value trend analysis unit 1131 analyzes the trend of the second SHAP value of each explanatory variable based on the second SHAP value calculated from the inference database.

[0038] The SHAP interaction value trend analysis unit 1132 analyzes the trend of the second SHAP values ​​that have interactions from the calculated second SHAP values. In other words, the explanatory information includes sentences that explain the influence of combinations of other explanatory variables that have interactions with an explanatory variable that has a second SHAP value that is counterintuitive.

[0039] The description generation unit 1133 generates descriptions for each second SHAP value displayed on the score screen SC2 (see Figure 10) based on the descriptions for each second SHAP value generated by the SHAP value trend analysis unit 1131 and the SHAP interaction value trend analysis unit 1132, respectively.

[0040] <Example of operation procedure for the model-related processing unit> Next, an example of the operation procedure of the model-related processing unit 111 will be described with reference to Figure 3. Figure 3 is a flowchart showing an example of the operation procedure of the model-related processing unit in Embodiment 1. Note that each flowchart in this disclosure may be interpreted not only as a method flowchart but also as a program processing flowchart.

[0041] First, the model-related processing unit 111 retrieves training data 20 from the training database and inputs it into the AI ​​model to perform model training (St1). The training database may be an external device connected to the display device P1 for communication, or it may be stored in the memory 12.

[0042] The model-related processing unit 111 calculates a first SHAP value corresponding to each explanatory variable based on the model training results and the training data 20, and stores it in memory 12 (St2).

[0043] The interaction target extraction unit 112 performs a deviation degree calculation process using the calculated first SHAP value and the training data 20 (St3).

[0044] <Example of deviation calculation procedure> Next, with reference to Figure 4, an example of the procedure for calculating the degree of deviation of the interaction target extraction unit 112 will be described. Figure 4 is a flowchart showing an example of the procedure for calculating the degree of deviation of the interaction target extraction unit 112 in Embodiment 1.

[0045] The data splitting unit 1121 retrieves training data 20 from the training database and selects the features (explanatory variables) to be split (St11). The data splitting unit 1121 generates four decomposed data sets by splitting the selected features from among the multiple features contained in the training data 20 into quartiles (St12). In this disclosure, an example of splitting features into quartiles is shown, but the splitting process may be performed by other methods (e.g., octaltiles). Each split data set is stored in memory 12 for each feature (explanatory variable).

[0046] The data trend calculation unit 1122 calculates the average of the positive sample rate and the first SHAP value for each divided data set (St13). The data trend calculation unit 1122 combines the calculated positive sample rates and the first SHAP value for each of the four divided data sets for each feature (explanatory variable) and stores them in memory 12 (St14).

[0047] The data trend calculation unit 1122 compares the calculated positive example rate with the first SHAP value for each divided data set to determine whether the first SHAP value is positive when the positive example rate of the divided data is high, or negative when the proportion of positive examples is low (positive correlation), or whether the first SHAP value is negative when the positive example rate of the divided data is high, or positive when the proportion of positive examples is low (negative correlation) (St15).

[0048] If the data trend calculation unit 1122 determines that there is a positive correlation between the positive example rate and the SHAP value (St15, positive correlation), such that the first SHAP value increases as the positive example rate increases, it calculates a linear approximation function that approximates the first SHAP value with respect to the positive example rate using a linear function, and the Mean Absolute Error (hereinafter referred to as "MAE") based on the average values ​​of the positive example rate and the SHAP value (St16). The MAE calculated here is the mean of the absolute values ​​of the errors between the first SHAP value shown by the linear approximation function and the first SHAP value of the divided data. The data trend calculation unit 1122 stores the calculated linear approximation function and MAE in memory 12.

[0049] On the other hand, if the data trend calculation unit 1122 determines that there is a negative correlation between the rate of positive examples and the first SHAP value, such that the first SHAP value increases as the rate of positive examples decreases (St15, negative correlation), it refers to the memory 12 and calculates the absolute error between the actual first SHAP value and the first SHAP value calculated from the linear approximation function for the segmented data for which a negative correlation was determined (St17).

[0050] The explanatory text target extraction unit 1123 determines whether the absolute error calculated in step St17 is greater than or equal to the MAE calculated in step St16 if there is a negative correlation in a certain segmented data (St18).

[0051] If the explanatory text target extraction unit 1123 determines that the absolute error is greater than or equal to the MAE (St18, YES), it determines that there are other features (explanatory variables) that it can interact with, and stores them in memory 12 as features that it can interact with (St19).

[0052] On the other hand, if the explanatory text target extraction unit 1123 determines that the absolute error is not greater than or equal to the MAE (St18, NO), it determines that this feature (explanatory variable) is not subject to interaction and stores it in memory 12 as a feature not subject to interaction (St20).

[0053] This allows the interaction target extraction unit 112 to extract features (explanatory variables) that are objects with which interactions occur in the prediction results of the learning model, and that produce counterintuitive prediction results.

[0054] <Example of the procedure for generating explanatory text> Next, with reference to Figure 5, an example of the explanation text generation process procedure of the explanation text output unit 113 will be described. Figure 5 is a flowchart showing an example of the explanation text generation process procedure of the explanation text output unit 113 in Embodiment 1.

[0055] The SHAP value trend analysis unit 1131 uses the inference database and the AI ​​model to calculate and analyze the second SHAP value and SHAP interaction value for each explanatory variable and record them in the memory 12 (St21). The SHAP interaction value trend analysis unit 1132 also uses the inference database and the AI ​​model to calculate and analyze the second SHAP value data and the SHAP interaction value that interacts with it and record them in the memory 12 (St21).

[0056] The explanatory text generation unit 1133 uses the information of the explanatory variables that are the target of interaction explanatory text generation, obtained by the deviation degree calculation procedure in step St3, and the SHAP interaction value to determine the interval (partition interval) where counterintuitive correlations occur and the explanatory variables to be used to explain the second SHAP value. The explanatory text generation unit 1133 associates the multiple explanatory variables that have been determined to have a correlation effect with the interval (partition interval) where counterintuitive correlations occur, and generates interval data (interaction explanatory text target interval data) for which interaction explanatory texts are to be generated (St22). The explanatory text generation unit 1133 records the generated interaction explanatory text target interval data in memory 12.

[0057] The explanatory text generation unit 1133 generates an explanatory text for the SHAP interaction value, which is the result of the interaction between multiple different second SHAP values, and the interaction explanatory text target interval data, based on the SHAP interaction value, which is the result of the interaction between multiple different explanatory variables, in the interval where multiple different explanatory variables interact (St23).

[0058] Furthermore, the explanatory text generation unit 1133 generates an explanatory text for the SHAP value that shows the basis for calculating the second SHAP value, based on the second SHAP value data (St24).

[0059] The explanatory text generation unit 1133 combines the SHAP value explanatory text and the SHAP interaction value explanatory text generated for each explanatory variable to generate one explanatory text data for each second SHAP value (St25). The explanatory text generation unit 1133 stores the generated explanatory text data in memory 12 for each explanatory variable (St26).

[0060] As a result, the explanatory text output unit 113 can generate an explanatory text that explains the basis for calculating each second SHAP value. Therefore, by generating and outputting the explanatory text, the display device P1 can help the person being evaluated understand which evaluation items they should improve in order to improve their predicted probability (performance probability in this disclosure).

[0061] Next, the various data generated by the processor 11 will be described with reference to Figures 6 to 8. Figure 6 is a diagram showing an example of training data and SHAP value data (first SHAP value data and second SHAP value data) in Embodiment 1. Figure 7 is a diagram showing an example of SHAP interaction value data and interaction description target interval data in Embodiment 1. Figure 8 is a diagram showing an example of description data in Embodiment 1. Note that the various data shown in Figures 6 to 8 are examples and are not limited thereto.

[0062] The training data is data that stores the scores of the explanatory variables x1, x2, ..., xn (n: an integer greater than or equal to 2) used for training, and the flag value of the target variable Y. Note that the target variable Y may not only be an activity flag indicating that the evaluated person's activity level is above a threshold (1) and below a threshold (0), but may also be a score from 0 to 100 indicating the degree of the evaluated person's activity level. In the example shown in Figure 6, the training data shows that the evaluated person "0000001" with employee ID has an explanatory variable "x1 (sociability)" of 4.5, an explanatory variable "x2 (leadership)" of 3.3, an explanatory variable "x3 (stability)" of 4.0, and a target variable "Y (activity level)" of 0.

[0063] SHAP value data (first SHAP value data and second SHAP value data) is data that stores the SHAP values ​​of each explanatory variable x1, x2, ..., xn. In the example shown in Figure 6, the SHAP value data for employee "0000001" with employee ID shows that the SHAP value of explanatory variable "x1 (sociability)" is 0.5, the SHAP value of explanatory variable "x2 (leadership)" is -0.6, and the SHAP value of explanatory variable "x3 (stability)" is 0.2.

[0064] SHAP interaction value data stores the self-interaction values ​​of each explanatory variable x1, x2, ..., xn, along with the interaction explanatory variable and its interaction value. In the example shown in Figure 7, the explanatory variable "sociability" has a self-interaction value of "0.36", the interaction explanatory variable is "leadership", and the interaction value between the explanatory variable "leadership" and the explanatory variable "sociability" is "0.14". Note that the self-interaction value here is a value that indicates the influence of the explanatory variable on the prediction.

[0065] Interaction explanation interval data stores information about the explanatory variables for which interaction explanations are generated, and the intervals (scores) in which they interact. In the example shown in Figure 7, the explanatory variable "sociability" interacts with the interaction explanatory variable "leadership" in the interval "4.4~".

[0066] The explanatory text data stores sentences that explain the influence of each explanatory variable x1, x2, ..., xn on the inference results. In the example shown in Figure 8, the explanatory variable "sociability" stores the explanatory text "High sociability has a positive effect on predictions. High sociability and high leadership have an even more positive effect."

[0067] Next, examples of displaying prediction results and explanatory text will be described with reference to Figures 9 and 10, respectively. Figure 9 is a diagram showing an example of prediction results in Embodiment 1. Figure 10 is a diagram showing examples of displaying explanatory texts At21 and At22 in Embodiment 1. Note that the examples shown in Figures 9 and 10 are just examples and are not limited thereto.

[0068] The prediction result data SCR2 shown in Figure 9 indicates the performance of the prediction result (i.e., prediction probability) as predicted by the display device P1. Furthermore, the score screen SC2 shown in Figure 10 is displayed on the display unit 13 and visualizes a second SHAP value and explanatory text for each explanatory variable to assist in interpreting the prediction result score.

[0069] When the cursors Cr21 and Cr22 are positioned over the second SHAP value, or when the user selects the second SHAP value, the processor 11 overlays the explanatory texts At21 and At22 of the corresponding explanatory variables.

[0070] For example, in the example shown in Figure 10, when the cursor Cr21 is positioned over the second SHAP value of the explanatory variable "highest rating in the last two years", the processor 11 generates a score screen SC2 that visualizes the explanatory statement At21, which shows the basis for the prediction of the second SHAP value "+0.6" for the explanatory variable "highest rating in the last two years": "People with a highest rating of A / 4 or higher in the last two years will have a positive effect on their prediction score, while people with a highest rating of B / 3 or lower will have a negative effect."

[0071] Furthermore, in the example shown in Figure 10, when the cursor Cr22 is positioned over the explanatory variable "Years of work experience", the processor 11 generates a score screen SC2 that visualizes the explanatory statement At22, which shows the basis for the prediction of the second SHAP value "-0.2" for the explanatory variable "Years of work experience": "People with more than 25.0 years of work experience will have a negative prediction score, while those with less than 25.0 years will have a positive prediction score. When the years of work experience are less than 5.5 years, having work experience will have a more positive effect." Note that explanatory statement At22 is generated by combining explanatory statements At221 and At222 through an explanatory statement merging process.

[0072] As a result, the display device P1 in Embodiment 1 can generate not only explanatory text explaining the conditions and reasons under which each second SHAP value influences the prediction result in a counterintuitive manner, but also explanatory text explaining the conditions and reasons under which multiple second SHAP values ​​interact with each other and have a counterintuitive effect on the prediction result. Therefore, the person being evaluated can understand how to improve the explanatory variables to increase the prediction probability based on the conditions and reasons shown in the explanatory text.

[0073] Furthermore, the display device P1 according to Embodiment 1 generates not only explanatory text explaining the conditions and reasons under which each second SHAP value influences the conventional prediction results in a non-intuitive manner, but also explanatory text explaining the conditions and reasons under which multiple second SHAP values ​​interact with each other to influence the prediction results in a non-intuitive manner, in addition to explanatory text explaining the conditions and reasons under which multiple second SHAP values ​​interact with each other to influence the prediction results, in order to achieve various purposes such as weight maintenance, performance improvement, or improved marketing effectiveness.

[0074] (Background leading to Embodiment 2) In recent years, there has been a growing demand for learning models that not only explain the process leading to prediction and estimation results to the person being evaluated, but also provide guidance on how to efficiently improve the score, which represents the predicted probability. Specifically, the person being evaluated wants to receive concrete recommendations on which improvementable evaluation items (explanatory variables) to focus on in order to bring their current score up to a target value, thereby providing them with efficient and effective ways to improve their score.

[0075] Therefore, in Embodiment 2, the objective is to provide a display device, a display method, and a display program that can display recommendation information for each person being evaluated (employee) to improve the prediction probability of the prediction results of the learning model, which were devised in view of the conventional circumstances described above. In the following description, the same reference numerals are used for components that are the same as in Embodiment 1, and their explanations are omitted.

[0076] (Embodiment 2) Referring to Figure 11, the display device P1A according to Embodiment 2 will be described. Figure 11 is a block diagram showing an example of the internal configuration of the processor 11A in Embodiment 2.

[0077] The display device P1A according to Embodiment 2 generates and outputs recommendation information, which is the amount of change in explanatory variables necessary to improve the probability of success of the person being evaluated, based on training data that associates the person being evaluated with the features of each explanatory variable. The display device P1A includes a communication unit 10, a processor 11A, and a memory 12.

[0078] The processor 11A includes a model-related processing unit 111A and a recommendation action generation unit 114. The model-related processing unit 111A includes a learning unit 1111, a first SHAP value calculation unit 1112, and an activity probability calculation unit 1113.

[0079] The performance probability calculation unit 1113 (prediction probability calculation unit) calculates the performance probability for each person being evaluated, taking the AI ​​model (performance model) and the explanatory variables of the training data 20 as input. The performance model may be a model that calculates (predicts) the performance probability for a specific job type, position, organization, etc. This allows the performance probability calculation unit 1113 to generate recommendation actions to help the person being evaluated perform better in their job type, position, organization, etc.

[0080] The recommendation action generation unit 114 generates and outputs recommendation actions, which are recommendation information. The recommendation action generation unit 114 includes a data approximation function generation unit 1141, an evaluation function generation unit 1142, and an evaluation unit 1143.

[0081] The data approximation function generation unit 1141 generates approximation functions based on SHAP values. The data approximation function generation unit 1141 generates probability density functions from the training data 20 that show the relationship between explanatory variables and SHAP values. Details of each function will be described later.

[0082] The evaluation function generation unit 1142 receives the target performance probability (target predicted probability) and personnel dataset of the person being evaluated, set by the administrator, and evaluates the feasibility and efficiency of improving the SHAP value by the calculated amount based on the amount of change in SHAP value necessary to improve the predicted probability (performance probability in this disclosure) in order to achieve the target performance probability.

[0083] The evaluation unit 1143 determines the recommended action for each person being evaluated based on the evaluation results for feasibility and efficiency. The evaluation unit 1143 outputs the determined recommended action.

[0084] <Example of procedure for generating activity probability data and SHAP value data> Next, with reference to Figure 12, an example of the procedure for generating activity probability data and SHAP value data by the model-related processing unit 111A will be described. Figure 12 is a flowchart showing an example of the procedure for generating activity probability data and SHAP value data by the model-related processing unit 111A in Embodiment 2. Note that the following flowchart may also be interpreted as the processing of a display program. Also, the following equations such as functions are just examples, and different equations may be used.

[0085] First, the model-related processing unit 111A trains the training data 20 using a performance model to evaluate whether the person being evaluated can perform well (St41). The training data 20 is the same as in Embodiment 1 and has multiple explanatory variables and a target variable (performance flag). The model-related processing unit 111A calculates the performance probability p for each person being evaluated (employee) by inputting the explanatory variable data (score and SHAP value calculation data) from the training data (St42). The model-related processing unit 111A outputs the calculated performance probability p to the recommendation action generation unit 114.

[0086] Furthermore, the first SHAP value calculation unit 1112 of the model-related processing unit 111A takes the active model and score / SHAP value calculation data as input and calculates the SHAP value of each explanatory variable (St43). The SHAP value is calculated for each subject. The first SHAP value calculation unit 1112 outputs the calculated SHAP value of each explanatory variable to the data approximation function generation unit 1141. After processing in steps St42 and St43, the processor 11A proceeds to the flow shown in Figure 13 or the flow shown in Figure 16.

[0087] <Example of recommended action generation procedure 1> Next, with reference to Figure 13, an example of the recommendation action generation procedure 1 of the recommendation action generation unit 114 will be described. Figure 13 is a flowchart showing an example of the recommendation action generation procedure 1 of the recommendation action generation unit 114 in Embodiment 2. Note that the following flowchart may also be interpreted as processing of the display program. Also, the following equations such as functions are just examples, and different equations may be used.

[0088] Recommendation Action Generation Procedure Example 1 is a recommendation action generation procedure that is executed when determining the single explanatory variable that is best suited to improving the prediction probability based on the realism and efficiency of each explanatory variable x1, x2, ..., xn, and recommending the determined explanatory variable as the target for improvement.

[0089] The evaluation function generation unit 1142 calculates the increase (difference) in SHAP value necessary to improve the predicted probability of success, based on the calculated success probability p data, the person being evaluated specified (input) by the user (for example, a person who manages the evaluation results of the person being evaluated), and the target success probability p' of this person being evaluated. Based on the increase (difference) in SHAP value, the evaluation function generation unit 1142 calculates the required SHAP value improvement range ΔZ (St44). The required SHAP value improvement range ΔZ, which is the amount of change in the logit value, is the amount of change in SHAP value necessary to improve the predicted probability of success.

[0090] The required improvement range ΔZ for the SHAP value is calculated based on the following equations (Equations 1-1) to (Equations 1-3). Note that the logit function shown in (Equation 1-3) is a function that converts the probability of success p into the logit variable Z.

[0091]

number

[0092] The data approximation function generation unit 1141 generates an approximation function S from the dependency relationship between the calculated SHAP value and the values ​​of each explanatory variable included in the score-SHAP value calculation data. i The function is calculated (St45). The data approximation function generation unit 1141 calculates the probability density function f for each explanatory variable included in the score and SHAP value calculation data. i (x i Calculate (St46).

[0093] The evaluation function generation unit 1142 generates the SHAP value improvement range ΔZ and the approximation function S calculated by the data approximation function generation unit 1141. iBased on the above, for each explanatory variable xi, the required raw value improvement Δx required to increase the SHAP value by the required SHAP value improvement ΔZ i is calculated (St47). (Equation 2) represents the required raw value improvement Δx i is a calculation formula for calculation. x i is the current (evaluation value) of the explanatory variable xi. For the explanatory variable xi of the evaluated person, the data for score and SHAP value calculation is used.

[0094]

Formula

[0095] The evaluation function generation unit 1142 is configured to, based on the required SHAP value improvement ΔZ, the required raw value improvement Δx i and the probability density function f calculated by the data approximation function generation unit 1141 i (x i ), calculate the feasibility of increasing the required raw value improvement Δx of each explanatory variable xi i (St48). The feasibility is calculated by (Equation 3).

[0096]

Formula

[0097] Further, the evaluation function generation unit 1142 is configured to, based on the required SHAP value improvement ΔZ and the required raw value improvement Δx i , calculate the efficiency of increasing the required raw value improvement Δx of the explanatory variable xi among the plurality of explanatory variables i (St49). The efficiency is calculated by (Equation 4).

[0098]

Formula

[0099] Based on the calculated feasibility and efficiency, the evaluation function generation unit 1142 obtains the evaluation function Efficiency(x shown in (Equation 5) iUsing ), the required range of improvement Δx for the raw values ​​of each explanatory variable xi. i The evaluation function generator 1142 evaluates whether the improvement is efficient and realistic. The evaluation function generator 1142 uses information on the explanatory variable xi and the raw value of the corresponding explanatory variable xi to determine the required improvement range Δx. i The evaluation unit 1143 outputs the evaluation results, which assess whether the improvement is efficient and realistic.

[0100]

number

[0101] The evaluation unit 1143 generates an evaluation value list (St50) that associates an explanatory variable xi with the evaluation result of the corresponding explanatory variable xi. The evaluation unit 1143 stores the generated evaluation value list in memory 12.

[0102] The evaluation unit 1143 selects the explanatory variable xi with the highest evaluation (i.e., the most realistic or efficient) among the evaluation results of explanatory variables x1, x2, ..., xn as the highly-rated action (St51). Based on the selected highly-rated action, the evaluation unit 1143 executes a recommendation action that outputs a notification to the display unit 13 recommending that the person being evaluated improve the explanatory variable xi corresponding to the highly-rated action. For example, the evaluation unit 1143 generates a notification such as recommendation action ACT1 "By raising sociability from 4.3 to 4.6, the probability of success improves from 50% to 70%" and displays it on the display unit 13.

[0103] As described above, the display device P1A according to Embodiment 2 can present explanatory variables that are recommended for improvement in order to achieve an improvement in the predicted probability (in this disclosure, the probability of success) that is appropriate for each person being evaluated (for example, an employee or a manager), and the amount of change in the value of the explanatory variables necessary to achieve an improvement in the predicted probability.

[0104] Here, the evaluation unit 1143 evaluates the probability density function f i (x i If there are multiple explanatory variables with the same ) value, the required range of raw value improvement Δx iThis allows us to select an explanatory variable xi that improves the SHAP value more efficiently. Furthermore, efficiency (=ΔZ / Δx) i If there are multiple explanatory variables that have the same ) the probability density function f i (x i ) Required range of raw value improvement Δx i We can select an explanatory variable xi that has a higher probability of existence after the transition (i.e., after improvement). The evaluation function Efficiency(x) shown in (Equation 5) i The evaluation method for improving the required range Δxi of raw value improvement of the explanatory variable xi using ) is adopted as the simplest case for achieving improvement in the predicted probability of success by improving a single explanatory variable.

[0105] Here, another evaluation function Efficiency(x i This document describes an evaluation method using the evaluation function Efficiency(x i ) may also be the function shown in (Equation 6-1).

[0106]

number

[0107] The evaluation function shown in (Equation 6-1) is Efficiency(x i In ) P expresses reality after / P before When the value of is large, it is judged that the change in the probability of success (predicted probability) is more general and realistic within the data distribution (i.e., highly realistic). On the other hand, the evaluation function Efficiency(x i ) in P after / P before If the value is small, it is determined that the change in the probability of success (predicted probability) is rare within the data distribution and is likely unrealistic (i.e., unrealistic).

[0108] Thus, the evaluation function Efficiency(x) shown in (Equation 6-1) is expressed as follows: i When using ), the display device P1A is evaluated from a different perspective of feasibility.

[0109] The functions described above are merely examples and are not limited to them. Other functions may be used to calculate feasibility and efficiency.

[0110] Furthermore, the display device P1A in this disclosure may extract SHAP values ​​for each explanatory variable (evaluation item) for each group of evaluators using clustering of a dependency plot, and output the extracted SHAP values ​​in association with the recommended actions.

[0111] Furthermore, the display device P1A in this disclosure may generate recommendation actions that most efficiently improve the sum of the SHAP values ​​of the person being evaluated, using multiple activity models that can evaluate activity levels from different perspectives.

[0112] Next, the various data generated by the processor 11A will be described with reference to Figures 14 and 15, respectively. Figure 14 shows an example of training data and score / SHAP value calculation data in Embodiment 2. Figure 15 shows an example of SHAP value data and activity probability data in Embodiment 2. Note that the various data shown in Figures 14 and 15 are examples and are not limited thereto. Note that the training data shown in Figure 14 is the same as the training data shown in Figure 6, so its explanation will be omitted.

[0113] The data used to calculate the score and SHAP values ​​is the data used to calculate the SHAP values ​​for each explanatory variable x1, x2, ..., xn, and does not include predicted probabilities. In the example shown in Figure 14, the SHAP value data for employee "0000001" with employee ID shows that the SHAP value for explanatory variable "x1 (sociability)" is 4.5, the SHAP value for explanatory variable "x2 (leadership)" is 3.3, and the SHAP value for explanatory variable "x3 (stability)" is 4.0.

[0114] Note that the training data and the score / SHAP value calculation data may have the same explanatory variables. The processor 11A can basically train a model on a training dataset that has both explanatory and target variables, and perform inference on score / SHAP value calculation data that has only explanatory variables and no target variable.

[0115] The SHAP value data stores the SHAP values ​​for each explanatory variable x1, x2, ..., xn. In the example shown in Figure 15, the SHAP value data for the employee with employee ID "0000001" indicates that the SHAP value for explanatory variable "x1 (sociability)" is 0.5, the SHAP value for explanatory variable "x2 (leadership)" is -0.6, and the SHAP value for explanatory variable "x3 (stability)" is 0.2.

[0116] The performance probability data stores the predicted performance probability for each person being evaluated. In the example shown in Figure 15, the performance probability of the person being evaluated with employee ID "0000001" is "0.36".

[0117] The evaluation value list stores evaluation values ​​for each explanatory variable x1, x2, ..., xn, based on the feasibility and efficiency of improving the predicted probability of success through improvements to each explanatory variable. The evaluation value list is generated for each person being evaluated (in this case, an employee). In the example shown in Figure 15, the raw value improvement range Δx for the explanatory variable "sociability" is... i This indicates that improving the probability of success by increasing the value by "0.1" results in a feasibility and efficiency evaluation score of "0.2".

[0118] <Example of recommended action generation procedure 2> Next, with reference to Figure 16, an example of the recommendation action generation procedure 2 of the recommendation action generation unit 114 will be described. Figure 16 is a flowchart showing an example of the recommendation action generation procedure 2 of the recommendation action generation unit 114 in Embodiment 2. Note that the following flowchart may also be interpreted as processing of the display program. Also, the following equations such as functions are just examples, and different equations may be used.

[0119] Recommendation Action Generation Procedure Example 2 is a recommendation action generation procedure that is executed when determining the optimal combination of multiple explanatory variables for improving or enhancing the prediction probability based on the realism or efficiency of each of the explanatory variables x1, x2, ..., xn, and recommending each of the determined explanatory variables as a target for improvement. In this disclosure, for the sake of clarity, an example of determining a combination of two explanatory variables is described, but the number of explanatory variables to be combined may be three or more.

[0120] Note that the processing in steps St45-St46 and St48 of the recommendation action generation procedure example 2 shown in Figure 16 is the same as the processing in steps St45-St46 and St48 of the recommendation action generation procedure example 1 shown in Figure 13, so the explanation is omitted.

[0121] The evaluation function generation unit 1142 calculates the increase (difference) in SHAP value necessary to improve the target performance probability, based on the difference between the calculated performance probability p data and the target performance probability p' specified (input) by the user. Based on the increase (difference) in SHAP value, the evaluation function generation unit 1142 calculates the required SHAP value improvement range ΔZ (St64).

[0122] The evaluation function generation unit 1142 generates the evaluation function R(x) shown in (Equation 7). i ,Δx i The feasibility is evaluated (calculated) using the following evaluation function R(x) (St67). i ,Δx i ) is the probability of existence f of the modified explanatory variable xi. i (x i +Δx i ) is high, and the required range for raw value improvement is Δx i The smaller the value, the more realistic it is considered to be.

[0123]

number

[0124] Thus, the evaluation function R(x) shown in (Equation 7) is expressed as follows:i ,Δx i Even when using ), the display device P1A determines the required range Δx for improving the raw values ​​of each explanatory variable xi. i This allows us to evaluate whether the improvement is efficient and realistic. Note that the standard deviation σ in (Equation 7) i The explanatory variable x i This is the standard deviation.

[0125] The evaluation function generation unit 1142 calculates the increase (difference) in SHAP value necessary to improve the probability of success, based on the calculated success probability p data, the person to be evaluated specified (input) by the user, and the target success probability of this person. Based on the increase (difference) in SHAP value, the evaluation function generation unit 1142 calculates the required SHAP value improvement range ΔZ (St64).

[0126] The evaluation function generation unit 1142 calculates a correlation matrix C between multiple different explanatory variables xi and xj in order to consider the correlation between multiple different explanatory variables, based on the calculated realism and efficiency (St69). Here, j is an integer greater than or equal to 2, and i ≠ j.

[0127] The evaluation function generation unit 1142 generates the evaluation function based on the realism and efficiency of the explanatory variables xi and xj, and the elements C of the correlation matrix C. ij Based on this, an evaluation function Q is generated (St70).

[0128]

number

[0129] Here, the parameters α, β, and γ in (Equation 8) are each of the elements C ij These are parameters that adjust the weights. Each of the parameters α, β, and γ has a positive contribution from the correlation term, making it easy to change between the highly correlated explanatory variables xi and xj.

[0130] The evaluation unit 1143 formulates an optimization problem for the combination of multiple explanatory variables xi and xj that maximizes the evaluation function Q, based on (Equation 9-1) and (Equation 9-2), respectively. By solving the optimization problem, the evaluation unit 1143 calculates the optimal combination of multiple explanatory variables xi and xj, taking into account feasibility, efficiency, and the correlation between the explanatory variables xi and xj, while satisfying the required SHAP value improvement range ΔZ necessary to achieve the target performance probability p' (St71).

[0131]

number

[0132] The evaluation unit 1143 performs a recommendation action that outputs a notification to the display unit 13 recommending improvements to the explanatory variables corresponding to the highly rated actions, based on the calculated combination of multiple explanatory variables xi and xj and the required improvement range ΔZ of the SHAP values ​​of each of the multiple explanatory variables xi and xj necessary to achieve the target success probability p'. For example, the evaluation unit 1143 generates a notification such as recommendation action ACT2 "By improving sociability from 4.3 to 4.4 and leadership from 4.5 to 4.6, the success probability will improve from 50% to 70%" and displays it on the display unit 13.

[0133] As described above, the display device P1A according to Embodiment 2 can present a combination of multiple explanatory variables that are recommended for improvement in order to achieve an improvement in the predicted probability (in this disclosure, the probability of success) that is suitable for each person being evaluated (for example, an employee or a manager), and the amount of change in the value of each explanatory variable necessary to achieve an improvement in the predicted probability (in this disclosure, the probability of success).

[0134] Furthermore, while the display device P1A according to Embodiment 2 presents a combination of multiple explanatory variables recommended for improvement in order to achieve an improved probability of success (predicted probability) in a specific occupation, position, or organization, and the amount of change in the value of each explanatory variable necessary to achieve an improved predicted probability (in this disclosure, the probability of success), its use is not limited to this. The display device P1A can also present a combination of multiple explanatory variables recommended for improvement in order to achieve an improved predicted probability, and the amount of change in the value of each explanatory variable necessary to achieve an improved predicted probability, according to various other purposes such as weight maintenance, performance improvement, or improved marketing effectiveness.

[0135] (Note) Based on the descriptions of the embodiments described above, the following technologies are disclosed.

[0136] (Technology 1-1) A learning unit 1111 generates a learning model by training each of the multiple individuals being evaluated with information about the individuals being evaluated (e.g., employee ID, etc.) and training data 20 that associates multiple explanatory variables with the target variable. A first SHAP value calculation unit 1112 calculates a first SHAP value for each of the explanatory variables for each of the evaluated individuals from the learning model and the learning data 20, Based on the first SHAP value for each of the aforementioned explanatory variables, the interaction target extraction unit 112 extracts interacting features, A second SHAP value calculation unit (SHAP value trend analysis unit 1131) calculates the predicted probability and a second SHAP value based on the learning model and inference data, The system includes an output unit (explanatory text output unit 113) that generates and outputs explanatory information indicating the basis for calculating the second SHAP value, based on the combination of explanatory variables that interact with the second SHAP value and the feature quantities. Display device P1. As a result, the display device P1 according to Embodiment 1 can generate and output explanatory text showing the basis for the prediction of explanatory items for explanatory items in which the prediction results differ from intuition among the prediction results of the machine learning model. Therefore, the person being evaluated can understand how to improve the explanatory variables to increase the prediction probability (in this case, the SHAP value) based on the conditions and reasons shown in the explanatory text.

[0137] (Technology 1-2) The aforementioned explanatory variable includes at least one of the subject's attribute information or evaluation data. Display device P1 as described in (Technical 1-1). As a result, the display device P1 according to Embodiment 1 can generate an explanatory text that explains the basis for predicting (calculating) SHAP values ​​obtained from factors such as years of work experience, experience working away from home, years of working overseas, sociability, leadership, and stability.

[0138] (Technology 1-3) The aforementioned target variable is the activity flag. Display device P1 as described in (Technical 1-1) or (Technical 1-2). As a result, the display device P1 according to Embodiment 1 can generate an explanatory text that explains the basis for the prediction (calculation) of each SHAP value used in an evaluation in which the objective variable is whether or not the person being evaluated is a person who is active in a predetermined position or environment.

[0139] (Technology 1-4) The explanatory information is a sentence that explains the basis for generating the predicted probability for each of the explanatory variables. Display device P1 as described in any one of (Technology 1-1) to (Technology 1-3). As a result, the display device P1 according to Embodiment 1 can generate explanatory texts that indirectly improve the prediction probability by generating explanatory texts that explain the basis for predicting (calculating) the SHAP values ​​of the explanatory variables. Since the person being evaluated can understand the actions to take to improve each SHAP value through the explanatory texts, they can understand how to improve the explanatory variables to increase the prediction probability based on the calculation conditions of the SHAP values ​​indicated in the explanatory texts for each SHAP value.

[0140] (Technology 1-5) The explanatory information includes, in the second SHAP value for each explanatory variable, a sentence explaining the basis for calculating the second SHAP value by combining it with other explanatory variables that interact with the explanatory variable having a second SHAP value that is counterintuitive. Display device P1 as described in any one of (Technology 1-1) to (Technology 1-4). As a result, the display device P1 according to Embodiment 1 can further generate explanatory text that describes the conditions and reasons for counterintuitive effects caused by the interaction of multiple SHAP values. Therefore, the person being evaluated can understand how to improve the explanatory variables to increase the prediction probability based on the conditions and reasons shown in the explanatory text.

[0141] (Technology 1-6) A display method performed by at least one processor 11, For each of the multiple individuals being evaluated, a learning model is generated by training the model with training data 20 that associates the information of the evaluated individual with multiple explanatory variables and the target variable. From the learning model and the training data, a first SHAP value is calculated for each of the explanatory variables for each of the individuals being evaluated. Based on the first SHAP value for each of the aforementioned explanatory variables, interacting features are extracted. Based on the aforementioned learning model and inference data, the prediction probability and the second SHAP value are calculated. Based on the combination of explanatory variables that interact with the second SHAP value and the feature quantities, explanatory information showing the basis for calculating the second SHAP value is generated and output. Display method. As a result, the display device P1 according to Embodiment 1 can generate and output explanatory text showing the basis for the prediction of explanatory items for explanatory items in which the prediction results differ from intuition among the prediction results of the machine learning model. Therefore, the person being evaluated can understand how to improve the explanatory variables to increase the prediction probability based on the conditions and reasons shown in the explanatory text.

[0142] (Technology 1-7) A display program performed by at least one processor 11, The steps include: generating a learning model by training each of the multiple individuals being evaluated with training data 20 that associates the information of the individuals being evaluated with multiple explanatory variables and the target variable; A step of calculating a first SHAP value for each of the explanatory variables for each of the individuals being evaluated, from the learning model and the training data. The steps include extracting interacting features based on the first SHAP values ​​for each of the aforementioned explanatory variables, The steps include calculating the predicted probability and a second SHAP value based on the learning model and inference data, To realize the steps of generating and outputting explanatory information that shows the basis for calculating the second SHAP value, based on the combination of explanatory variables that interact with the second SHAP value and the feature quantities, Display program. As a result, the display device P1 according to Embodiment 1 can generate and output explanatory text showing the basis for the prediction of explanatory items for explanatory items in which the prediction results differ from intuition among the prediction results of the machine learning model. Therefore, the person being evaluated can understand how to improve the explanatory variables to increase the prediction probability (in this case, the SHAP value) based on the conditions and reasons shown in the explanatory text.

[0143] (Technology 2-1) A learning unit 1111 generates a learning model by training it with training data 20 that associates information about each of the multiple people being evaluated (e.g., employee ID, etc.) and multiple explanatory variables (e.g., aptitude evaluation value, amount of work experience, etc.) with an objective variable (performance flag), A SHAP value calculation unit (first SHAP value calculation unit 1112) calculates the SHAP value for each explanatory variable based on the learning model and score / SHAP value calculation data, Based on the score and SHAP value calculation data, a prediction probability calculation unit (performance probability calculation unit 1113) calculates the prediction probability for each person being evaluated, The system includes an output unit (recommendation action generation unit 114) that determines at least one explanatory variable and the amount of change of the explanatory variable from among the plurality of explanatory variables to bring the predicted probability closer to or match the target predicted probability, based on the score / SHAP value calculation data, the SHAP value, the predicted probability, and the target predicted probability set for each person being evaluated, and generates and outputs recommendation information including the determined explanatory variable and the amount of change of the value of the explanatory variable. Display device P1A. As a result, the display device P1A according to Embodiment 2 can present explanatory variables and the range of improvement that are recommended for improvement in order to achieve a target of predicted probability (in this disclosure, success probability) that is appropriate for each person being evaluated (for example, an employee or a manager).

[0144] (Technology 2-2) The output unit (recommendation action generation unit 114) determines the explanatory variables based on the efficiency and feasibility of bringing the predicted probability closer to or matching the target predicted probability. Display device P1A as described in (Technical 2-1). As a result, the display device P1A according to Embodiment 2 has a required range Δx for improving the raw values ​​of each explanatory variable xi. iBased on the efficiency and feasibility of improvement, the explanatory variables recommended for improvement and the extent of improvement can be presented in order to achieve the target of the predicted probability (in this disclosure, the probability of success). Therefore, the display device P1A can present recommendation information for each person being evaluated to achieve the target.

[0145] (Technology 2-3) The efficiency is calculated based on the increase in the SHAP value of the explanatory variable in response to an increase in the explanatory variable. Display device P1A as described in (Technical 2-2). As a result, the display device P1A according to Embodiment 2 has a required range Δx for improving the raw values ​​of each explanatory variable xi. i The efficiency of improving this can be evaluated.

[0146] (Technology 2-4) The aforementioned reality is calculated based on the magnitude of the probability of existence of the explanatory variable necessary for the prediction probability and the target prediction probability to match in the distribution of the explanatory variable. Display device P1A as described in (Technical 2-2) or (Technical 2-3). As a result, the display device P1A according to Embodiment 2 has a required range Δx for improving the raw values ​​of each explanatory variable xi. i We can assess the feasibility of improving this.

[0147] (Technology 2-5) The aforementioned explanatory variable includes at least one of the subject's attribute information or evaluation data. Display device P1A as described in any one of (Technology 2-1) to (Technology 2-4). As a result, the display device P1A according to Embodiment 2 can help improve the predicted probability based on factors such as years of work experience, experience of working away from home, years of working overseas, sociability, leadership, and stability.

[0148] (Technology 2-6) The aforementioned predicted probability is the probability of success. The aforementioned target predicted probability is the target success probability. Display device P1A as described in any one of (Technology 2-1) to (Technology 2-5). As a result, the display device P1A according to Embodiment 2 can support the improvement of the probability of success, indicating whether or not the person being evaluated is a person who will be successful in a predetermined job, position, or organization.

[0149] (Technology 2-7) A display method performed by at least one processor 11A, For each of the multiple individuals being evaluated, a learning model is generated by training it with training data 20 that associates the information of the person being evaluated (e.g., employee ID, etc.) and multiple explanatory variables (e.g., aptitude evaluation value, amount of work experience, etc.) with the target variable (performance flag). Based on the aforementioned learning model and data for calculating scores and SHAP values, the SHAP values ​​for each explanatory variable are calculated. Based on the aforementioned score and SHAP value calculation data, the predicted probability is calculated for each person being evaluated. Based on the score / SHAP value calculation data, the SHAP value, the predicted probability, and the target predicted probability set for each person being evaluated, the system determines at least one explanatory variable and the amount of change of the explanatory variable among the plurality of explanatory variables to bring the predicted probability closer to or match the target predicted probability, and generates and outputs recommendation information including the determined explanatory variable and the amount of change of the value of the explanatory variable. Display method. As a result, the display device P1A according to Embodiment 2 can present explanatory variables and the range of improvement that are recommended for improvement in order to achieve a target of predicted probability (in this disclosure, success probability) that is appropriate for each person being evaluated (for example, an employee or a manager).

[0150] (Technology 2-8) A display program executed by at least one processor 11A, For each of the multiple individuals being evaluated, a learning model is generated by training it with training data 20 that associates the information of the person being evaluated (e.g., employee ID, etc.) and multiple explanatory variables (e.g., aptitude evaluation value, amount of work experience, etc.) with the target variable (performance flag). The steps include: calculating the SHAP value for each explanatory variable based on the learning model and the score / SHAP value calculation data; The steps include: calculating the probability of success for each person being evaluated based on the score and SHAP value calculation data; To realize the steps of determining at least one explanatory variable and the amount of change of the explanatory variable from among the plurality of explanatory variables to bring the predicted probability closer to or match the target predicted probability, based on the score / SHAP value calculation data, the SHAP value, the predicted probability, and the target predicted probability set for each person being evaluated, and generating and outputting recommendation information including the determined explanatory variable and the amount of change of the value of the explanatory variable, Display program. As a result, the display device P1A according to Embodiment 2 can present explanatory variables that are recommended for improvement in order to achieve a target of predicted probability (in this disclosure, success probability) that is appropriate for each person being evaluated (for example, an employee or a manager).

[0151] Although various embodiments have been described above with reference to the attached drawings, this disclosure is not limited to such examples. It will be clear to those skilled in the art that various modifications, alterations, substitutions, additions, deletions, and equivalents can be conceived within the scope of the claims, and these will also be understood to fall within the technical scope of this disclosure. Furthermore, the components of the various embodiments described above can be combined arbitrarily without departing from the spirit of the invention. [Industrial applicability]

[0152] This disclosure is useful as a display device, display method, and display program that can output appropriate explanatory information even when the prediction results of a learning model differ from intuitive results. [Explanation of Symbols]

[0153] 11,11A processor 111,111A Model-related processing unit 1111 Learning Department 1112 First SHAP value calculation unit 1113 Probability of Success Calculation Unit 112 Interaction target extraction unit 1121 Data partitioning section 1122 Data Trend Calculation Unit 1123 Description text target extraction unit 113 Explanation Output Section 1131 SHAP Value Trend Analysis Department 1132 SHAP Interaction Value Trend Analysis Department 1133 Description Generation Unit 114 Recommendation Action Generation Unit 1141 Data Approximation Function Generation Unit 1142 Evaluation function generation unit 1143 Evaluation Department 12 memory 13 Display section 20 Training Data P1,P1A Display device

Claims

1. A learning unit that generates a learning model by training each of several individuals being evaluated with training data that associates the information of the individuals being evaluated with multiple explanatory variables and a target variable, A first SHAP value calculation unit calculates a first SHAP value for each of the explanatory variables for each of the evaluated individuals from the learning model and the learning data, An interaction target extraction unit extracts interacting features based on the first SHAP value for each of the aforementioned explanatory variables, A second SHAP value calculation unit calculates the predicted probability and a second SHAP value based on the learning model and inference data, The system includes an output unit that generates and outputs explanatory information indicating the basis for calculating the second SHAP value, based on a combination of explanatory variables that interact with the second SHAP value and the feature quantities. Display device.

2. The aforementioned explanatory variable includes at least one of the subject's attribute information or evaluation data. The display device according to claim 1.

3. The aforementioned target variable is the activity flag. The display device according to claim 1 or 2.

4. The explanatory information is a sentence that explains the basis for generating the predicted probability for each of the explanatory variables. The display device according to claim 1 or 2.

5. The explanatory information includes, in the second SHAP value for each explanatory variable, a sentence explaining the basis for calculating the second SHAP value by combining it with other explanatory variables that interact with the explanatory variable having a second SHAP value that is counterintuitive. The display device according to claim 1 or 2.

6. A display method performed by at least one processor, For each of the multiple individuals being evaluated, a learning model is generated by training it with training data that associates the information of the person being evaluated with multiple explanatory variables and the target variable. From the learning model and the training data, a first SHAP value is calculated for each of the explanatory variables for each of the individuals being evaluated. Based on the first SHAP value for each of the aforementioned explanatory variables, interacting features are extracted. Based on the aforementioned learning model and inference data, the prediction probability and the second SHAP value are calculated. Based on the combination of explanatory variables that interact with the second SHAP value and the feature quantities, explanatory information showing the basis for calculating the second SHAP value is generated and output. Display method.

7. A display program run by at least one processor, A step of generating a learning model by training a learning model on each of the multiple individuals being evaluated, by training the model with training data that associates the information of the person being evaluated with multiple explanatory variables and the target variable, A step of calculating a first SHAP value for each of the explanatory variables for each of the persons being evaluated from the learning model and the learning data, The steps include extracting interacting features based on the first SHAP value for each of the aforementioned explanatory variables, The steps include calculating the predicted probability and a second SHAP value based on the learning model and inference data, To realize the steps of generating and outputting explanatory information that shows the basis for calculating the second SHAP value, based on the combination of explanatory variables that interact with the second SHAP value and the feature quantities, Display program.

8. A learning unit generates a learning model by training each of the multiple individuals being evaluated with training data that associates the information of the individuals being evaluated with multiple explanatory variables and an objective variable. A SHAP value calculation unit calculates the SHAP value for each explanatory variable based on the learning model and score / SHAP value calculation data, A prediction probability calculation unit calculates the prediction probability for each person being evaluated based on the score / SHAP value calculation data, The system includes an output unit that, based on the score / SHAP value calculation data, the SHAP value, the predicted probability, and the target predicted probability set for each person being evaluated, determines at least one explanatory variable and the amount of change of the explanatory variable from among the plurality of explanatory variables to bring the predicted probability closer to or match the target predicted probability, and generates and outputs recommendation information including the determined explanatory variable and the amount of change of the value of the explanatory variable. Display device.

9. The output unit determines the explanatory variables based on the efficiency and feasibility of bringing the predicted probability closer to or matching the target predicted probability. The display device according to claim 8.

10. The efficiency is calculated based on the increase in the SHAP value of the explanatory variable in response to an increase in the explanatory variable. The display device according to claim 9.

11. The aforementioned reality is calculated based on the magnitude of the probability of existence of the explanatory variable necessary for the prediction probability and the target prediction probability to match in the distribution of the explanatory variable. The display device according to claim 9 or 10.

12. The aforementioned explanatory variable includes at least one of the subject's attribute information or evaluation data. The display device according to any one of claims 8 to 11.

13. The aforementioned predicted probability is the probability of success. The aforementioned target predicted probability is the target success probability. The display device according to claim 8.

14. A display method performed by at least one processor, For each of the multiple individuals being evaluated, a learning model is generated by training it with training data that associates the information of the person being evaluated with multiple explanatory variables and the target variable. Based on the aforementioned learning model and data for calculating the score and SHAP values, the SHAP value for each explanatory variable is calculated. Based on the data used to calculate the score and SHAP value, the predicted probability is calculated for each person being evaluated. Based on the score / SHAP value calculation data, the SHAP value, the predicted probability, and the target predicted probability set for each person being evaluated, the system determines at least one explanatory variable and the amount of change of the explanatory variable among the multiple explanatory variables to bring the predicted probability closer to or match the target predicted probability, and generates and outputs recommendation information including the determined explanatory variable and the amount of change of the value of the explanatory variable. Display method.

15. A display program that runs on at least one processor, A step of generating a learning model by training a learning model on each of the multiple individuals being evaluated, by training the model with training data that associates the information of the person being evaluated with multiple explanatory variables and the target variable, A step of calculating the SHAP value for each explanatory variable based on the learning model and the data for calculating the score and SHAP value, The steps include: calculating the predicted probability for each person being evaluated based on the score / SHAP value calculation data; To realize the steps of determining at least one explanatory variable and the amount of change of the explanatory variable from among the plurality of explanatory variables in order to bring the predicted probability closer to or match the target predicted probability, based on the score / SHAP value calculation data, the SHAP value, the predicted probability, and the target predicted probability set for each person being evaluated, and generating and outputting recommendation information including the determined explanatory variable and the amount of change of the value of the explanatory variable, Display program.

Images