Generation apparatus, generation method, and generation program

Abstract

To optimize annotation selection.SOLUTION: A generation device comprises a processor that executes a program and a storage device that stores the program. The generation device stores behavior information in which, for each of factors in a factor group, the factor is associated with a behavior when the factor is applicable. The processor executes: acquisition processing of acquiring, for each of samples, a predicted probability based on whether or not each factor in the factor group is applicable and an importance level of each factor in the factor group which is a basis for the predicted probability; extraction processing of extracting a specific factor from the factor group on the basis of the importance level acquired by the acquisition processing; and generation processing of acquiring, from the behavior information, a specific behavior corresponding to the specific factor extracted by the extraction processing and generating annotation information that presents the specific behavior to the sample to which the specific factor is applicable.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a generation device, a generation method, and a generation program for generating information.

Background Art

[0002] The following Patent Document 1 discloses a model-assisted annotation system. This model-assisted annotation system is operable to receive a first set of annotation data for a first set of medical scans from a set of client devices. A computer vision model is trained using the first set of medical scans and the first set of annotation data. A second set of annotation data for a second set of medical scans is generated by using the computer vision model. The second set of medical scans and the second set of annotation data are transmitted to the set of client devices, and in response, a set of additional annotation data is received. An updated computer vision model is generated using the set of additional annotation data. A third set of annotation data is generated for a third set of medical scans by using the updated computer vision model for transmission to a set of client devices for display.

[0003] The following Patent Document 2 discloses a counterfeit detection system. This counterfeit detection system includes one or more data stores for storing images of genuine items and a processor. The processor executes instructions to obtain an input image, determine a difference between a region of an image of a genuine item and a corresponding region of the input image by a discriminator of a generative adversarial network (GAN), generate a classification as to whether the input image is an image of a genuine item based on the determined difference by the discriminator, identify a corresponding region of the input image that contributed to the classification by a class activation module (CAM), obtain an annotation by the CAM as to whether the corresponding region indicates whether the input image is an image of a genuine item, and retrain the discriminator and / or the CAM based on the annotation. ​​Patent Document 3 below discloses a system for reviewing medical images. This system includes an electronic processor configured to display electronic medical images, collect (compile) clinical information associated with the electronic medical images, determine the probability of a disease associated with a patient associated with the electronic medical image based on the collected clinical information, and display the probability of the disease together with the medical image. The electronic processor is also configured to receive annotations related to the electronic medical images, determine an updated probability of the disease based on the clinical information and annotations, and display the updated probability of the disease.

[0005] Patent Document 4 below discloses a healthcare risk estimation system. This healthcare risk estimation system includes a risk-related term collection unit that includes terms related to risks in the form of potential disabilities, terms related to risk factors that increase the likelihood of disability, and terms related to treatments for medical conditions; a medical entity reconciliation unit that uses a standard vocabulary of terms to standardize and expand physician terms to include synonyms and related terms; a topic detection and tagging unit that retrieves a set of documents linked to the expanded terms from a medical document database; a NERD module that extracts entities from the set of documents and adjusts each document to a standardized vocabulary; and a relationship extraction unit that scores the relationships between entities based on the co-occurrence of two entities in the retrieved set of documents, and generates a risk knowledge graph that stores these relationships. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] U.S. Patent Application Publication No. 2020 / 0160979 [Patent Document 2] U.S. Patent Application Publication No. 2019 / 0236614 [Patent Document 3] International Publication No. 2018 / 042272 [Patent Document 4] Japanese Patent Publication No. 2017-174406 [Overview of the project] [Problems that the invention aims to solve]

[0007] However, Patent Documents 1 to 4 do not consider annotation selection based on key factors extracted from a set of factors of explainable artificial intelligence (XAI).

[0008] The present invention aims to optimize annotation selection. [Means for solving the problem]

[0009] A generating device that represents one aspect of the invention disclosed in this application is a generating device having a processor that executes a program and a storage device that stores the program, wherein for each factor of the factor group Risk reduction behavior information that associates the aforementioned factors with the risk reduction behaviors recommended for the sample when the aforementioned factors apply. The processor stores the following: The aforementioned For the sample, predictive accuracy based on whether each factor in the aforementioned factor group is applicable. rate The importance of each factor in the aforementioned group of factors that serve as the basis degree An acquisition process for obtaining information, and an extraction process for extracting specific factors from the group of factors whose importance is less than a threshold value obtained by the acquisition process, and Risk reduction From the behavioral information, a specific factor corresponding to the specific factor extracted by the extraction process Risk reduction The behavior is acquired and the sample corresponding to the specific factor is assigned the specific Risk reduction The system is characterized by performing a generation process that generates annotation information that indicates an action. [Effects of the Invention]

[0010] According to a typical embodiment of the present invention, the selection of annotations can be optimized. Other issues, configurations, and effects not mentioned above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 is an explanatory diagram showing an example of generation of annotation information. [Figure 2] Figure 2 is a block diagram showing an example of the hardware configuration of a computer. [Figure 3] Figure 3 is an explanatory diagram showing an example of the machine learning dataset shown in Figure 1. [Figure 4] Figure 4 is an explanatory diagram showing an example of improvement action information. [Figure 5] Figure 5 is an explanatory diagram showing an example of risk reduction action information. [Figure 6] Figure 6 is a flowchart showing an example of an annotation information generation processing procedure by a generation device. [Figure 7] Figure 7 is an explanatory diagram showing an example of an importance matrix. [Figure 8] Figure 8 is an explanatory diagram showing Example 1 of the coupling information according to Example 1. [Figure 9] Figure 9 is an explanatory diagram showing Example 2 of the coupling information according to Example 1. [Figure 10] Figure 10 is an explanatory diagram showing Example 1 of annotation information. [[ID=�4]] [Figure 11] Figure 11 is an explanatory diagram showing Example 2 of annotation information. [Figure 12] Figure 12 is an explanatory diagram showing Example 3 of annotation information. [Figure 13] Figure 13 is an explanatory diagram showing an example of risk reduction action information. [Figure 14] Figure 14 is an explanatory diagram showing an example of the coupling information according to Example 2. [Figure 15] Figure 15 is an explanatory diagram showing Example 1 of the change of the annotation of the coupling information. [Figure 16] Figure 16 is an explanatory diagram showing Example 2 of the change of the annotation of the coupling information. [Figure 17] Figure 17 is an explanatory diagram showing an example of combined action information. [Modes for carrying out the invention] [Examples]

[0012] <Example of generating annotation information> Figure 1 is an explanatory diagram showing an example of annotation information generation. During training, XAI100 is trained using the machine learning dataset 101. The machine learning dataset 101 is a combination of a target variable 111 called the ground truth data and explanatory variables 112 called the training data. The explanatory variables 112 are data that have the values ​​of each factor in a factor group for each sample (patients in this example). Factors are, for example, various test values ​​for patients such as BMI (Body Mass Index) and HDL (High Density Lipoprotein).

[0013] The target variable 111 is the result corresponding to the factor value for each sample. In this example, it indicates independence or requiring care. XAI100 is trained to minimize the value of the loss function based on the difference between the output result obtained when explanatory variable 112 is input to XAI100 and the target variable 111.

[0014] During prediction, XAI100 receives the data to be predicted 102 as input. The data to be predicted 102 has the same factor group values ​​as the explanatory variable 112. The data to be predicted 102 may be at least a part of the explanatory variable 112. In this example, for the sake of explanation, the explanatory variable 112 is used as the data to be predicted 102.

[0015] When the data to be predicted 102 is input to XAI100, it outputs an importance matrix 103. The importance matrix 103 includes the predicted probability 131 of the dependent variable 111 and the importance 132 of the independent variable 112. The importance 132 of the independent variable 112 is information that shows the basis for XAI100's calculation of the predicted probability 131 of the dependent variable 111.

[0016] Behavioral information 104 is information that associates the range of values ​​that each factor constituting the explanatory variable 112 can take with the recommended guidelines for the target population that has the predicted data 102. In this example, the target population is patients, and the recommended guidelines are the actions recommended for those patients (such as quitting smoking).

[0017] The annotation information generation function 110 generates annotation information 120 for the prediction target using the importance matrix 103 and the behavioral information 104. The annotation information 120 includes specific factors selected based on the importance 132 of the explanatory variables 112, and recommendation guidelines for the prediction target that are assigned as annotations to those specific factors.

[0018] <Example of computer hardware configuration> Figure 2 is a block diagram showing an example of the hardware configuration of a computer. Computer 200 includes a processor 201, a memory device 202, an input device 203, an output device 204, and a communication interface (communication IF) 205. The processor 201, memory device 202, input device 203, output device 204, and communication IF 205 are connected by a bus 206. The processor 201 controls computer 200. The memory device 202 serves as the work area for the processor 201. The memory device 202 is a non-temporary or temporary recording medium that stores various programs and data. Examples of memory devices 202 include ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hard Disk Drive), and flash memory. The input device 203 takes data in. Examples of input devices 203 include a keyboard, mouse, touch panel, numeric keypad, scanner, microphone, and sensor. The output device 204 outputs data. Output devices 204 include, for example, displays, printers, and speakers. The communication IF 205 connects to the network and sends and receives data.

[0019] The XAI 100 and annotation information generation function 110 shown in Figure 1 are implemented on computer 200. The XAI 100 and annotation information generation function 110 may be implemented on the same computer 200 or on different computers 200. Computer 200 on which at least the annotation information generation function 110 is implemented is referred to as a generation device. If the XAI 100 and annotation information generation function 110 are implemented on different computers 200, the generation device receives the importance matrix 103 from other computers on which the XAI 100 is implemented via a network such as the Internet, LAN (Local Area Network), or WAN (Wide Area Network).

[0020] <Machine Learning Dataset 101> Figure 3 is an explanatory diagram showing an example of the machine learning dataset 101 shown in Figure 1. The machine learning dataset 101 has an ID number 300, a target variable 111, and an explanatory variable 112. The ID number 300 is an identification number that uniquely identifies a sample (a patient in this example). The target variable 111 is the ground truth data for the sample identified by ID number 300, and in this example, it is binary data indicating whether the person is independent or requires care.

[0021] Explanatory variable 112 is a set of factors. In this example, factors 311 to 315 constitute the group of factors that make up explanatory variable 112, but the number of factors can be 1 to 4 or 6 or more.

[0022] Factor 1, 311, indicates whether the BMI is < 25. The value of Factor 1, 311, is binary data indicating "applicable" or "not applicable".

[0023] Factor 2, 312, indicates whether the individual meets the criteria for HbA1c ≥ 6.5. The value of Factor 2, 312, is a binary data point indicating either "applicable" or "not applicable."

[0024] The third factor, 313, indicates whether HDL < 35 applies. The value of the third factor, 313, is binary data indicating "applicable" or "not applicable".

[0025] Factor 4, 314, indicates whether the triglyceride level is <30. The value of Factor 4, 314, is binary data indicating "applicable" or "not applicable".

[0026] Factor 5, 315, indicates whether or not the individual has a history of diabetes. The value of Factor 5, 315, is binary data indicating "applicable" or "not applicable".

[0027] <Improvement action information> Figure 4 is an explanatory diagram showing an example of improvement behavior information. Improvement behavior information 400 is information indicating recommended improvement behaviors for factor 411. Improvement behavior information 400 is included in behavior information 104. Improvement behavior information 400 includes first factor improvement behavior information 401 to fifth factor improvement behavior information 405, which indicate recommended improvement behaviors for each factor 411 of the explanatory variable 112. For example, second factor improvement behavior information 402 is improvement behavior information related to second factor 312, and third factor improvement behavior information 403 is improvement behavior information related to third factor 313. First factor improvement behavior information 401 to fifth factor improvement behavior information 405 each include factor 411 and recommended improvement behavior 412. The following explanation will use second factor improvement behavior information 402 and third factor improvement behavior information 403 as examples.

[0028] Factor 411 in Factor 2 Improvement Behavior Information 402 is Factor 2 312. Recommended Improvement Behavior 412 is the improvement behavior recommended when Factor 411 (Factor 2 312) is involved. The numbers in parentheses within Recommended Improvement Behavior 412 indicate priority (ascending order).

[0029] Factor 411 in the third factor improvement behavior information 403 is third factor 313. Recommended improvement behavior 412 is an improvement behavior recommended when factor 411 (third factor 313) is involved. The number in parentheses within recommended improvement behavior 412 indicates an identification number that uniquely identifies the behavior.

[0030] <Risk Reduction Action Information> Figure 5 is an explanatory diagram showing an example of risk reduction behavior information. Risk reduction behavior information 500 is information that indicates recommended risk reduction behaviors for factor 411. Risk reduction behavior information 500 is included in behavior information 104. Risk reduction behavior information 500 includes factor 1 risk reduction behavior information 501 to factor 5 risk reduction behavior information 505, which indicate recommended risk reduction behaviors for each factor 411 of the explanatory variable 112.

[0031] For example, Factor 1 risk reduction behavior information 501 is risk reduction behavior information related to Factor 1 311, and Factor 4 risk reduction behavior information 504 is risk reduction behavior information related to Factor 4 314. Factor 1 risk reduction behavior information 501 to Factor 5 risk reduction behavior information 505 each include Factor 411 and recommended risk reduction behavior 512. The following explanation will use Factor 1 risk reduction behavior information 501 and Factor 4 risk reduction behavior information 504 as examples.

[0032] Factor 411 in the first factor risk reduction behavior information 501 is the first factor 311. Recommended risk reduction behavior 512 is the risk reduction behavior recommended when factor 411 (first factor 311) applies. The numbers in parentheses within Recommended Risk Reduction Behavior 512 indicate priority (ascending order).

[0033] Factor 411 of the risk reduction behavior information for Factor 4 (Factor 4, 314) is Factor 4. Recommended risk reduction behavior (Factor 4, 314) is a risk reduction behavior recommended when Factor 411 is identified. The number in parentheses within Recommended Risk Reduction Behavior (Factor 4, 314) indicates an identification number that uniquely identifies the behavior.

[0034] <Annotation Information Generation Process Procedure> Figure 6 is a flowchart illustrating an example of the annotation information generation process by the generation device. The generation device learns XAI100 by performing machine learning on the machine learning dataset 101 (step S601). Next, the generation device calculates the importance matrix 103 by inputting the prediction target data 102 into the XAI100 learned in step S601 (step S602).

[0035] If XAI100 is not implemented in the generating device, the generating device will receive the importance matrix 103 from another computer that has XAI100 implemented, without executing steps S601 and S602.

[0036] [Importance matrix 103] Figure 7 is an explanatory diagram showing an example of the importance matrix 103. In Figure 7, for the sake of explanation, it is described using a table structure. The importance matrix 103 has an ID number 300, a predicted probability 131 for the dependent variable 111, and an importance 132 for the independent variable 112. The importance 132 for the independent variable 112 has first importance 701 to fifth importance 705. First importance 701 to fifth importance 705 represent the importance of the first factor 311 to fifth factor 315.

[0037] A positive importance value indicates that the factor is a risk amplification factor, while a negative importance value indicates that the factor is a risk reduction factor. An importance value of 0.0 may, depending on the settings, be included in either the risk amplification or risk reduction categories, or not included in either.

[0038] Cells with a shaded background are the risk amplification factors with the highest importance and positive values ​​among the risk amplification factors in that sample, and are referred to as the most important risk amplification factors. Cells with a hatched background are the risk reduction factors with the lowest importance and negative values ​​in that sample, and are referred to as the most important risk reduction factors.

[0039] For example, for patient ID number 300, "BMI < 25," identified as a first-priority factor (701), is the most important risk reduction factor, and "HDL < 35," identified as a third-priority factor (703), is the most important risk amplification factor. When the most important risk amplification factor and the most important risk reduction factor are not distinguished, they are referred to as the most important factors.

[0040] Furthermore, even if a factor has the minimum importance in a given sample, if its value is positive, it will not be considered the most important risk reduction factor. Similarly, even if a factor has the maximum importance in a given sample, if its value is 0.0 or negative, it will not be considered the most important risk amplification factor.

[0041] figure 6 Returning to the previous step, the generating device extracts the most important factors for each sample from the importance matrix 103 (step S603).

[0042] Next, the generation device associates the recommended improvement behavior 412 with the most important factor of each sample as an annotation (step S604). The information obtained by associating the recommended improvement behavior 412 with the most important factor of each sample as an annotation is called combined information.

[0043] [Join information] Figure 8 is an explanatory diagram showing Example 1 of the combined information according to Example 1. The combined information 800 includes an ID number 300, a predicted probability 131 of the target variable 111, a most important factor 801, an importance score 802, and an annotation 803.

[0044] In the combined information 800, the ID number 300 and the predicted probability 131 of the dependent variable 111 are obtained from the importance matrix 103. The most important factor 801 is the most important risk selected for each sample. amplification This is a factor. Importance 802 is the importance of the most important factor 801 in the importance matrix 103.

[0045] Annotation 803 is selected from improvement behavior information 400. Specifically, for example, the generation device selects improvement behavior information 400 corresponding to the most important factor 801 for each sample, and sets the recommended improvement behavior 412 of the selected improvement behavior information 400 in annotation 803.

[0046] For example, the entry for patient ID number 300 is "0010". 8In case 10, the most important risk amplification factor, factor 801, is "HbA1c ≥ 6.5". Therefore, the generator selects second factor improvement behavior information 402 related to second factor 312 from the improvement behavior information 400, and selects one or more recommended improvement behaviors 412. In the example in Figure 8, the highest priority recommended improvement behavior 412, "(1) Resistance exercise 3 or more times per week," is selected. The generator adds the selected recommended improvement behavior 412 as annotation 803 to the entry 810 of the patient with ID number 300 "0010".

[0047] Similarly, in entries 820, 830, and 890 for patients with ID numbers 300 "0020", "0030", and "0090", the most important risk amplifying factor 801 is "HDL < 35". Therefore, the generator selects third factor improvement behavior information 403 related to third factor 313 from the improvement behavior information 400, and selects one or more recommended improvement behaviors 412. In the example in Figure 8, the highest priority recommended improvement behavior 412 is "(3) Walking 8000 steps". / day The following is an example of when "the above" is selected. The generator adds the selected recommended improvement action 412 as annotation 803 to entries 820, 830, and 890 for patients with ID numbers 300 "0020", "0030", and "0090".

[0048] In this way, 412 of the most important recommended improvement behaviors for each individual patient are identified.

[0049] Figure 9 is an explanatory diagram showing example 2 of the combined information according to Example 1. The combined information 900 includes an ID number 300, a predicted probability 131 of the target variable 111, a most important factor 801, an importance score 802, and an annotation 803.

[0050] In the combined information 900, the ID number 300 and the predicted probability 131 of the dependent variable 111 are obtained from the importance matrix 103. In Figure 9, the most important factor 801 is the most important risk reduction factor with the highest importance 802 selected for each sample. The importance 802 is the importance of the most important factor 801 in the importance matrix 103.

[0051] Annotation 803 is selected from improvement behavior information 400. Specifically, for example, the generation device selects risk reduction behavior information 500 corresponding to the most important factor 801 for each sample, and sets the recommended risk reduction behavior 512 of the selected risk reduction behavior information 500 as annotation 803.

[0052] For example, in entry 1010 for patient ID number 300, the most important risk reduction factor 801 is " BMI < 25 Therefore, the generating device selects the fourth factor risk reduction behavior information 504 related to the fourth factor 314 from the risk reduction behavior information 500, and selects one or more recommended risk reduction behaviors 512. In the example in Figure 9, the recommended risk reduction behavior 512 with the highest priority is "(8) More than 8,000 steps per day The following is an example of when " is selected. The generator adds the selected recommended risk reduction behavior 512 as annotation 803 to the patient entry 1010 with ID number 300 "0010".

[0053] Similarly, in patient entries 1020, 1030, and 1090, where ID number 300 is "0020", "0030", and "0090", the most important risk reduction factor 801 is " Neutral fat <30 Therefore, the generating device selects first-factor risk reduction behavior information 501 related to the first factor 311 from the risk reduction behavior information 500, and selects one or more recommended risk reduction behaviors 512.

[0054] In the example in Figure 9, the highest priority recommended risk reduction action 512 is "(3) Walking 8,000 steps." / day The following is an example of when "the above" is selected. The generator adds the selected recommended risk reduction behavior 512 as annotation 803 to entries 1020, 1030, and 1090 for patients with ID numbers 300 "0020", "0030", and "0090".

[0055] In this way, 512 of the most important recommended risk-reducing behaviors for each individual patient are identified.

[0056] Returning to Figure 5, the generation device generates and outputs annotation information based on the combined information 800 and 900 (step S605). The generated annotation information may be displayed on a display device, which is an example of the output device 204 of the generation device, printed out from a printer, which is an example of the output device 204 of the generation device, or transmitted via the communication IF 205 in a format that can be displayed or printed to another computer.

[0057] [Annotation Information] Figure 10 is an explanatory diagram showing example 1 of annotation information. Annotation information 1000, like the combined information 800, has an ID number 300, a predicted probability 131 for the target variable 111, a most important factor 801, an importance score 802, and an annotation 803. Annotation information 1000 is information extracted by sorting the combined information 800 in descending order of the predicted probability 131 for the target variable 111.

[0058] This clarifies the predicted probability 131 (probability of needing long-term care) of the dependent variable 111, i.e., the important factors (most important factors 801) for reducing the risk of needing long-term care, and also clarifies the priority of actions that the group should take. Note that the entries extracted from the combined information 800 may be entries from 1st place up to a predetermined rank, or all entries.

[0059] Figure 11 is an explanatory diagram showing example 2 of annotation information. Annotation information 1100 is combined information. 9 Similar to 00, it has ID number 300, predicted probability 131 of the target variable 111, most important factor 801, importance 802, and annotation 803. Annotation information 1100 is combined information 9 The data for 00 is sorted in descending order of the predicted probability 131 of the dependent variable 111, and the information is extracted for each of the most important factors 801. In Figure 11, the most important factor 801 is " Neutral fat <30 This shows annotation information 1100 in the case of ".

[0060] Therefore, in order to reduce the predicted probability 131 (probability of needing long-term care) of the dependent variable 111, the most important factor 801 is " Neutral fat <30 For those who have not yet achieved this, the practical information of those who have achieved it (ID numbers 300 "0090" and "0020") can be clarified as collective intelligence. In addition, combined information is available for each of the 801 most important factors. 9 The entries extracted from 00 may be entries from 1st place up to a predetermined rank, or they may be all entries corresponding to the most important factor 801.

[0061] Figure 12 is an explanatory diagram showing example 3 of annotation information. Annotation information 1200 is combined information. 9 Similar to 00, it has ID number 300, predicted probability 131 for the target variable 111, most important factor 801, importance 802, and annotation 803. Annotation information 1200 is combined information 9 Risk reduction factors and risk increase factors included in 00 width This is information extracted from factor entries by ID number 300. Figure 12 shows annotation information 1200 when ID number 300 is "0030". Entry 1201 is an entry for the most important factor 801 which is a risk reduction factor, and entry 1202 is an entry for the most important factor 801 which is a risk increase factor width This is the entry point for the factor.

[0062] Alternatively, in step S605, the generating device may generate and output evaluation information 1210, which is a written representation of the annotation information 1200, using a predetermined template.

[0063] Thus, according to Example 1, it becomes possible to select annotations based on important factors extracted from the XAI100 factor group. In Example 1, the most important factor 801 was applied in Figures 8 to 12, but factors 411 with importance 802 from the second most important to a predetermined rank, or factors 411 with importance 802 above a threshold, may be used as important factors and applied in place of or together with the most important factor 801. [Examples]

[0064] Next, we will describe Example 2. In Example 1, we described an example in which improvement behavior information 400 was used as behavior information 104, but in Example 2, we will describe an example in which risk reduction behavior information is used. Note that in Example 2, we will mainly explain the differences from Example 1, so we will omit the explanation of the parts that are common to both Examples 1 and 2.

[0065] <Risk Reduction Action Information> Figure 13 is an explanatory diagram showing an example of risk reduction behavior information. Risk reduction behavior information 1300 is information that indicates recommended risk reduction behaviors for factor 411. Risk reduction behavior information 1300 is included in behavior information 104. For each factor 301 which acts as a factor filter, risk reduction behavior information 1300 includes factor-related risk reduction behavior information that is recommended for factor 411 when it is related to factor 301, and factor-inappropriate risk reduction behavior information that is recommended for factor 411 when it is not related to factor 301.

[0066] Figure 13 shows recommended risk reduction behaviors for Factor 411 (Factor 313: HDL < 35) when Factor 1 is present (Factor 311: BMI < 25), and recommended risk reduction behaviors for Factor 411 (Factor 313: HDL < 35) when Factor 1 is not present (Factor 311: BMI < 25), as shown in Figure 1301.

[0067] In this example, since factor 301 includes factors 311 to 315, 20 sets of risk-reducing behavioral information corresponding to the factors and risk-reducing behavioral information not corresponding to the factors are included in behavioral information 104. The following explanation will use risk-reducing behavioral information 1301 corresponding to factor 1 and risk-reducing behavioral information 1302 not corresponding to factor 1 as examples.

[0068] Factor 411 in the Factor 1 applicable risk reduction behavior information 1301 and the Factor 1 non-applicable risk reduction behavior information 1302 is Factor 313. Rank 1311 indicates the priority of the pre-set recommended risk reduction behavior 1312. In the Factor 1 applicable risk reduction behavior information 1301, the recommended risk reduction behavior 1312 is the risk reduction behavior recommended when the factor filter Factor 1 311 (BMI < 25) is applied. In the Factor 1 non-applicable risk reduction behavior information 1302, the recommended risk reduction behavior 1322 is the risk reduction behavior recommended when the factor filter Factor 1 311 (BMI < 25) is not applied. The number in parentheses within the recommended risk reduction behavior 1312 indicates an identification number that uniquely identifies the behavior.

[0069] Figure 14 is an explanatory diagram showing an example of combined information according to Example 2. The combined information 1400 includes an ID number 300, a predicted probability 131 of the target variable 111, a most important factor 801, an importance score 802, a factor filter 1401, and an annotation 803.

[0070] In the combined information 1400, the factor filter 1401 indicates whether the sample identified by ID number 300 corresponds to a specific factor 411. In Figure 14, the first factor 311 (BMI < 25) is used as the specific factor 411, but the generating device may also generate the combined information 1400 by applying each of the second factors 312 to the fifth factors 315 to the factor filter 1401.

[0071] In the example in Figure 14, for entry 1420, the most important factor 801 is the third factor 313, and factor filter 1401 (first factor 311: BMI < 2 5 Since )) is "applicable", the first-ranked recommended risk reduction behavior 1312, "(3) walking 8,000 steps / day or more", which is the first-ranked risk reduction behavior information 1301 of the first factor applicable risk reduction behavior information 1311, is added as annotation 803 along with the most important factor 801 (third factor 313: HDL < 35).

[0072] Furthermore, for entry 1430, the most important factor 801 is the third factor 313, and factor filter 1401 (first factor 311: BMI < 2 5 Since )) is "not applicable", the first-ranked recommended risk reduction action 1322, "(7) Replace saturated fatty acids with monounsaturated fatty acids", which is the first-ranked risk reduction action 1311 of the first factor not applicable risk reduction action information 1302, is added as annotation 803 along with the most important factor 801 (third factor 313: HDL < 35).

[0073] Furthermore, for entry 1490, the most important factor 801 is the third factor 313, and factor filter 1401 (first factor 311: BMI < 2 5 Since )) is "applicable", the first-ranked recommended risk reduction behavior 1312, "(3) walking 8,000 steps / day or more", which is the first-ranked risk reduction behavior information 1301 of the first factor applicable risk reduction behavior information 1311, is added as annotation 803 along with the most important factor 801 (third factor 313: HDL < 35).

[0074] The generating device may apply each of the factors 301 to the factor filter 1401 and generate binding information 1400 for each factor filter 1401, or it may apply the factors 301 selected by the user of the generating device to the factor filter 1401 and generate binding information 1400 for each factor filter 1401.

[0075] In this way, the generation device generates the coupling information 1400 in step S604. Then, the generation device uses the coupling information 1400 to generate Figure 1 0 Annotation information 1100, 1000, and 1200, as shown in Figure 12, can be generated and output. [Examples]

[0076] Next, we will describe Example 3. Example 3 describes an example of modifying annotation 803 of the joint information. Note that in Example 3, we will mainly explain the differences from Examples 1 and 2, and will omit explanations of the parts that are common to Examples 1 and 2.

[0077] Figure 15 is an explanatory diagram showing example 1 of the modification of annotation 803 of the join information 800. Compared with Figure 8, the entry 8 In annotation 803 of 10, the recommended improvement behavior 412 in the second factor improvement behavior information 402, "(1) resistance exercise at least 3 times a week," has been changed to "(2) aerobic exercise at least 3 times a week."

[0078] Similarly, entry 8 In annotation 803 of 20, the recommended improvement behavior 412 of the third factor improvement behavior information 403 is "(3) Walking 8000 steps." / day The phrase "(6) No Smoking" has been changed to "(6) No Smoking".

[0079] Such changes are implemented when the user of the generating device selects the target recommended improvement action from the second factor improvement action information 402 and third factor improvement action information 403 displayed on the display screen using the input device 203. Alternatively, the generating device can implement the change when it receives a signal from a user of a terminal that can communicate with the generating device, indicating that the user has selected the target recommended improvement action from the second factor improvement action information 402 and third factor improvement action information 403 displayed on the terminal's display screen using the terminal's input device 203.

[0080] Figure 16 is an explanatory diagram showing an example of change 2 in annotation 803 of linked information 1400. Compared with Figure 14, in annotation 803 of entry 1430, the recommended risk reduction action 1322 of the risk reduction action information 1302 for the first factor not applicable, "(7) Replace saturated fatty acids with monounsaturated fatty acids," has been changed to "(6) Quit smoking."

[0081] Similarly, in annotation 803 of entry 1490, the recommended risk reduction behavior 1312 for the first factor corresponding risk reduction behavior information 1301 is "(3) Walking 8000 steps." / day The phrase "(5) Dietary fiber-rich meals" has been changed to "(5) Dietary fiber-rich meals."

[0082] Such changes are performed when the user of the generating device selects the target risk reduction action from the first factor applicable risk reduction action information 1301 and the first factor non-applicable risk reduction action information 1302 displayed on the display screen using the input device 203. Alternatively, the generating device can perform the change when it receives a signal from a user of a terminal that can communicate with the generating device, indicating that the user has selected the target risk reduction action from the first factor applicable risk reduction action information 1301 and the first factor non-applicable risk reduction action information 1302 displayed on the display screen of that terminal using the input device 203 of that terminal.

[0083] Thus, the generating device can change the annotation 803 of the combined information 800,900 via external operation. Therefore, the annotation suitable for the sample identified by ID number 300 803 We can provide this. [Examples]

[0084] Next, we will describe Example 4. Example 4 describes an example in which combined behavioral information is used as behavioral information. Note that in Example 4, we will mainly explain the differences from Examples 1 to 3, and will omit explanations of the common parts with Examples 1 to 3.

[0085] Figure 17 is an explanatory diagram showing an example of combined behavioral information. Combined behavioral information 1700 is included in behavioral information 104. Combined behavioral information 1700 has a risk reduction factor 1701, a risk amplification factor 1702, and a combined annotation 1703. The combined annotation 1703 is an annotation that combines the risk reduction factor 1701 and the risk amplification factor 1702.

[0086] For example, in the importance matrix 103, in the sample where ID number 300 is "0030", the risk reduction factor 1701 is "BMI < 25" and the risk amplification factor 1702 is "HDL < 35". Therefore, in step S604, the generator selects an annotation 1703 of entries where the combination of risk reduction factor 1701 and risk amplification factor 1702 is "BMI < 25" and "HDL < 35", and generates annotation information 1710 (step S605).

[0087] The annotation information 1710 includes an ID number 300, a predicted probability 131 for the target variable 111, a risk reduction factor 1701, a risk amplification factor 1702, and a combined annotation 1703. In the case of Example 4, it is not necessary to perform step S604.

[0088] Thus, 4 According to this, it is possible to provide combination annotation 1703 according to the sample.

[0089] As explained above, Examples 1 to 4 According to this, it is possible to optimize the selection of annotations.

[0090] It should be noted that the present invention is not limited to the embodiments described above, but includes various modifications and equivalent configurations within the spirit of the attached claims. For example, the embodiments described above are described in detail to make the present invention easier to understand, and the present invention is not necessarily limited to having all of the described configurations. Furthermore, some of the configurations of one embodiment may be replaced with those of another embodiment. Furthermore, some of the configurations of one embodiment may be added to those of another embodiment. Furthermore, some of the configurations of each embodiment may be added, deleted, or replaced with other configurations.

[0091] Furthermore, each of the aforementioned configurations, functions, processing units, and processing means may be implemented in hardware, for example, by designing them as integrated circuits, or they may be implemented in software by having a processor interpret and execute programs that realize each function.

[0092] Information such as programs, tables, and files that implement each function can be stored in memory, hard disks, SSDs (Solid State Drives), or on recording media such as IC (Integrated Circuit) cards, SD cards, and DVDs (Digital Versatile Discs).

[0093] Furthermore, the control lines and information lines shown are those deemed necessary for explanation purposes and do not necessarily represent all control lines and information lines required for implementation. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0094] 100 XAI 101 Machine Learning Datasets 102 Data to be predicted 103 Importance matrix 104 Activity Information 110 Annotation Information Generation Function 111 Dependent variable 120 Annotation Information 112 explanatory variables 131 Prediction Probability 132 Importance 200 calculator 201 Processor 202 Storage Devices 400 Improvement action information 411 factor 412 Recommended Improvement Actions 500 Risk Reduction Action Information 512 Recommended Risk Reduction Actions 700 annotations 800,900,1400 Join information 801 Most important factor 802 Importance 803 Annotation 1000,1100,1200,1710 Annotation information 1300 Risk Reduction Action Information 1401-factor filter 1700 Combination behavior information 1701 Risk Reduction Factors 1702 Risk Amplifying Factors 1703 Combination Annotation

Claims

1. A generating apparatus having a processor for executing a program and a storage device for storing the program, For each factor in the group of factors, risk reduction behavior information is stored that associates the factor with the risk reduction behavior recommended for the sample when the factor applies. The aforementioned processor, For the aforementioned sample, an acquisition process is performed to obtain the importance of each factor in the factor group, which serves as the basis for the predicted probability based on whether each factor in the factor group is applicable or not. An extraction process for extracting specific factors from the group of factors whose importance is less than a threshold, obtained through the acquisition process, A generation process that obtains specific risk reduction behaviors corresponding to specific factors extracted by the extraction process from the aforementioned risk reduction behavior information, and generates annotation information that presents the specific risk reduction behaviors to the samples corresponding to the specific factors, A generating device characterized by performing the following actions.

2. The generating apparatus according to Claim 1, In the extraction process, the processor extracts the factors whose importance is less than a threshold and is the minimum as the specific factors. A generating apparatus characterized by the following features.

3. The generating apparatus according to Claim 1, In the acquisition process, the processor acquires the predicted probability for the sample, In the generation process, the processor generates annotation information for samples where the predicted probability is greater than or equal to a predetermined probability, or up to a predetermined rank. A generating apparatus characterized by the following features.

4. A generating apparatus having a processor for executing a program and a storage device for storing the program, For each of the second factors other than the first factor in the group of factors, if the sample corresponds to the first factor and the second factor, the system stores information on the risk reduction behavior corresponding to the first factor, which associates the second factor with the risk reduction behavior recommended for the sample. The aforementioned processor, For the aforementioned sample, an acquisition process is performed to obtain the importance of each factor in the factor group, which serves as the basis for the predicted probability based on whether each factor in the factor group is applicable or not. An extraction process is performed to extract from the group of factors a second factor whose importance obtained by the acquisition process is less than a threshold, as a specific second factor. For a specific sample corresponding to the first factor, a generation process is performed to obtain a specific risk reduction behavior corresponding to the specific second factor extracted by the extraction process from the risk reduction behavior information corresponding to the first factor, and to generate annotation information that presents the specific risk reduction behavior to the specific sample corresponding to the specific second factor. A generating device characterized by performing the following actions.

5. A generating apparatus comprising a processor for executing a program and a storage device for storing the program, For each of the second factors other than the first factor in the group of factors, if the sample does not correspond to the first factor but does correspond to the second factor, then information on risk reduction behaviors that are recommended for the sample that does not correspond to the first factor is stored. The aforementioned processor, For the aforementioned sample, an acquisition process is performed to obtain the importance of each factor in the factor group, which serves as the basis for the predicted probability based on whether each factor in the factor group is applicable or not. An extraction process is performed to extract from the group of factors a second factor whose importance obtained by the acquisition process is less than a threshold, as a specific second factor. For specific samples that do not fall under the first factor, a generation process is performed to obtain specific risk reduction behaviors corresponding to the specific second factor from the risk reduction behavior information for which the first factor is not applicable, and to generate annotation information that presents the specific risk reduction behaviors to the specific samples that do fall under the specific second factor. A generating device characterized by performing the following actions.

6. A generating apparatus comprising a processor for executing a program and a storage device for storing the program, The system stores combined behavioral information that associates combinations of risk amplification factors and risk reduction factors within a group of factors with the corresponding actions when those combinations occur. The aforementioned processor, For a sample, an acquisition process is performed to obtain the importance of each factor in the factor group, which serves as the basis for the predicted probability based on whether each factor in the factor group is applicable or not. Based on the importance obtained by the acquisition process, an extraction process is performed to extract a specific combination of risk amplification factors and a specific risk reduction factor from the group of factors for each sample. A generation process that obtains specific behaviors corresponding to specific risk amplification factors and specific risk reduction factors extracted by the extraction process from the aforementioned combined behavior information, and generates annotation information that presents the specific behaviors to the samples corresponding to the specific risk amplification factors and specific risk reduction factors. A generating device characterized by performing the following actions.

7. A generation method performed by a generation device having a processor for executing a program and a storage device for storing the program, The generating device stores risk reduction behavior information for each factor in the factor group, which associates the factor with the risk reduction behavior recommended for the sample when the factor applies. The aforementioned processor, For the aforementioned sample, an acquisition process is performed to obtain the importance of each factor in the factor group, which serves as the basis for the predicted probability based on whether each factor in the factor group is applicable or not. An extraction process for extracting specific factors from the group of factors whose importance is less than a threshold, obtained through the acquisition process, A generation process that obtains specific risk reduction behaviors corresponding to specific factors extracted by the extraction process from the aforementioned risk reduction behavior information, and generates annotation information that presents the specific risk reduction behaviors to the samples corresponding to the specific factors, A generation method characterized by performing the following.

8. A generation method performed by a generation device having a processor for executing a program and a storage device for storing the program, The generating device stores, for each second factor other than the first factor in the factor group, first factor corresponding risk reduction behavior information, which associates the second factor with the risk reduction behavior recommended for the sample when the sample corresponds to the first factor and the second factor. The aforementioned processor, For the aforementioned sample, an acquisition process is performed to obtain the importance of each factor in the factor group, which serves as the basis for the predicted probability based on whether each factor in the factor group is applicable or not. An extraction process is performed to extract from the group of factors a second factor whose importance obtained by the acquisition process is less than a threshold, as a specific second factor. For a specific sample corresponding to the first factor, a generation process is performed to obtain a specific risk reduction behavior corresponding to the specific second factor extracted by the extraction process from the risk reduction behavior information corresponding to the first factor, and to generate annotation information that presents the specific risk reduction behavior to the specific sample corresponding to the specific second factor. A generation method characterized by performing the following.

9. A generation method performed by a generation device having a processor for executing a program and a storage device for storing the program, The generating device stores, for each second factor other than the first factor in the group of factors, information on risk reduction behaviors that do not apply to the first factor, where the second factor and the recommended risk reduction behavior for the sample are associated when the sample does not apply to the first factor but does apply to the second factor. The aforementioned processor, For the aforementioned sample, an acquisition process is performed to obtain the importance of each factor in the factor group, which serves as the basis for the predicted probability based on whether each factor in the factor group is applicable or not. An extraction process is performed to extract from the group of factors a second factor whose importance obtained by the acquisition process is less than a threshold, as a specific second factor. For specific samples that do not fall under the first factor, a generation process is performed to obtain specific risk reduction behaviors corresponding to the specific second factor from the risk reduction behavior information for which the first factor is not applicable, and to generate annotation information that presents the specific risk reduction behaviors to the specific samples that do fall under the specific second factor. A generation method characterized by performing the following.

10. A generation method performed by a generation apparatus having a processor for executing a program and a storage device for storing the program, The generating device stores combined behavior information that associates combinations of risk amplification factors and risk reduction factors within a group of factors with the corresponding actions when those combinations occur. The aforementioned processor, For a sample, an acquisition process is performed to obtain the importance of each factor in the factor group, which serves as the basis for the predicted probability based on whether each factor in the factor group is applicable or not. Based on the importance obtained by the acquisition process, an extraction process is performed to extract a specific combination of risk amplification factors and a specific risk reduction factor from the group of factors for each sample. A generation process that obtains specific behaviors corresponding to specific risk amplification factors and specific risk reduction factors extracted by the extraction process from the aforementioned combined behavior information, and generates annotation information that presents the specific behaviors to the samples corresponding to the specific risk amplification factors and specific risk reduction factors. A generation method characterized by performing the following.

11. A generation program characterized by causing the processor to execute the generation method described in any one of claims 7 to 10.

Images