Generation device, generation method, and generation program

Abstract

To optimize annotation selection.SOLUTION: A generation device comprises a processor that executes a program and a storage device that stores the program. The generation device stores behavior information in which, for each of factors in a factor group, the factor is associated with a behavior when the factor is applicable. The processor executes: acquisition processing of acquiring, for each of samples, a predicted probability based on whether or not each factor in the factor group is applicable and an importance level of each factor in the factor group which is a basis for the predicted probability; extraction processing of extracting a specific factor from the factor group on the basis of the importance level acquired by the acquisition processing; and generation processing of acquiring, from the behavior information, a specific behavior corresponding to the specific factor extracted by the extraction processing and generating annotation information that presents the specific behavior to the sample to which the specific factor is applicable.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a generating device, a generating method, and a generating program for generating information. [Background technology]

[0002] The following patent document 1 discloses a model-aided annotation system. The model-aided annotation system is operable to receive a first set of annotation data for a first set of medical scans from a set of client devices. A computer vision model is trained utilizing the first set of medical scans and the first set of annotation data. A second set of annotation data for a second set of medical scans is generated by utilizing the computer vision model. The second set of medical scans and the second set of annotation data are transmitted to the set of client devices, which in response receive an additional set of annotation data. An updated computer vision model is generated by utilizing the additional set of annotation data. A third set of annotation data is generated for the third set of medical scans by utilizing the updated computer vision model for transmission to the set of client devices for display.

[0003] The following Patent Document 2 discloses a counterfeit detection system, which includes one or more data stores for storing images of genuine items, and a processor, which executes instructions to obtain an input image, determine a difference between a region of the image of the genuine item and a corresponding region of the input image by a generative adversarial network (GAN) discriminator, generate a classification of whether the input image is an image of a genuine item or not based on the determined difference by the discriminator, identify the corresponding region of the input image that contributed to the classification by a class activation module (CAM), obtain an annotation by the CAM indicating whether the corresponding region indicates whether the input image is an image of a genuine item or not, and retrain the discriminator and / or the CAM based on the annotation.

[0004] The following patent document 3 discloses a system for reviewing medical images. The system includes an electronic processor configured to display electronic medical images, collect (compile) clinical information associated with the electronic medical images, determine a probability of a disease associated with a patient associated with the electronic medical images based on the collected clinical information, and display the probability of the disease together with the medical images. The electronic processor is also configured to receive annotations on the electronic medical images, determine an updated probability of the disease based on the clinical information and the annotations, and display the updated probability of the disease.

[0005] The following Patent Document 4 discloses a healthcare risk estimation system. This healthcare risk estimation system includes a risk-related term collection unit including terms related to risk in the form of potential disorders, terms related to risk factors that increase the likelihood of disorders, and terms related to treatments for medical conditions, a medical entity reconciliation unit that uses a standard vocabulary of terms to standardize and expand terms by doctors to include synonyms and homonyms, a topic detection and tagging unit that retrieves a set of documents linked to the extended terms from a medical document database, a NERD module that extracts entities from the set of documents and adjusts each document to the standardized vocabulary, and a relationship extraction unit that scores relationships between entities based on the co-occurrence of two entities in the retrieved set of documents, and generates a risk knowledge graph that stores those relationships. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] US Patent Application Publication No. 2020 / 0160979 [Patent Document 2] US Patent Application Publication No. 2019 / 0236614 [Patent Document 3] International Publication No. 2018 / 042272 [Patent Document 4] JP 2017-174406 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, Patent Documents 1 to 4 do not take into consideration annotation selection based on important factors extracted from a group of factors of eXplainable Artificial Intelligence (hereinafter, XAI).

[0008] An object of the present invention is to optimize annotation selection. [Means for solving the problem]

[0009] A generation device which is one aspect of the invention disclosed in the present application is a generation device having a processor which executes a program and a storage device which stores the program, and stores behavioral information which associates each factor of a group of factors with a behavior when the factor corresponds to the factor, and the processor executes an acquisition process which acquires, for a sample, a predicted probability based on whether each factor of the group of factors corresponds to the factor and the importance of each factor of the group of factors which is the basis of the predicted probability, an extraction process which extracts a specific factor from the group of factors based on the importance acquired by the acquisition process, and a generation process which acquires from the behavioral information a specific behavior corresponding to the specific factor extracted by the extraction process, and generates annotation information which presents the specific behavior for the sample that corresponds to the specific factor. Effect of the Invention

[0010] According to the representative embodiment of the present invention, it is possible to optimize annotation selection. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is an explanatory diagram showing an example of generating annotation information. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of a computer. [Diagram 3] FIG. 3 is a diagram illustrating an example of the machine learning dataset illustrated in FIG. [Figure 4] FIG. 4 is an explanatory diagram illustrating an example of improvement action information. [Diagram 5] FIG. 5 is an explanatory diagram illustrating an example of risk reduction behavior information. [Figure 6] FIG. 6 is a flowchart illustrating an example of a procedure of an annotation information generating process performed by the generating device. [Figure 7] FIG. 7 is an explanatory diagram illustrating an example of the importance matrix. [Figure 8] FIG. 8 is a diagram of Example 1 of the combined information according to the first embodiment. [Figure 9] FIG. 9 is a diagram of Example 2 of the combined information according to the first embodiment. [Figure 10] FIG. 10 is an explanatory diagram illustrating an example 1 of annotation information. [Figure 11] FIG. 11 is an explanatory diagram showing a second example of annotation information. [Figure 12] FIG. 12 is an explanatory diagram illustrating an example 3 of annotation information. [Figure 13] FIG. 13 is an explanatory diagram illustrating an example of risk reduction behavior information. [Figure 14] FIG. 14 is a diagram of an example of the combined information according to the second embodiment. [Figure 15] FIG. 15 is an explanatory diagram showing a first modification of the annotation of the combination information. [Figure 16] FIG. 16 is an explanatory diagram showing a second modification of the annotation of the combination information. [Figure 17] FIG. 17 is an explanatory diagram illustrating an example of combined behavior information. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS EXAMPLES

[0012] <Example of annotation information generation> FIG. 1 is an explanatory diagram showing an example of generating annotation information. During learning, the XAI 100 learns using a machine learning dataset 101. The machine learning dataset 101 is a combination of a target variable 111 called ground truth data and an explanatory variable 112 called learning data. The explanatory variable 112 is data having values ​​of each factor of a factor group for each sample (patient in this example). The factors are various test values ​​for the patient, such as BMI (Body Mass Index) and HDL (High Density Lipoprotein).

[0013] The objective variable 111 is a result corresponding to the value of the factor of each sample. In this example, it indicates independence or need for care. The XAI 100 is trained so that the value of the loss function becomes smaller based on the difference between the output result obtained as a result of inputting the explanatory variable 112 to the XAI 100 and the objective variable 111.

[0014] At the time of prediction, the XAI 100 inputs the data 102 to be predicted. The data 102 to be predicted has the same factor group values ​​as the explanatory variables 112. The data 102 to be predicted may be at least a part of the explanatory variables 112. In this example, for convenience of explanation, the explanatory variables 112 are used as the data 102 to be predicted as they are.

[0015] When the prediction target data 102 is input, the XAI 100 outputs an importance matrix 103. The importance matrix 103 includes a prediction probability 131 of the dependent variable 111 and an importance 132 of the explanatory variable 112. The importance 132 of the explanatory variable 112 is information indicating the basis on which the XAI 100 calculates the prediction probability 131 of the dependent variable 111.

[0016] The behavior information 104 is information that associates the range of values ​​that each factor constituting the explanatory variables 112 can take with a recommended guideline for a prediction target having prediction target data 102. In this example, the prediction target is a patient, and the recommended guideline is a behavior (such as quitting smoking) that is recommended for the patient.

[0017] The annotation information generation function 110 generates annotation information 120 for the prediction target by using the importance matrix 103 and the behavioral information 104. The annotation information 120 is information including a specific factor selected based on the importance 132 of the explanatory variables 112 and a recommendation guideline for the prediction target that is assigned as an annotation to the specific factor.

[0018] <Example of computer hardware configuration> FIG. 2 is a block diagram showing an example of a hardware configuration of a computer. The computer 200 includes a processor 201, a storage device 202, an input device 203, an output device 204, and a communication interface (communication IF) 205. The processor 201, the storage device 202, the input device 203, the output device 204, and the communication IF 205 are connected by a bus 206. The processor 201 controls the computer 200. The storage device 202 serves as a working area for the processor 201. The storage device 202 is a non-transient or temporary recording medium that stores various programs and data. Examples of the storage device 202 include a read only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), and a flash memory. The input device 203 inputs data. Examples of the input device 203 include a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, a microphone, and a sensor. The output device 204 outputs data. The output device 204 may be, for example, a display, a printer, or a speaker. The communication IF 205 connects to a network and transmits and receives data.

[0019] 1 are implemented in a computer 200. The XAI 100 and the annotation information generating function 110 may be implemented in the same computer 200, or may be implemented in different computers 200. The computer 200 in which at least the annotation information generating function 110 is implemented is referred to as a generating device. When the XAI 100 and the annotation information generating function 110 are implemented in different computers 200, the generating device receives the importance matrix 103 from another computer in which the XAI 100 is implemented, via a network such as the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network).

[0020] <Machine Learning Dataset 101> Fig. 3 is an explanatory diagram showing an example of the machine learning dataset 101 shown in Fig. 1. The machine learning dataset 101 has an ID number 300, an objective variable 111, and an explanatory variable 112. The ID number 300 is an identification number that uniquely identifies a sample (a patient in this example). The objective variable 111 is correct answer data for the sample identified by the ID number 300, and in this example, is binary data indicating independence or need for care.

[0021] The explanatory variables 112 are a set of factors. In this example, the first factor 311 to the fifth factor 315 are the group of factors that make up the explanatory variables 112, but the number of factors may be 1 to 4 or 6 or more.

[0022] The first factor 311 indicates the relevance of BMI<25. The value of the first factor 311 is binary data indicating "relevant" or "not applicable."

[0023] The second factor 312 indicates whether or not HbA1c is 6.5 or higher. The value of the second factor 312 is binary data indicating "applicable" or "not applicable."

[0024] The third factor 313 indicates whether or not HDL is less than 35. The value of the third factor 313 is binary data indicating "applicable" or "not applicable."

[0025] The fourth factor 314 indicates whether or not the patient has triglyceride levels < 30. The value of the fourth factor 314 is binary data indicating "applicable" or "not applicable."

[0026] The fifth factor 315 indicates whether or not the patient has a history of diabetes. The value of the fifth factor 315 is binary data indicating "yes" or "no".

[0027] <Improvement action information> FIG. 4 is an explanatory diagram showing an example of improvement action information. The improvement action information 400 is information indicating improvement actions recommended for a factor 411. The improvement action information 400 is included in the action information 104. The improvement action information 400 includes first factor improvement action information 401 to fifth factor improvement action information 405 indicating improvement actions recommended for each factor 411 of the explanatory variable 112. For example, the second factor improvement action information 402 is improvement action information related to the second factor 312, and the third factor improvement action information 403 is improvement action information related to the third factor 313. The first factor improvement action information 401 to the fifth factor improvement action information 405 each include a factor 411 and a recommended improvement action 412. Hereinafter, the second factor improvement action information 402 and the third factor improvement action information 403 will be described as examples.

[0028] The factor 411 of the second factor improvement action information 402 is the second factor 312. The recommended improvement action 412 is an improvement action recommended when the factor 411 (second factor 312) applies. The numbers in parentheses within the recommended improvement action 412 indicate the priority (ascending order).

[0029] The factor 411 of the third factor improvement action information 403 is the third factor 313. The recommended improvement action 412 is an improvement action recommended when the factor 411 (third factor 313) applies. The number in parentheses in the recommended improvement action 412 indicates an identification number that uniquely identifies the action.

[0030] <Risk reduction behavior information> 5 is an explanatory diagram showing an example of risk reduction behavior information. Risk reduction behavior information 500 is information indicating risk reduction behaviors recommended for factors 411. The risk reduction behavior information 500 is included in the behavior information 104. The risk reduction behavior information 500 includes first factor risk reduction behavior information 501 to fifth factor risk reduction behavior information 505 indicating risk reduction behaviors recommended for each factor 411 of the explanatory variables 112.

[0031] For example, the first factor risk reduction behavior information 501 is risk reduction behavior information related to the first factor 311, and the fourth factor risk reduction behavior information 504 is risk reduction behavior information related to the fourth factor 314. The first factor risk reduction behavior information 501 to the fifth factor risk reduction behavior information 505 each include the factor 411 and a recommended risk reduction behavior 512. The first factor risk reduction behavior information 501 and the fourth factor risk reduction behavior information 504 will be described below as examples.

[0032] The factor 411 of the first factor risk reduction behavior information 501 is the first factor 311. The recommended risk reduction behavior 512 is a risk reduction behavior recommended when the factor 411 (first factor 311) applies. The numbers in parentheses within the recommended risk reduction behavior 512 indicate the priority (ascending order).

[0033] The factor 411 of the fourth factor risk reduction behavior information 504 is the fourth factor 314. The recommended risk reduction behavior 512 is a risk reduction behavior recommended when the factor 411 (fourth factor 314) applies. The number in parentheses in the recommended risk reduction behavior 512 indicates an identification number that uniquely identifies the behavior.

[0034] <Annotation information generation process> 6 is a flowchart showing an example of an annotation information generation process performed by the generating device. The generating device learns the XAI 100 by machine learning using the machine learning dataset 101 (step S601). Next, the generating device inputs the prediction target data 102 to the XAI 100 learned in step S601 to calculate the importance matrix 103 (step S602).

[0035] If the generation device does not have the XAI 100 implemented, the generation device will receive the importance matrix 103 from another computer in which the XAI 100 is implemented, without executing steps S601 and S602.

[0036] [Importance matrix 103] FIG. 7 is an explanatory diagram showing an example of the importance matrix 103. For convenience of explanation, FIG. 7 will be explained using a table structure. The importance matrix 103 has an ID number 300, a predicted probability 131 of the objective variable 111, and an importance 132 of the explanatory variable 112. The importance 132 of the explanatory variable 112 has a first importance 701 to a fifth importance 705. The first importance 701 to the fifth importance 705 indicate the importance of the first factor 311 to the fifth factor 315.

[0037] A positive importance value indicates that the factor is a risk amplification factor, and a negative importance value indicates that the factor is a risk reduction factor. Note that an importance value of 0.0 means that the factor is included in either a risk amplification factor or a risk reduction factor, or is included in neither, depending on the settings.

[0038] The shaded cells are the risk amplification factors that have the highest importance among the risk amplification factors in the sample and are positive values, and are called the most important risk amplification factors. The hatched cells are the risk reduction factors that have the lowest importance among the risk amplification factors in the sample and are negative values, and are called the most important risk reduction factors.

[0039] For example, for a patient with ID number 300 of "0020", "BMI<25" specified by the first importance 701 is the most important risk reducing factor, and "HDL<35" specified by the third importance 703 is the most important risk amplifying factor. When there is no distinction between the most important risk amplifying factor and the most important risk reducing factor, they are referred to as the most important factor.

[0040] Note that even if the importance of a sample is the minimum, if it is a positive value, it will not be the most important risk reduction factor.Similarly, even if the importance of a sample is the maximum, if it is 0.0 or a negative value, it will not be the most important risk amplification factor.

[0041] figure 6 Returning to step S603, the generating device extracts the most important factor for each sample from the importance matrix 103.

[0042] Next, the generating device associates the most important factor of each sample with the recommended improvement action 412 as an annotation (step S604). Information in which the most important factor of each sample is associated with the recommended improvement action 412 as an annotation is referred to as combined information.

[0043] [Join information] 8 is an explanatory diagram illustrating Example 1 of the combined information according to Example 1. The combined information 800 has an ID number 300, a predicted probability 131 of the objective variable 111, a most important factor 801, an importance level 802, and an annotation 803.

[0044] In the combined information 800, the ID number 300 and the predicted probability 131 of the dependent variable 111 are obtained from the importance matrix 103. The most important factor 801 is the most important risk factor selected for each sample. amplification The importance 802 is the importance of the most important factor 801 in the importance matrix 103.

[0045] The annotation 803 is selected from the improvement action information 400. Specifically, for example, the generating device selects the improvement action information 400 corresponding to the most important factor 801 for each sample, and sets the recommended improvement action 412 of the selected improvement action information 400 to the annotation 803.

[0046] For example, the entry for patient 300 with ID number "0010" 8In FIG. 10, the most important factor 801, which is the most important risk amplification factor, is "HbA1c≧6.5". Therefore, the generation device selects second factor improvement action information 402 related to the second factor 312 from the improvement action information 400, and selects one or more recommended improvement actions 412. The example in FIG. 8 shows an example in which "(1) Resistance exercise 3 times or more per week", which is the recommended improvement action 412 with the highest priority, is selected. The generation device adds the selected recommended improvement action 412 as an annotation 803 to the entry 810 of the patient whose ID number 300 is "0010".

[0047] Similarly, in the entries 820, 830, 890 of patients with ID numbers 300 of "0020," "0030," and "0090," respectively, the most important factor 801, which is the most important risk amplification factor, is "HDL<35." Therefore, the generating device selects third factor improvement action information 403 related to the third factor 313 from the improvement action information 400, and selects one or more of its recommended improvement actions 412. In the example of FIG. 8, the recommended improvement action 412 with the highest priority, "(3) Walking 8,000 steps," is selected. / day 8 shows an example in which "the above" has been selected. The generation device adds the selected recommended improvement action 412 as an annotation 803 to the entries 820, 830, and 890 of the patients whose ID numbers 300 are "0020," "0030," and "0090," respectively.

[0048] In this manner, the most important recommended remedial actions 412 are identified for the individual patient.

[0049] 9 is an explanatory diagram showing Example 2 of the combined information according to Example 1. The combined information 900 has the ID number 300, the predicted probability 131 of the objective variable 111, the most important factor 801, the importance 802, and an annotation 803.

[0050] In the combined information 900, the ID number 300 and the predicted probability 131 of the objective variable 111 are obtained from the importance matrix 103. In Fig. 9, the most important factor 801 is the most important risk reduction factor with the largest importance 802 selected for each sample. The importance 802 is the importance of the most important factor 801 in the importance matrix 103.

[0051] The annotation 803 is selected from the improvement action information 400. Specifically, for example, the generating device selects the risk reduction action information 500 corresponding to the most important factor 801 for each sample, and sets the recommended risk reduction action 512 of the selected risk reduction action information 500 to the annotation 803.

[0052] For example, in entry 1010 of patient ID number 300, "0010", the most important risk reduction factor, the most important factor 801, is " BMI<25 Therefore, the generating device selects the fourth factor risk reduction action information 504 related to the fourth factor 314 from the risk reduction action information 500, and selects one or more of its recommended risk reduction actions 512. In the example of FIG. 9, the recommended risk reduction action 512 with the highest priority, "(8) Walking 8,000 steps or more per day 8 shows an example in which "Recommended risk reduction behavior 512" has been selected. The generation device adds the selected recommended risk reduction behavior 512 as an annotation 803 to the entry 1010 for the patient whose ID number 300 is "0010".

[0053] Similarly, in the entries 1020, 1030, and 1090 of the patients whose ID numbers 300 are “0020,” “0030,” and “0090,” respectively, the most important factor 801, which is the most important risk reduction factor, is “ Neutral fat <30 Therefore, the generating device selects the first factor risk reduction action information 501 related to the first factor 311 from the risk reduction action information 500, and selects one or more recommended risk reduction actions 512 thereof.

[0054] In the example of FIG. 9, the recommended risk reduction action 512 with the highest priority, “(3) Walk 8,000 steps” / day 8 shows an example in which "the above" has been selected. The generation device adds the selected recommended risk reduction behavior 512 as an annotation 803 to the entries 1020, 1030, and 1090 of the patients whose ID numbers 300 are "0020," "0030," and "0090," respectively.

[0055] In this manner, the most important recommended risk reduction actions 512 are identified for the individual patient.

[0056] 5, the generating device generates and outputs annotation information based on the combined information 800, 900 (step S605). The generated annotation information may be displayed on a display device, which is an example of the output device 204 of the generating device, or may be printed out from a printer, which is an example of the output device 204 of the generating device, or may be transmitted via the communication IF 205 to another computer so that it can be displayed or printed.

[0057] [Annotation information] 10 is an explanatory diagram showing Example 1 of annotation information. Like the combination information 800, the annotation information 1000 has an ID number 300, a prediction probability 131 of the objective variable 111, a most important factor 801, an importance 802, and an annotation 803. The annotation information 1000 is information extracted by sorting the combination information 800 in descending order of the prediction probability 131 of the objective variable 111.

[0058] This makes clear the predicted probability 131 (probability of needing care) of the objective variable 111, i.e., the important factor (most important factor 801) for reducing the risk of needing care, and also makes clear the priority order of actions to be taken as a group. Note that the entries extracted from the combined information 800 may be the entries ranked from 1st to a predetermined rank, or all entries.

[0059] FIG. 11 is an explanatory diagram showing an example 2 of annotation information. The annotation information 1100 is combination information 9 00, the annotation information 1100 includes an ID number 300, a predicted probability 131 of the objective variable 111, a most important factor 801, an importance level 802, and an annotation 803. 9 11, the most important factor 801 is " Neutral fat <30 11 shows annotation information 1100 in the case where ".

[0060] As a result, in order to reduce the predicted probability 131 (probability of needing nursing care) of the objective variable 111, the most important factor 801, Neutral fat <30 For those who have not achieved the above, the practical information of those who have achieved the above (ID numbers 300 are "0090" and "0020") can be clarified as collective knowledge. 9 The entries extracted from 00 may be the entries from 1st place to a predetermined rank, or may be all the entries that correspond to the most important factor 801 .

[0061] FIG. 12 is an explanatory diagram showing an example 3 of annotation information. The annotation information 1200 is combination information 9 00, the annotation information 1200 includes an ID number 300, a predicted probability 131 of the objective variable 111, a most important factor 801, an importance level 802, and an annotation 803. 9 Risk reduction and increased risk factors included in 00 width This is information extracted from the factor entries by ID number 300. FIG. 12 shows annotation information 1200 when the ID number 300 is "0030". Entry 1201 is an entry in which the most important factor 801 is a risk reduction factor, and entry 1202 is an entry in which the most important factor 801 is a risk increase factor. width This is the factor entry.

[0062] In addition, in step S605, the generating device may generate and output evaluation information 1210 by converting the annotation information 1200 into text using a predetermined template.

[0063] In this way, according to the first embodiment, it becomes possible to select annotations based on important factors extracted from the factor group of the XAI 100. In the first embodiment, the most important factor 801 is applied in Fig. 8 to Fig. 12, but factors 411 whose importance 802 is second or higher up to a predetermined rank or factors 411 whose importance 802 is equal to or higher than a threshold value may be applied as important factors instead of the most important factor 801 or together with the most important factor 801. EXAMPLES

[0064] Next, a description will be given of Example 2. In Example 1, an example was described in which improvement action information 400 was used as the action information 104, but in Example 2, an example will be described in which risk reduction action information is used. Note that in Example 2, differences from Example 1 will be mainly described, and therefore descriptions of common parts with Example 1 will be omitted.

[0065] <Risk reduction behavior information> 13 is an explanatory diagram showing an example of risk reduction behavior information. The risk reduction behavior information 1300 is information indicating risk reduction behaviors recommended for the factor 411. The risk reduction behavior information 1300 is included in the behavior information 104. For each factor 301 that serves as a factor filter, the risk reduction behavior information 1300 has factor-applicable risk reduction behavior information recommended for the factor 411 when the factor 301 applies, and factor-non-applicable risk reduction behavior information recommended for the factor 411 when the factor 301 does not apply.

[0066] FIG. 13 shows first factor corresponding risk reduction behavior information 1301 recommended for factor 411 (third factor 313: HDL<35) when the first factor 311 (BMI<25) applies, and first factor non-corresponding risk reduction behavior information 1302 recommended for factor 411 (third factor 313: HDL<35) when the first factor 311 (BMI<25) does not apply.

[0067] In this example, the factor 301 includes a first factor 311 to a fifth factor 315, so 20 sets of factor-applicable risk reduction behavior information and factor-unapplicable risk reduction behavior information are included in the behavior information 104. Hereinafter, the first factor-applicable risk reduction behavior information 1301 and the first factor-unapplicable risk reduction behavior information 1302 will be described as examples.

[0068] The factor 411 of the first factor corresponding risk reduction behavior information 1301 and the first factor non-corresponding risk reduction behavior information 1302 is the third factor 313. The rank 1311 indicates the priority of the recommended risk reduction behavior 1312 set in advance. In the first factor corresponding risk reduction behavior information 1301, the recommended risk reduction behavior 1312 is a risk reduction behavior recommended when the first factor 311 (BMI<25) which is a factor filter applies. In the first factor non-corresponding risk reduction behavior information 1302, the recommended risk reduction behavior 1322 is a risk reduction behavior recommended when the first factor 311 (BMI<25) which is a factor filter does not apply. The number in parentheses in the recommended risk reduction behavior 1312 indicates an identification number that uniquely identifies the behavior.

[0069] 14 is an explanatory diagram illustrating an example of the combined information according to the embodiment 2. The combined information 1400 includes an ID number 300, a predicted probability 131 of the objective variable 111, a most important factor 801, an importance level 802, a factor filter 1401, and an annotation 803.

[0070] In the combined information 1400, the factor filter 1401 is information indicating whether or not the sample identified by the ID number 300 corresponds to a specific factor 411. In FIG. 14, the first factor 311 (BMI<25) is used as the specific factor 411, but the generating device may apply each of the second factor 312 to the fifth factor 315 to the factor filter 1401 to generate the combined information 1400.

[0071] In the example of FIG. 14, for the entry 1420, the most important factor 801 is the third factor 313, and the factor filter 1401 (first factor 311: BMI<2 5 ) is “applicable,” the recommended risk reduction behavior 1312 “(3) Walking 8,000 steps / day or more,” which is ranked first in the rank 1311 of the risk reduction behavior information 1301 applicable to the first factor, is added as annotation 803 together with the most important factor 801 (third factor 313: HDL<35).

[0072] For the entry 1430, the most important factor 801 is the third factor 313, and the factor filter 1401 (first factor 311: BMI<2 5 ) is “not applicable,” the recommended risk reduction action 1322 “(7) Replace saturated fatty acids with monounsaturated fatty acids,” which is ranked first in the rank 1311 of the risk reduction action information 1302 for the first factor not applicable, is added as annotation 803 together with the most important factor 801 (third factor 313: HDL<35).

[0073] For the entry 1490, the most important factor 801 is the third factor 313, and the factor filter 1401 (first factor 311: BMI<2 5 ) is “applicable,” the recommended risk reduction behavior 1312 “(3) Walking 8,000 steps / day or more,” which is ranked first in the rank 1311 of the risk reduction behavior information 1301 applicable to the first factor, is added as annotation 803 together with the most important factor 801 (third factor 313: HDL<35).

[0074] The generation device may apply each of the factors 301 to a factor filter 1401 to generate combination information 1400 for each factor filter 1401, or may apply factors 301 selected by a user of the generation device to the factor filter 1401 to generate combination information 1400 for each factor filter 1401.

[0075] In this manner, the generating device generates the combined information 1400 in step S604. As a result, the generating device uses the combined information 1400 to generate the combined information 1400 shown in FIG. 0 Annotation information 1100, 1000, and 1200 as shown in FIG. 12 can be generated and output. EXAMPLES

[0076] Next, a description will be given of Example 3. In Example 3, an example of changing the annotation 803 of the combined information will be described. Note that, in Example 3, the differences from Examples 1 and 2 will be mainly described, and therefore descriptions of common parts with Examples 1 and 2 will be omitted.

[0077] FIG. 15 is an explanatory diagram showing a first modification of the annotation 803 of the combined information 800. In comparison with FIG. 8 In the annotation 803 of No. 10, the recommended improvement behavior 412 in the second factor improvement behavior information 402, “(1) Resistance exercise at least three times a week”, has been changed to “(2) Aerobic exercise at least three times a week”.

[0078] Similarly, the entry 8 In the annotation 803 of 20, the recommended improvement action 412 of the third factor improvement action information 403 is “(3) Walk 8,000 steps.” / day "Above" has been changed to "(6) No smoking."

[0079] Such a change is executed by the user of the generating device by selecting a recommended improvement action to be changed to from the second factor improvement action information 402 and the third factor improvement action information 403 displayed on the display screen, using the input device 203. Also, the generating device receives a signal indicating that a user of a terminal capable of communicating with the generating device has selected a recommended improvement action to be changed to from the second factor improvement action information 402 and the third factor improvement action information 403 displayed on the display screen of the terminal, using the input device 203 of the terminal, thereby executing the change in the generating device.

[0080] Fig. 16 is an explanatory diagram showing a second modification example of the annotation 803 of the combined information 1400. Compared with Fig. 14, in the annotation 803 of the entry 1430, the recommended risk reduction behavior 1322 of the first factor non-corresponding risk reduction behavior information 1302, "(7) Replace saturated fatty acids with monounsaturated fatty acids", has been modified to "(6) Quit smoking".

[0081] Similarly, in the annotation 803 of the entry 1490, the recommended risk reduction behavior 1312 in the first factor corresponding risk reduction behavior information 1301 is “(3) Walk 8000 steps.” / day The above" has been changed to "(5) A diet high in dietary fiber."

[0082] Such a change is executed by the user of the generation device by selecting a new risk reduction action from the first factor relevant risk reduction action information 1301 and the first factor non-relevant risk reduction action information 1302 displayed on the display screen, using the input device 203. Also, the generation device receives a signal indicating that a user of a terminal capable of communicating with the generation device has selected a new risk reduction action from the first factor relevant risk reduction action information 1301 and the first factor non-relevant risk reduction action information 1302 displayed on the display screen of the terminal, using the input device 203 of the terminal, thereby executing the change in the generation device.

[0083] In this way, the generating device can change the annotation 803 of the binding information 800, 900 by external operation. Therefore, the annotation suitable for the sample identified by the ID number 300 803 can be provided. EXAMPLES

[0084] Next, a fourth embodiment will be described. In the fourth embodiment, an example in which combined behavior information is used as behavior information will be described. In the fourth embodiment, differences from the first to third embodiments will be mainly described, and therefore descriptions of common parts with the first to third embodiments will be omitted.

[0085] 17 is an explanatory diagram showing an example of combined behavior information. The combined behavior information 1700 is included in the behavior information 104. The combined behavior information 1700 has a risk reduction factor 1701, a risk amplification factor 1702, and a combined annotation 1703. The combined annotation 1703 is an annotation that combines the risk reduction factor 1701 and the risk amplification factor 1702.

[0086] For example, in the importance matrix 103, the risk reduction factor 1701 in the sample with the ID number 300 of "0030" is "BMI<25", and the risk amplification factor 1702 is "HDL<35". Therefore, in step S604, the generating device selects the combination annotation 1703 of the entry in which the combination of the risk reduction factor 1701 and the risk amplification factor 1702 is "BMI<25" and "HDL<35", and generates the annotation information 1710 (step S605).

[0087] The annotation information 1710 includes the ID number 300, the predicted probability 131 of the objective variable 111, a risk reduction factor 1701, a risk amplification factor 1702, and a combination annotation 1703. In the case of the fourth embodiment, it is not necessary to execute step S604.

[0088] Thus, the embodiment 4 According to the method, a combined annotation 1703 according to the sample can be provided.

[0089] As described above, the above-mentioned Examples 1 to 4 According to this, annotation selection can be optimized.

[0090] The present invention is not limited to the above-described embodiments, and includes various modified examples and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. Also, a part of the configuration of one embodiment may be replaced with a configuration of another embodiment. Also, a configuration of another embodiment may be added to a configuration of one embodiment. Also, a part of the configuration of each embodiment may be added, deleted, or replaced with another configuration.

[0091] Furthermore, each of the aforementioned configurations, functions, processing units, processing means, etc. may be realized in hardware, for example by designing some or all of them as an integrated circuit, or may be realized in software by a processor interpreting and executing a program that realizes each function.

[0092] Information such as programs, tables, files, etc. that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC (Integrated Circuit) card, an SD card, or a DVD (Digital Versatile Disc).

[0093] In addition, the control lines and information lines shown are those considered necessary for the explanation, and do not necessarily show all the control lines and information lines necessary for implementation. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0094] 100XAI 101 Machine Learning Datasets 102 Data to be predicted 103 Importance Matrix 104 Behavioral Information 110 Annotation information generation function 111 Response variable 120 Annotation Information 112 Explanatory variables 131 Predicted Probability 132 Importance 200 calculator 201 Processor 202 Storage Devices 400 Improvement action information 411 factors 412 Recommended Improvement Actions 500 Risk Reduction Action Information 512 Recommended Risk Reduction Actions 700 annotations 800,900,1400 Join information 801 Most important factor 802 Importance 803 Annotation 1000,1100,1200,1710 Annotation information 1300 Risk Reduction Action Information 1401 Factor Filter 1700 Combined Behavior Information 1701 Risk reduction factors 1702 Risk Amplification Factors 1703 Combined Annotation

Claims

1. A generating device having a processor that executes a program and a storage device that stores the program, storing, for each of the factors in the factor group, behavior information in which the factor is associated with a behavior corresponding to the factor; The processor, An acquisition process for acquiring a predicted probability based on the applicability of each factor of the group of factors for a sample and a degree of importance of each factor of the group of factors that is the basis of the predicted probability; an extraction process for extracting a specific factor from the group of factors based on the importance acquired by the acquisition process; a generation process of acquiring a specific behavior corresponding to the specific factor extracted by the extraction process from the behavior information, and generating annotation information that presents the specific behavior for the sample corresponding to the specific factor; A generating device characterized by executing the above.

2. The generating device according to claim 1 , the behavior information includes improvement behavior information in which the factors are associated with improvement behaviors recommended for the sample when the factors are met, In the extraction process, the processor extracts a factor having an importance level greater than a threshold as the specific factor, In the generation process, the processor acquires a specific improvement action corresponding to the specific factor from the improvement action information, and generates annotation information that presents the specific improvement action for the sample corresponding to the specific factor. A generating device characterized by:

3. The generating device according to claim 2, In the extraction process, the processor extracts a factor whose importance is greater than a threshold and is the largest as the specific factor. A generating device characterized by:

4. The generating device according to claim 1 , The behavior information includes risk reduction behavior information that associates the factor with a risk reduction behavior recommended for the sample when the factor applies, In the extraction process, the processor extracts a factor having an importance level smaller than a threshold as the specific factor, In the generation process, the processor acquires a specific risk reduction behavior corresponding to the specific factor from the risk reduction behavior information, and generates annotation information that presents the specific risk reduction behavior to the sample corresponding to the specific factor. A generating device characterized by:

5. The generating device according to claim 4, In the extraction process, the processor extracts a factor whose importance is smaller than a threshold and is the smallest as the specific factor. A generating device characterized by:

6. The generating device according to claim 1 , In the generation process, the processor generates the annotation information for samples whose predicted probabilities are equal to or greater than a predetermined probability or for samples up to a predetermined rank. A generating device characterized by:

7. The generating device according to claim 1 , the behavior information includes, for each second factor other than a first factor among the group of factors, first-factor corresponding risk reduction behavior information associating the second factor with a risk reduction behavior recommended for the sample when the sample corresponds to the first factor and the second factor; In the extraction process, the processor extracts the second factor having the importance smaller than a threshold as the specific second factor, In the generation process, the processor acquires a specific risk reduction behavior corresponding to the specific second factor from the first-factor-corresponding risk reduction behavior information for a specific sample corresponding to the first factor, and generates annotation information presenting the specific risk reduction behavior for the specific sample corresponding to the specific second factor. A generating device characterized by:

8. The generating device according to claim 1 , The behavior information includes, for each second factor other than a first factor among the group of factors, first-factor non-corresponding risk reduction behavior information associating the second factor with a risk reduction behavior recommended for the sample when the sample does not correspond to the first factor but corresponds to the second factor; In the extraction process, the processor extracts the second factor having the importance smaller than a threshold as the specific second factor, In the generation process, the processor acquires a specific risk reduction behavior corresponding to the specific second factor from the first factor non-corresponding risk reduction behavior information for a specific sample not corresponding to the first factor, and generates annotation information presenting the specific risk reduction behavior for the specific sample corresponding to the specific second factor. A generating device characterized by:

9. The generating device according to claim 1 , the behavior information includes combination behavior information that associates a combination of a risk amplification factor and a risk reduction factor in the group of factors with a behavior that corresponds to the combination, In the extraction process, the processor extracts a combination of a specific risk amplification factor and a specific risk reduction factor from the group of factors for each sample based on the importance acquired by the acquisition process; In the generation process, the processor acquires a specific behavior corresponding to the combination of a specific risk amplification factor and a specific risk reduction factor extracted by the extraction process from the combined behavior information, and generates annotation information presenting the specific behavior for the sample corresponding to the combination of the specific risk amplification factor and the specific risk reduction factor. A generating device characterized by:

10. A generation method executed by a generation device having a processor that executes a program and a storage device that stores the program, comprising: the generation device stores, for each of a group of factors, behavior information in which the factor is associated with a behavior corresponding to the factor; The processor, An acquisition process for acquiring a predicted probability based on the applicability of each factor of the group of factors for a sample and a degree of importance of each factor of the group of factors that is the basis of the predicted probability; an extraction process for extracting a specific factor from the group of factors based on the importance acquired by the acquisition process; a generation process of acquiring a specific behavior corresponding to the specific factor extracted by the extraction process from the behavior information, and generating annotation information that presents the specific behavior for the sample corresponding to the specific factor; A generating method comprising:

11. a processor that can access behavior information that associates each of a group of factors with a behavior that corresponds to the factor; An acquisition process for acquiring a predicted probability based on the applicability of each factor of the group of factors for a sample and a degree of importance of each factor of the group of factors that is the basis of the predicted probability; an extraction process for extracting a specific factor from the group of factors based on the importance acquired by the acquisition process; a generation process of acquiring a specific behavior corresponding to the specific factor extracted by the extraction process from the behavior information, and generating annotation information that presents the specific behavior for the sample corresponding to the specific factor; A generating program for causing a user to execute the above steps.

Images