Risk estimation device, risk estimation method, and program

Abstract

To estimate risk with high accuracy. [Solution] In the risk estimation device, the acquisition means acquires data from multiple different modalities. The encoder converts the data from each modality into data representing a probability distribution in a latent space. The predictor predicts the risk corresponding to each modality based on the probability distribution. The calculation means integrates the risks corresponding to each modality using the weights corresponding to each modality to calculate the estimation result. By using the risk estimation device to estimate disease risk, it is possible to support decision-making regarding the lifestyle habits of the subject.

Description

[Technical Field]

[0001] This disclosure relates to risk estimation. [Background technology]

[0002] Techniques for estimating disease risk using machine learning models are known. For example, Patent Document 1 describes a multimodal machine learning model that predicts the progression of dementia using multiple types of input data. In Patent Document 1, the prediction results based on multiple input data are integrated according to the prediction interval, which is the interval from a reference point to a future point in time where the prediction is made, to generate the final prediction result. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] International Publication No. WO2023 / 276976 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] However, the method described in Patent Document 1 integrates prediction results based on multiple input data according to the prediction interval, so it does not necessarily guarantee highly accurate prediction results.

[0005] One objective of this disclosure is to provide a risk estimation device capable of estimating risk with high accuracy. [Means for solving the problem]

[0006] From one perspective of this disclosure, the risk estimation device is A means of acquiring data from multiple different modalities, An encoder that converts data from each modality into data representing the probability distribution in the latent space, A predictor that predicts the risk corresponding to each modality based on the aforementioned probability distribution, Calculation means for calculating an estimation result by integrating risks corresponding to each modality using weights corresponding to each modality; It is provided with.

[0007] In another aspect of the present disclosure, a risk estimation method executed by a computer is as follows. Obtain data of a plurality of different modalities; Convert the data of each modality into data indicating a probability distribution in a latent space; Predict the risk corresponding to each modality based on the probability distribution; Integrate the risks corresponding to each modality using weights corresponding to each modality and calculate an estimation result.

[0008] In still another aspect of the present disclosure, a program causes a computer to execute the following processing. Obtain data of a plurality of different modalities; Convert the data of each modality into data indicating a probability distribution in a latent space; Predict the risk corresponding to each modality based on the probability distribution; Integrate the risks corresponding to each modality using weights corresponding to each modality and calculate an estimation result.

Advantages of the Invention

[0009] According to the present disclosure, it becomes possible to estimate risks with high accuracy.

Brief Description of the Drawings

[0010] [Figure 1] Shows the overall configuration of a risk estimation device according to the present disclosure. [Figure 2] It is a block diagram showing the hardware configuration of a risk estimation device. [Figure 3] It is a block diagram showing the functional configuration of a learning device for a risk estimation model. [Figure 4] It is a flowchart of a learning process. [Figure 5] It is a block diagram showing the functional configuration of a risk estimation device. [Figure 6] It is a flowchart of risk estimation processing. [Figure 7] It is a block diagram showing other functional configurations of the risk estimation device. [Figure 8] It is a block diagram showing other functional configurations of the risk estimation device. [Figure 9] It is a flowchart of other risk estimation processing.

Embodiments for Carrying Out the Invention

[0011] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the drawings. <First Embodiment> [Overall Configuration] FIG. 1 shows the overall configuration of a risk estimation device according to the present disclosure. The risk estimation device 100 estimates the disease risk of a subject based on data related to the health of the subject. Specifically, multi-modal data, that is, data of a plurality of different modalities, is input to the risk estimation device 100. Here, the modality means a method or means for expressing information, and multi-modal data refers to data of different data formats such as, for example, text, image, audio, and sensor data. In the present embodiment, the multi-modal data includes, for example, various data obtained from a health check such as the height, weight, gender, blood pressure, BMI (Body Mass Index), body fat percentage, triglyceride level, presence or absence and amount of smoking, presence or absence and amount of drinking, etc. of the subject.

[0012] As shown in Figure 1, the risk estimation device 100 receives multiple data from different modalities (in this example, data D1 to D3). The risk estimation device 100 predicts the disease risk based on the data from each input modality and integrates the prediction results from each modality to output the final estimation result. In this process, the risk estimation device 100 converts the data from each modality into a probability distribution in latent space and integrates the prediction results from each modality according to the similarity between the obtained probability distribution and a predetermined reference distribution. This allows the risk estimation device 100 to integrate the prediction results from each modality in an appropriate proportion according to the characteristics of the data from each modality, enabling high-precision estimation of disease risk.

[0013] The risk estimation device 100 can be suitably applied to the medical or healthcare field. For example, the risk estimation device 100 can be used to estimate the risk of lifestyle-related diseases based on data obtained from regular health checkups.

[0014] [Hardware configuration] Figure 2 is a block diagram showing the hardware configuration of the risk estimation device 100. As shown in the figure, the risk estimation device 100 comprises a processor 11, an interface (IF) 12, a ROM (Read Only Memory) 13, a RAM (Random Access Memory) 14, a database (DB) 15, and a storage medium 16. Each component is connected to the others, for example, via a bus 18.

[0015] The processor 11 is a computer such as a CPU (Central Processing Unit) and controls the entire risk estimation device 100 by executing a pre-prepared program. Specifically, the processor 11 can be a CPU, GPU (Graphics Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, microcontroller, or a combination thereof.

[0016] Furthermore, the processor 11 loads the program stored in the ROM 13 or storage medium 16 into the RAM 14 and executes each process coded in the program. The processor 11 functions as part or all of the risk estimation device 100. Specifically, the processor 11 performs the learning process and risk estimation process described later.

[0017] IF12 transmits and receives data to and from external devices. Specifically, during the learning phase, the risk estimation device 100 receives multimodal data of multiple individuals as learning data through IF12. During the estimation phase, i.e., when estimating risk, the risk estimation device 100 receives multimodal data of the subject through IF12 and outputs the disease risk estimation result to a display device or other external device.

[0018] ROM 13 stores various programs executed by processor 11. RAM 14 is used as working memory while processor 11 is executing various processes.

[0019] DB15 stores various algorithms, data, machine learning models, etc., that the risk estimation device 100 uses when it performs the learning process and risk estimation process described later.

[0020] The storage medium 16 is a non-volatile, non-temporary storage medium such as a disk-shaped recording medium or semiconductor memory. The storage medium 16 may be configured to be detachable from the risk estimation device 100. The storage medium 16 records various programs executed by the processor 11.

[0021] In addition to the above, the risk estimation device 100 may also be equipped with a display device such as a liquid crystal display, and an input device such as a keyboard or mouse. These display devices and input devices are used, for example, by the operator of the risk estimation device 100.

[0022] [Learning Phase] Next, we will explain the learning phase of the risk estimation model. (Learning device) The risk estimation device 100 estimates disease risk using a pre-trained risk estimation model. In the following, the risk estimation model is assumed to estimate disease risk from data D1-D3 of three different modalities, but the number of data types constituting the multimodal data is not limited to this.

[0023] Figure 3 is a block diagram showing the functional configuration of the learning device 20 for the risk estimation model. The learning device 20 comprises an encoder unit 21, a prediction unit 22, an integration unit 23, loss calculation units 24 and 25, a loss integration unit 26, and an optimization unit 27. The encoder unit 21 includes encoders 21a to 21c corresponding to modalities 1 to 3. The prediction unit 22 includes predictors 22a to 22c corresponding to modalities 1 to 3.

[0024] The risk estimation model consists of an encoder unit 21, a prediction unit 22, and an integration unit 23. Specifically, the encoder unit 21 and the prediction unit 22 are composed of neural networks. In the learning phase, the learning device 20 optimizes this neural network using training data and optimizes the weights used by the integration unit 23 to generate a trained risk estimation model.

[0025] The training data consists of multimodal disease risk data from multiple individuals. Specifically, the training data is collected from multiple individuals, including their attribute data and disease risk values. Attribute data such as height, weight, gender, blood pressure, BMI, triglyceride levels, blood glucose levels, smoking habits (both whether or not they smoke and the amount they smoke), are used if they are highly relevant to the disease risk being estimated. Each individual's disease risk value corresponds to the ground truth data in supervised learning, and this will also be referred to as the "true value" below. For example, suppose we use blood pressure, BMI, and triglyceride levels as data D1-D3, and estimate the risk of heart disease. In this case, the training data would include blood pressure, BMI, and triglyceride levels as input data from multiple individuals, and the presence or absence of heart disease as the true value.

[0026] In Figure 3, data D1 to D3 for each modality 1 to 3 are input to the encoder unit 21. Data D1 is input to encoder 21a, data D2 is input to encoder 21b, and data D3 is input to encoder 21c. Each encoder 21a to 21c projects the input data into the latent space. The "latent space" is an abstract space for representing the information contained in the original data in fewer dimensions, and in the latent space, the essential features and patterns of the data are represented in fewer dimensions. "Projecting into the latent space" means converting the original data into points on the latent space, and this is also called "mapping to the latent space".

[0027] Next, each encoder 21a to 21c calculates the probability distribution in the latent space of the input modality data D1 to D3 and outputs probability distribution data that shows that probability distribution. Specifically, the probability distribution data includes the mean μ, the standard deviation σ, and the latent representation z. The latent representation z is given by the following equation (1) and is also called the intermediate representation, hidden representation, or latent variable.

number

[0028] The probability distribution data output by each encoder 21a to 21c is input to the predictor 22a to 22c of the corresponding modality. In addition, the probability distribution data output by each encoder 21a to 21c is input to the loss calculation unit 25.

[0029] Each predictor 22a to 22c calculates disease risk scores (hereinafter referred to as "risk scores") s1 to s3 corresponding to the data of each modality based on the input latent representation z, and outputs them to the integration unit 23.

[0030] The integration unit 23 calculates the integrated risk score S by weighting and adding the risk scores s1 to s3 of each modality. Specifically, if the weights of modalities 1 to 3 are w1 to w3 respectively, the integration unit 23 calculates the integrated risk score S using the following equation (2) and outputs it to the loss calculation unit 24.

[0031]

number

[0032] The loss calculation unit 24 calculates the cross-entropy loss L between the integrated risk score S and the true values ​​corresponding to its input data D1 to D3. cross-entropy This is output to the loss integration unit 26.

[0033] Meanwhile, the loss calculation unit 25 uses the probability distribution data input from each encoder 21a to 21c to calculate the similarity between the probability distribution of each modality and the reference distribution. When the input data D is real data, a normal distribution is used as the reference distribution. Therefore, the loss calculation unit 25 uses the mean μ and standard deviation σ of each modality to calculate the KL (Kullback-Leibler) divergence between the probability distribution of each modality and the normal distribution N(0,1) as the loss L using the following equation (3). KL Calculate as follows.

[0034]

number

[0035] If the input data is not real-valued data, the loss calculation unit 25 can use, as a reference distribution, a lognormal distribution, a Poisson distribution, a multinomial logit, an ordered logit, etc., according to the format of the input data D.

[0036] The loss integration unit 26 calculates a weighted sum of the loss L KL and the loss L cross-entropy and outputs it to the optimization unit 27 as the total loss L total

[0037]

Equation

[0038] The optimization unit 27 optimizes the encoder unit 21 and the prediction unit 22 based on the total loss L total and also optimizes the weights w1 to w3 used by the integration unit 23. Specifically, the optimization unit 27 optimizes the parameters of the neural network constituting the encoder unit 21 and the prediction unit 22 so that the total loss L total becomes smaller, and also optimizes the weights w1 to w3 used by the integration unit 23. Here, since the total loss L total is the weighted sum of the loss L KL and the loss L cross-entropy the optimization unit 27 performs optimization so that the KL divergence between the probability distribution output by each encoder 21a to 21c and the reference distribution becomes smaller, that is, so that the similarity between the probability distribution and the reference distribution becomes higher. At the same time, the optimization unit 27 performs optimization so that the error between the integrated risk score S output by the integration unit 23 and the true value becomes smaller.

[0039] ​Through the optimization described above, risk scores s of modalities with probability distributions that are highly similar to the reference distribution are given high confidence and are therefore reflected in the combined risk score S with a large weight. Conversely, risk scores s of modalities with probability distributions that are less similar to the reference distribution are given low confidence and are therefore reflected in the combined risk score S with a small weight. In this way, the trained risk estimation model can calculate the combined risk score S using appropriate weights according to the characteristics of the modalities in the input data.

[0040] (Learning process) Next, the learning process performed by the learning device 20 described above will be explained. Figure 4 is a flowchart of the learning process. This process is realized when the processor 11 shown in Figure 2 executes a pre-prepared program and operates as the components shown in Figure 3.

[0041] First, the encoder unit 21 acquires data for each modality included in the training data (step S11). Next, the encoder unit 21 projects each data into the latent space using each encoder 21a to 21c to generate probability distribution data (step S12). Next, the prediction unit 22 calculates risk scores s1 to s3 for each modality using each predictor 22a to 22c (step S13). Next, the integration unit 23 integrates the risk scores s1 to s3 for each modality using weights w1 to w3 to calculate an integrated risk score S (step S14).

[0042] Next, the loss calculation unit 24 calculates the loss L based on the integrated risk score S and the true value. cross-entropy The loss calculation unit 25 calculates the loss L using the mean μ and standard deviation σ of each modality. KL The loss L is calculated (step S16). Next, the loss integration unit 26 calculates the loss L cross-entropy and loss L KL Total loss L total The calculation is performed (step S17). Next, the optimization unit 27 calculates the total loss L total Based on this, the parameters of the encoder unit 21 and the prediction unit 22, and the weights w1 to w3 of the integration unit 23 are optimized (step S18).

[0043] Next, the learning device 20 determines whether predetermined learning termination conditions have been met (step S19). Examples of learning termination conditions include using a predetermined number of attribute data prepared as learning data, the total loss falling below a predetermined value, and the total loss converging. If the learning termination conditions are not met (step S19: No), the process returns to step S12. On the other hand, if the learning termination conditions are met (step S19: Yes), the learning process ends.

[0044] [Estimated Phase] Next, the estimation phase by the risk estimation device will be described. In the estimation phase, the risk estimation device 100 estimates the disease risk of a given subject based on their multimodal data. In this process, the risk estimation device 100 uses the risk estimation model trained in the learning phase, specifically the encoder unit 21, the prediction unit 22, and the integration unit 23.

[0045] (Risk estimation device) Figure 5 is a block diagram showing the functional configuration of the risk estimation device. The risk estimation device 100 includes an encoder unit 21, a prediction unit 22, and an integration unit 23 that are optimized during the learning phase.

[0046] The encoder unit 21 receives data D1 to D3 from three different modalities for a given subject. Each encoder 21a to 21c projects the input data D1 to D3 into the latent space, generates probability distribution data including the mean μ, standard deviation σ, and latent representation z, and outputs it to each predictor 22a to 22c.

[0047] Each predictor 22a to 22c calculates risk scores s1 to s3 for each modality based on the input latent representation z and outputs them to the integration unit 23. The integration unit 23 uses weights w1 to w3 optimized in the learning phase to weight and sum the risk scores s1 to s3 for each modality, outputting an integrated risk score S. In this way, it becomes possible to predict the disease risk for a specific subject using the trained risk estimation model.

[0048] (Risk estimation process) Next, the risk estimation process performed by the risk estimation device 100 described above will be explained. Figure 6 is a flowchart of the risk estimation process. This process is realized when the processor 11 shown in Figure 2 executes a pre-prepared program and operates as the components shown in Figure 5.

[0049] First, the encoder unit 21 acquires data D1 to D3 for each modality for the subject (step S21). Next, the encoder unit 21 generates probability distribution data from the data D1 to D3 (step S22). Next, the prediction unit 22 calculates risk scores s1 to s3 for each modality using the probability distribution data (step S23). Next, the integration unit 23 calculates and outputs an integrated risk score S from the risk scores s1 to s3 of each modality using the weights w1 to w3 optimized in the learning phase (step S24). Then, the risk estimation process is completed.

[0050] [Differentiation] The following describes some modifications of the above embodiment. These modifications can be combined as appropriate and applied to the above embodiment.

[0051] (First variation) In the above embodiment, the weights w(w1~w3) of the integration unit 23 are determined in the learning phase using data from multiple individuals. In contrast, in the inference phase, disease risk is estimated using data from a subject, so the weights w determined in the learning phase are not necessarily optimal for that subject. For example, there may be individual circumstances where the reliability of data from a certain modality is low for a particular subject. For example, subject X may have unstable blood pressure measurement data with low reliability.

[0052] From this perspective, the first modified version allows the weights w used by the integration unit 23 to be corrected during the inference phase. Figure 7 shows the functional configuration of the risk estimation device 100x according to the first modified version. As can be understood by comparing it with Figure 5, the risk estimation device 100x according to the first modified version includes a weight correction unit 28 in addition to the configuration of Figure 5.

[0053] The weight correction unit 28 receives probability distribution data from each encoder 21a to 21c. The weight correction unit 28 corrects the weights w(w1 to w3) used by the integration unit 23 based on the similarity between the probability distribution obtained from the data D1 to D3 input to each encoder 21a to 21c in the inference phase and the reference distribution. Specifically, the weight correction unit 28 calculates the KL divergence between the probability distribution obtained from each data D1 to D3 and the reference distribution, and calculates a correction coefficient q based on the obtained KL divergence. i To decide.

[0054] Specifically, the weight correction unit 28 determines the correction coefficient qi using the following formula.

number

[0055] In other words, the weight correction unit 28 determines that the KL divergence of a certain modality i is at a threshold t. KL If it is greater, the correction factor q iThis is set to "0". As a result, if the similarity between the probability distribution of modality i and the reference distribution is low, the risk score of that modality will be considered to have low confidence and will not be reflected in the combined risk score. On the other hand, the weight correction unit 28 sets the KL divergence of a certain modality to threshold t KL In the following cases, the correction coefficient q i This is set to "1". As a result, if the similarity between the probability distribution of modality i and the reference distribution is high, the risk score of that modality is reflected in the integrated risk score at the rate determined in the learning phase. In this way, by correcting the weight w of the integration unit 23 based on the data actually input in the inference phase, it becomes possible to estimate disease risk according to the individual characteristics of the subject.

[0056] Note that the threshold t KL The value of may be a common value for all modalities, or it may be a different value for each modality. Also, the threshold t KL The value of may be the same for all subjects in the disease risk estimation, or it may be a different value for each subject.

[0057] (Second variation) In the first embodiment described above, the risk estimation device is applied to the generation of attribute data relating to human health, but the application of this disclosure is not limited to this. For example, this disclosure may be applied to the inspection and diagnosis of machines and devices. That is, the method of this disclosure may be applied to estimate the state of a machine or device based on data from multiple modalities detected and collected in an inspection or diagnosis.

[0058] <Second Embodiment> Figure 8 is a block diagram showing the functional configuration of the risk estimation device according to the second embodiment. The risk estimation device 70 comprises an acquisition means 71, an encoder 72, a predictor 73, and a calculation means 74.

[0059] Figure 9 is a flowchart of the processing performed by the risk estimation device of the second embodiment. The acquisition means 71 acquires data from multiple different modalities (step S71). The encoder 72 converts the data from each modality into data representing the probability distribution in the latent space (step S72). The predictor 73 predicts the risk corresponding to each modality based on the probability distribution (step S73). The calculation means 74 calculates the estimation result by integrating the risks corresponding to each modality using the weights corresponding to each modality (step S74).

[0060] According to the risk estimation device 70 of the second embodiment, it is possible to estimate risk with high accuracy.

[0061] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0062] (Note 1) A means of acquiring data from multiple different modalities, An encoder that converts data from each modality into data representing the probability distribution in the latent space, A predictor that predicts the risk corresponding to each modality based on the aforementioned probability distribution, A computational means for integrating the risks corresponding to each modality using the weights corresponding to each modality to calculate the estimation result, A risk estimation device equipped with the following features.

[0063] (Note 2) The risk estimation device according to Appendix 1, comprising an optimization means for optimizing the weights corresponding to each modality based on the similarity between the probability distribution corresponding to each modality and a predetermined reference distribution.

[0064] (Note 3) The risk estimation device according to Appendix 2, wherein the optimization means sets the weight to a larger value the higher the similarity between the probability distribution and the reference distribution, and sets the weight to a smaller value the lower the similarity between the probability distribution and the reference distribution.

[0065] (Note 4) The data showing the aforementioned probability distribution includes the mean and standard deviation. The risk estimation device described in Appendix 2, in which the similarity is represented by the KL divergence between the probability distribution and the reference distribution.

[0066] (Note 5) The risk estimation device according to Appendix 1, comprising a weight correction means for correcting the weights corresponding to each modality based on the probability distribution corresponding to each modality.

[0067] (Note 6) The weight correction means is a risk estimation device according to Appendix 5 that calculates a correction coefficient for correcting the weights corresponding to each modality based on the similarity between the probability distribution of each modality and the reference distribution.

[0068] (Note 7) The weight correction means is a risk estimation device according to Appendix 6, wherein the correction coefficient is set to 0 when the similarity is greater than a predetermined threshold, and the correction coefficient is set to 1 when the similarity is less than or equal to a predetermined threshold.

[0069] (Note 8) The predictor is a risk estimation device as described in Appendix 1, which uses a trained machine learning model to predict the disease risk of a subject based on data from multiple modalities related to the subject's health.

[0070] (Note 9) A risk estimation method performed by a computer, By obtaining data from multiple different modalities, The data for each modality is transformed into data that represents the probability distribution in the latent space. Based on the aforementioned probability distribution, the risk corresponding to each modality is predicted. A risk estimation method that calculates an estimation result by integrating the risks associated with each modality using the weights corresponding to each modality.

[0071] (Note 10) By obtaining data from multiple different modalities, The data for each modality is transformed into data that represents the probability distribution in the latent space. Based on the aforementioned probability distribution, the risk corresponding to each modality is predicted. A program that causes a computer to perform a process of integrating the risks associated with each modality using the corresponding weights for each modality and calculating an estimation result.

[0072] Although the present disclosure has been described above with reference to embodiments and examples, the present disclosure is not limited to the above embodiments and examples. Various modifications to the structure and details of the present disclosure can be understood by those skilled in the art within the scope of the present disclosure. [Explanation of symbols]

[0073] 11 processors 20 Learning device 21 Encoder section 21a~21c encoder 22 Prediction Section 22a~22c Predictor 23. Integration Department 24, 25 Loss calculation section 26 Loss integration section 27 Optimization Section 28 Weight Correction Unit 100, 100x Risk Estimator

Claims

1. A means of acquiring data from multiple different modalities, An encoder that converts data from each modality into data representing the probability distribution in the latent space, A predictor that predicts the risk corresponding to each modality based on the aforementioned probability distribution, A computational means for integrating the risks corresponding to each modality using the weights corresponding to each modality to calculate the estimation result, A risk estimation device equipped with the following features.

2. The risk estimation device according to claim 1, further comprising an optimization means for optimizing the weights corresponding to each modality based on the similarity between the probability distribution corresponding to each modality and a predetermined reference distribution.

3. The risk estimation device according to claim 2, wherein the optimization means sets the weight to a larger value the higher the similarity between the probability distribution and the reference distribution, and sets the weight to a smaller value the lower the similarity between the probability distribution and the reference distribution.

4. The data showing the aforementioned probability distribution includes the mean and standard deviation. The risk estimation device according to claim 2, wherein the similarity is represented by the KL divergence between the probability distribution and the reference distribution.

5. The risk estimation device according to claim 1, further comprising a weight correction means for correcting the weights corresponding to each modality based on the probability distribution corresponding to each modality.

6. The risk estimation device according to claim 5, wherein the weight correction means calculates a correction coefficient for correcting the weights corresponding to each modality based on the similarity between the probability distribution of each modality and the reference distribution.

7. The risk estimation device according to claim 6, wherein the weight correction means sets the correction coefficient to 0 when the similarity is greater than a predetermined threshold, and sets the correction coefficient to 1 when the similarity is less than or equal to a predetermined threshold.

8. The risk estimation device according to claim 1, wherein the predictor predicts the disease risk of a subject based on data from multiple modalities related to the subject's health using a trained machine learning model.

9. A risk estimation method performed by a computer, By obtaining data from multiple different modalities, The data for each modality is transformed into data that represents the probability distribution in the latent space. Based on the aforementioned probability distribution, the risk corresponding to each modality is predicted. A risk estimation method that calculates an estimation result by integrating the risks associated with each modality using the weights corresponding to each modality.

10. By obtaining data from multiple different modalities, The data for each modality is transformed into data that represents the probability distribution in the latent space. Based on the aforementioned probability distribution, the risk corresponding to each modality is predicted. A program that causes a computer to perform a process of integrating the risks associated with each modality using the corresponding weights for each modality and calculating an estimation result.

Images