Learning data generation method, program, prediction model, and prediction system

Abstract

To realize a technique capable of realizing more efficient and accurate machine learning and improving prediction accuracy of disease onset.SOLUTION: A learning data generation method includes steps of: generating multiple labels that associate a range of measurement values defined by start and end values with a numerical value related to a crisis rate of the disease from clinical data that associates measurement values obtained as continuous type variables or discrete type variables with the presence or absence of the onset of a specific disease; and generating a label pattern consisting of labels that consecutively arrange the range of measurement values by extracting the labels generated by the label generation processing with reference to the numerical value related to the crisis rate of the disease.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a training data generation method, a program, a prediction model, and a prediction system, and more particularly to a training data generation method, a program, a prediction model, and a prediction system that realizes more efficient and highly accurate machine learning and realizes technology that can improve the accuracy of predicting the onset of disease. [Background technology]

[0002] It is known that blockage or damage in the coronary arteries due to the accumulation of plaque can cause diseases such as heart disease. The Coronary Artery Calcification Score (CACS) is used as an index to indicate the degree of plaque accumulation. It is believed that the CACS value of a subject is useful for predicting the onset of the subject's disease.

[0003] For example, a technique has been proposed in which a set of images is analyzed to create a three-dimensional (3D) bounding volume that includes the heart, reticles are added on the created bounding volume to divide the bounding volume into regions, each region is analyzed to identify calcified plaques that exceed a predetermined value, and a calcium score is generated for identified calcified plaques that exceed the predetermined value (see, for example, Patent Document 1).

[0004] In recent years, technology has been developed that uses AI to input information such as the subject's measurements obtained during tests and predict the onset of disease in subjects and patients. For example, when predicting the onset of heart disease, the accuracy of predictions can be improved by using CACS as one of the measurements.

[0005] In general, when creating an AI predictive model, the preprocessing of the variables used affects the prediction accuracy. For example, it is desirable to convert continuous or discrete variables used as explanatory variables into ordinal numbers. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2008-161688 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, the distribution of the disease incidence rate for the measured values ​​included in clinical information such as CACS is not necessarily normal. Therefore, even if the widely used ordinal numbering methods (equal width binning, equal frequency binning, etc.) are used to convert the data into ordinal numbers, it is difficult to improve the prediction accuracy.

[0008] One aspect of the present invention aims to realize more efficient and accurate machine learning, and to realize technology that can improve the accuracy of predicting the onset of disease. [Means for solving the problem]

[0009] A training data generation method according to one aspect of the present invention is a method for generating training data used in machine learning of a predictive model that predicts the onset of a disease, and includes a label generation step of generating a plurality of labels that correspond to a range of measurement values ​​defined by a start value and an end value and a numerical value related to the incidence rate of the disease from clinical data in which measurement values ​​obtained as continuous variables or discrete variables are associated with the presence or absence of the onset of a predetermined disease, and a label pattern generation step of generating a label pattern consisting of labels in which the range of measurement values ​​is arranged consecutively by extracting the labels generated by the processing of the label generation step with reference to the numerical value related to the incidence rate of the disease.

[0010] Each aspect of the present invention may be realized by a computer. In this case, a program that causes a computer to execute each step of the above-described method, and a computer-readable recording medium having the program recorded thereon, also fall within the scope of the present invention.

[0011] A predictive model according to one embodiment of the present invention predicts the onset of the disease using model parameters trained by an arbitrary machine learning algorithm using training data generated by a training data generation method according to one embodiment of the present invention.

[0012] A prediction system according to one embodiment of the present invention accepts input of measurements including a patient's CACS, and predicts the onset of MACE for the patient based on the input measurements using a prediction model according to one embodiment of the present invention. Effect of the Invention

[0013] According to one aspect of the present invention, more efficient and accurate machine learning can be realized, thereby improving the accuracy of predicting the onset of disease. [Brief description of the drawings]

[0014] [Figure 1] 1 is a graph showing the distribution of the number of people who develop a disease corresponding to the measured values ​​included in the clinical data. [Diagram 2] FIG. 2 is a diagram illustrating an example of supervised binning of the same distribution as in FIG. 1 using the 95% confidence interval of the odds ratio for the development of MACE. [Diagram 3] FIG. 13 is a diagram showing an example of labels associating odds ratios, upper and lower limits of 95% confidence intervals, and CI difference values ​​with each binned CACS range. [Figure 4] 11 is a flowchart illustrating an example of a learning data generation process. [Diagram 5] FIG. 13 is a simplified diagram showing a label whose starting value in the CACS range is 125. [Figure 6] FIG. 13 is a simplified diagram showing a label whose starting value in the CACS range is 925. [Figure 7] 11 is a flowchart illustrating an example of a learning label pattern creation process. [Figure 8] 13 is a flowchart illustrating an example of a 25 multiple label production process. [Figure 9] 11 is a flowchart illustrating an example of a first pattern creation process. [Figure 10] 13 is a flowchart illustrating an example of a MAX label creation process. [Figure 11] FIG. 13 is a diagram showing an example of a label created by executing a MAX label creation process. [Figure 12] 13 is a flowchart illustrating an example of a final label determination process. [Figure 13] 11 is a flowchart illustrating an example of a second pattern creation process. [Figure 14] 13 is a flowchart illustrating an example of a label pattern selection process. [Figure 15] 13A and 13B are diagrams showing examples of label patterns created by another example of creating learning label patterns. [Figure 16] 1 is a block diagram showing an example of the configuration of a disease onset prediction system according to one embodiment of the present invention. [Figure 17] FIG. 13 is a diagram showing an example of a screen displayed on a display of an output unit. [Figure 18] FIG. 13 is a diagram showing another example of a screen displayed on the display of the output unit. [Figure 19] FIG. 13 is a diagram showing yet another example of a screen displayed on the display of the output unit. [Figure 20] FIG. 1 is a diagram illustrating an example of the configuration of a computer that executes instructions of a program, which is software that realizes each function. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. First, problems with the conventional technology will be described.

[0016] Figure 1 is a graph showing the distribution of the number of subjects who developed diseases corresponding to the measured values ​​contained in the clinical data. In this example, the Coronary Artery Calcification Score (CACS) values ​​of multiple subjects are used as the measured values, and the distribution of the number of subjects who developed MACE (Major Adverse Cardiovascular Events) corresponding to the measured values ​​is shown. In the distribution in Figure 1, the average CACS value of subjects who developed MACE was approximately 270.

[0017] It is also possible to use machine learning to train model parameters for a model that predicts the incidence of MACE using measurements (CACS) as explanatory variables, and predict the onset of MACE from input measurements. In such machine learning, it is preferable to create labels by converting the CACS values ​​into ordinal variables, rather than creating labels by associating the CACS values ​​(integer values) of individual subjects with the presence or absence of MACE onset.

[0018] In other words, the CACS value, which is the explanatory variable of the model for predicting the incidence of MACE, is a continuous or discrete variable, and when performing machine learning on a model using labels with such variables as explanatory variables, it is difficult to improve the prediction accuracy unless a very large number of labels are created.

[0019] On the other hand, if continuous or discrete variables are converted into ordinal variables and labels are created, it is expected that prediction accuracy will improve even if machine learning of the model is performed using a relatively small number of labels. One method for converting continuous or discrete variables into ordinal variables is to divide the continuous or discrete variables into multiple ranges (binning). In conventional technology, binning was generally performed using methods such as equal-width binning and equal-frequency binning.

[0020] Equal width binning is a method of dividing a range of values ​​into bins of equal width. For example, if you have values ​​between 0 and 100 and you divide them into 10 bins, each of the 10 bins will have a width of 10. Equal frequency binning is a method of dividing data points into an equal number of bins. For example, if you have 100 data points and you want to divide them into 10 bins, you will divide them so that each bin contains 10 data points.

[0021] The dashed rectangular box 11 in Figure 1 shows an example of binning the CACS values ​​of the distribution shown by the graph. In this example, the range of CACS values ​​1 to 300 shown in box 11 is treated as one bin, and equal-width binning is used to convert continuous or discrete variables into ordinal variables. Box 11 contains the average CACS value of subjects who developed MACE.

[0022] However, in order to improve the prediction accuracy of the model predicting the incidence of MACE, it is necessary to improve the accuracy of the labels used in machine learning. For example, since the distribution in Figure 1 is not a normal distribution, if equal-width binning is performed, the 95% confidence interval of the odds ratio of the occurrence of MACE becomes large, making it difficult to improve the accuracy of the labels.

[0023] For this reason, in this embodiment, supervised binning is used to convert measurements that are continuous or discrete variables into ordinal variables.

[0024] FIG. 2 is a diagram for explaining an example of supervised binning of the same distribution as in FIG. 1 using the 95% confidence interval of the odds ratio of the onset of MACE. The rectangular dashed frame 12a in FIG. 2 indicates that the CACS value range of 1 to 150 is set as one bin. The rectangular dashed frame 12b in FIG. 2 indicates that the CACS value range of 150 to 400 is set as one bin. The rectangular dashed frame 12c in FIG. 2 indicates that the CACS value range of 400 to MAX is set as one bin. MAX represents the maximum value of the measured value.

[0025] By performing binning as shown in boxes 12a to 12c in Figure 2, it is possible to bin the non-normal distribution in Figure 2 so that the 95% confidence interval of the odds ratio of MACE onset becomes smaller. In this way, if the CACS values ​​are converted into ordinal variables to create labels so that the 95% confidence interval of the odds ratio of MACE onset becomes smaller, the accuracy of the labels can be improved, and the prediction accuracy of the model that predicts the incidence of MACE can be improved.

[0026] <First embodiment> Next, a first embodiment of the present invention will be described. In this embodiment, an example of a method for generating learning data used in machine learning of a model predicting the onset of a predetermined disease will be described. Here, an example of a method for generating a label indicating the onset rate of MACE corresponding to a CACS value will be described as learning data used in machine learning of a model predicting the onset of MACE as an example of a disease.

[0027] As an example, tests are performed on about one thousand and several hundred subjects, and CACS is measured along with information such as height, weight, and age. A clinical diagnosis is then performed based on the test results, and the presence or absence of MACE is diagnosed for each subject. Clinical data is obtained that is information on the diagnostic results for each subject obtained in this way, and is made up of information that corresponds to the measured values ​​and the presence or absence of MACE. A label is then generated from the clinical data that indicates the incidence rate of MACE corresponding to the CACS value.

[0028] Prior to generating labels, the CACS values ​​are binned by dividing them according to a predetermined criterion. For example, the CACS values ​​are divided into predetermined ranges by multiples of a natural number N. For example, if N is 25, the CACS values ​​are divided as follows: 1-25, 1-50, 1-75, ...25-50, 25-75, ...75-100, ...

[0029] In this way, variables related to the range of the measurement value are set, and multiple labels having the range of the measurement value corresponding to the set variables are generated. The incidence rate of MACE for the label is calculated as an odds ratio against a preset comparison target (for example, when the CACS value is 0).

[0030] When calculating the odds ratio of the label, a CACS value (e.g., 0) to be compared in the calculation of the odds ratio is selected. Then, among the subjects whose measured CACS value was 0, the number of subjects who developed MACE (a) and the number of subjects who did not develop MACE (b) are identified. In addition, among the subjects whose measured CACS value was within the range of the measured value of the label (e.g., 100 to 125), the number of subjects who developed MACE (c) and the number of subjects who did not develop MACE (d) are identified.

[0031] In this case, it can be expressed as the odds ratio OR=(a / c) / (b / d).

[0032] In addition, the upper and lower limits of the 95% confidence interval of the odds ratio are given by the following formula.

number

[0033] The odds ratio OR, the upper and lower limits of the 95% confidence interval of the odds ratio, and the CI difference value calculated in this way are associated with each of the binned CACS ranges. In this way, data such as that shown in Figure 3 is generated. In other words, multiple labels including the CI difference value for the 95% confidence interval of the odds ratio are generated.

[0034] "col_name" in Figure 3 indicates the range of the binned CACS. In this example, "cacs1_25", "cacs1_50", "cacs1_75", and ... are shown, which indicate that the CACS ranges are 1 to 25, 1 to 50, 1 to 75, and ..., respectively.

[0035] Note that "cacs0_ref" indicates that the cacs value to be compared in calculating the odds ratio is 0.

[0036] In the example of Fig. 3, all start values ​​of the CACS range are 1, but the start value of the CACS range can be any multiple of 25, such as 25, 50, 75, 100, ... Note that the start value of 1 is the only exception where it is not a multiple of 25. Also, the end value of the CACS range is also a multiple of 25, such as 25, 50, 75, 100, ..., but the maximum value of the measured CACS (suitably referred to as MAX) is the only exception where it does not have to be a multiple of 25.

[0037] In addition, "odds_ratio" in FIG. 3 indicates the odds ratio corresponding to the range specified by "col_name". "ci_upper" indicates the upper limit of the 95% confidence interval of the odds ratio corresponding to the range specified by "col_name". "ci_lower" indicates the lower limit of the 95% confidence interval of the odds ratio corresponding to the range specified by "col_name". "ci_upper-ci_lower" indicates the CI difference value corresponding to the range specified by "col_name".

[0038] In this way, the training data generation method of the present embodiment is a method for generating training data used for training a model for predicting the onset of a disease, and includes a process of generating a plurality of labels that associate a range of measurement values, defined by a start value and an end value, with a numerical value relating to the onset rate of a specified disease, from clinical data in which measurement values ​​obtained as continuous variables or discrete variables are associated with the presence or absence of the onset of the disease.

[0039] (Learning data generation process) Next, an example of a learning data generating process, which is an example of a learning data generating method according to the present embodiment, will be described.

[0040] In this process, for example, labels generated as shown in Figure 3 are extracted by referring to numerical values ​​related to the incidence rate of the disease, thereby generating a label pattern consisting of labels that consecutively arrange the ranges of measurement values.

[0041] 4 is a flowchart for explaining an example of a learning data generation process. This process, for example, selects labels suitable for model learning and generates a label pattern consisting of multiple labels. This process is executed by, for example, a general-purpose computer.

[0042] In step S11, the computer CPU sets variables and conditions. The variables are, for example, a natural number N that is a reference for the range of the CACS, and a cutoff value that is an observation threshold of the CACS. In addition, a threshold value of the CI difference value is also set. The condition is, for example, the number of labels obtained as a result of the processing in FIG. 4.

[0043] The variables and conditions may be set by automatically selecting from pre-specified candidates, or may be set by user input. Here, it is assumed that the natural number N serving as the reference for the CACS range is set to 25, and the cutoff value of the CACS is set to 1000. In addition, a case where the threshold value of the CI difference value is set to 10, and the number of labels as a condition is any one of 3 to 10 will be described as an example.

[0044] In step S12, the CPU generates a plurality of labels, for example, labels having the range of the measurement value corresponding to the set natural number N, as shown in FIG.

[0045] In step S13, the CPU extracts the first label. At this time, for example, the label whose CACS range start value is 1, whose CI difference value is equal to or less than the threshold (10), and whose CI difference value is the smallest is selected and determined as the first label. In the example of FIG. 3, the CI difference value (2.201316461) of the label of "cacs1_125", which is the sixth record from the top, is equal to or less than the threshold and is the smallest, so the label of "cacs1_125" is extracted as the first label.

[0046] In step S14, the CPU determines the labels up to the cutoff value. At this time, the start value of the CACS range is 125, the CI difference value is equal to or smaller than the threshold, and the smallest label is selected and determined as the second label.

[0047] 5 is a diagram simply showing labels whose starting value in the CACS range is 125. In this example, the CI difference value (9.9) for labels whose CACS range is 125 to 425 is below the threshold and is the smallest, so the label "cacs125_425" is extracted as the second label.

[0048] Similarly, the label whose start value of the CACS range is 425, whose CI difference value is equal to or less than the threshold, and whose CI difference value is the smallest is selected and extracted as the third label. In this manner, the fourth, fifth, ... labels are extracted until the end value of the CACS range exceeds the cutoff value. For example, if the CACS range of the fourth label extracted by the process of step S14 is 750 to 1025, the end value (1025) exceeds the cutoff value (1000). In this case, the process of step S14 ends when the fourth label is extracted.

[0049] If there is no label with a CI difference value equal to or less than the threshold, the process of step S14 ends. For example, assume that the CACS range of the fourth label extracted by the process of step S14 is 750 to 925. FIG. 6 is a diagram showing a label with a CACS range start value of 925 in a simplified manner. In this example, there is no label with a CI difference value of 10 or less among the labels with a start value of 925 and an end value of 1000 or less. In this case, the process of step S14 ends when the fourth label is extracted.

[0050] In this way, the label is extracted by referring to a threshold so that the end value is equal to or less than the cutoff value of the measurement.

[0051] After the process of step S14, in step S15, the CPU determines the last label.

[0052] For example, if the end value of the label extracted at the end of step S14 is equal to or greater than the cutoff value, the label that is replaced with a label whose end value of the label extracted at the end of step S14 is set to MAX is determined as the last label. For example, if the CACS range of the fourth label extracted by the process of step S14 is 750 to 1025, the fourth label is replaced with a label whose CACS range is 750 to MAX, and the replaced label becomes the last label.

[0053] Also, if the end value of the label extracted at the end of step S14 is less than the cutoff value, the label whose start value is the end value of the label extracted at the end of step S14 and whose end value is MAX is determined as the last label. For example, suppose that the CACS range of the fourth label determined by the process of step S14 is 750 to 925. In this case, the label whose CACS range is 925 to MAX is determined as the last label (fifth label).

[0054] After the process of step S15, in step S16, the CPU determines whether or not the condition is satisfied, that is, whether or not the total number of labels determined by the processes of steps S13, S14, and S15 is any of 3 to 10. If it is determined in step S16 that the condition is not satisfied, the process returns to step S11, and the variables are newly set. That is, the variables, the natural number N, the cutoff value, and the threshold value of the CI difference value, are changed. Alternatively, the number of labels, which is a condition, may be changed. Then, the processes of steps S12 to S16 are executed as described above.

[0055] If it is determined in step S16 that the condition is satisfied, the learning data generation process ends.

[0056] In this way, a condition regarding the number of labels included in the label pattern is further set, and if the number of labels included in the generated label pattern does not satisfy the condition, the variables related to the natural number N, the cutoff value, and / or the threshold value of the CI difference value are reset, and multiple labels having measurement value ranges corresponding to the reset variables are generated.

[0057] (Effects of the First Embodiment) In this way, the measured values ​​of continuous or discrete variables can be converted into ordinal variables to create a label pattern consisting of an appropriate number of labels, thereby improving the efficiency of machine learning. In addition, all of the labels included in the created label pattern, except for the last label, always have a CI difference value below the threshold, improving the accuracy of the labels.

[0058] Furthermore, by executing the training data generation process of Figure 4, the variables and conditions can be optimized. For example, when generating training data for a model that predicts the onset of a similar disease, a label pattern can be generated using the variables and conditions optimized through the process of Figure 4.

[0059] In the above example, the natural number N as the reference of the CACS range, the cutoff value as the observation threshold of the CACS, and the threshold of the CI difference value are reset, but the present invention is not limited to this. For example, the CACS value to be compared in the calculation of the odds ratio may be reset. The calculated odds ratio is not limited to the presence or absence of MACE, and may be, for example, the presence or absence of stenosis, which is considered to be highly related to the onset of MACE. For example, a variable indicating either the presence or absence of MACE onset or the presence or absence of stenosis may be reset to calculate the odds ratio.

[0060] In this way, if the number of labels included in the generated label pattern does not satisfy the condition, the method of calculating the odds ratio may be further changed so that multiple labels are generated.

[0061] In the above description, the CI difference value is the difference value between the upper and lower limits of the 95% confidence interval, but it is not necessary to use the 95% confidence interval, although it is common in statistics to use the 95% confidence interval.

[0062] In addition, although the label pattern is described here as an example of learning data used in machine learning of a model predicting the onset of MACE, which is a disease, based on CACS, which is a measurement value, the present invention is not necessarily limited to this. For example, the present embodiment can also be applied when generating a label pattern as learning data used in machine learning of a model predicting the onset of other diseases, based on other measurement values.

[0063] <Second embodiment> Next, a second embodiment of the present invention will be described. In this embodiment, a disease is MACE, and a learning data generation method for machine learning a model for predicting the possibility that a patient will develop MACE based on the patient's CACS, for example, will be described.

[0064] In this embodiment, for example, it is assumed that the natural number N that is the reference for the range of the CACS, the cutoff value that is the observation threshold of the CACS, and the threshold value of the CI difference value have already been calculated as variables optimized by the above-mentioned first embodiment. In the following, an example in which the natural number N is 25, the cutoff value is 900, and the threshold value of the CI difference value is 10 will be described.

[0065] (Learning label pattern creation process) Next, a learning label pattern creation process according to this embodiment will be described. Fig. 7 is a flowchart illustrating an example of the learning label pattern creation process. This process is executed by, for example, a general-purpose computer or the like.

[0066] Prior to the execution of this process, for example, tests are performed on several hundred subjects to measure CACS along with information such as height, weight, age, etc., and the presence or absence of MACE is diagnosed. It is assumed that clinical data consisting of information such as the height, weight, age, CACS, etc., of each subject and information indicating the presence or absence of MACE is obtained.

[0067] In step S20, the computer CPU executes a 25-fold label creation process, which creates a plurality of labels based on, for example, clinical data, such as the data shown in FIG.

[0068] In step S21, the CPU executes a first pattern creation process, which extracts labels from the multiple labels created in the process of step S20 such that the CI difference values ​​are equal to or less than a threshold, and creates a first label pattern (first pattern) consisting of labels that consecutively arrange the ranges of measurement values ​​in ascending order.

[0069] In step S22, the CPU executes a MAX label creation process. This creates multiple labels whose starting value in the measurement range is 1 or a multiple of 25, and whose ending value is MAX (the maximum measurement value). That is, labels whose starting values ​​in the CACS range are 1, 25, 50, 75, 100, ..., and whose ending value is MAX are created.

[0070] In step S23, the CPU executes a final label determination process, whereby the last label in the first pattern (the final label) is replaced as necessary.

[0071] In step S24, the CPU determines whether or not the final label has been replaced in the process of step S23. If it is determined in the process of step S23 that the final label has been replaced, the process proceeds to step S25.

[0072] In step S25, the CPU executes a second pattern creation process, which uses the start value of the final label replaced in the process of step S24 as a reference and extracts labels with small CI difference values ​​from the multiple labels created in the process of step S20 to create a second label pattern (second pattern) consisting of labels arranged consecutively in descending order of the range of measurement values.

[0073] In step S26, the CPU executes a label pattern selection process, whereby either the first pattern or the second pattern is selected based on the sum of the CI difference values ​​of the labels of each label pattern.

[0074] If it is determined in step S24 that the final label has not been replaced in the process of step S23, or after the process of step S26, the learning label pattern creation process ends.

[0075] In this manner, the learning label pattern creation process is performed.

[0076] (25x label creation process) Next, a detailed description will be given of the 25 multiple label creation process in step S20 in Fig. 7. Fig. 8 is a flowchart illustrating an example of the 25 multiple label creation process.

[0077] In step S41, the CPU sets a variable m to 1, and sets a variable n to 1.

[0078] In step S42, the CPU sets the start value of the range of the CACS to the value of the variable m.

[0079] In step S43, the CPU sets the end value of the range of the CACS to 25 times the value of the variable n (=25×n).

[0080] In step S44, the CPU extracts information on subjects whose CACS was measured within the range defined by the start value set in the process of step S42 and the end value set in the process of step S43 from the clinical data. As described above, prior to the execution of the learning label pattern creation process, clinical diagnoses were performed on approximately one thousand and several hundred subjects, and clinical data was obtained. Therefore, by the process of step S44, it is possible to calculate from the clinical data how many subjects had a CACS measured within the range, and how many of those subjects developed MACE.

[0081] In step S45, the CPU calculates the odds ratio of the onset of a disease (MACE in this example), thereby calculating a numerical value related to the onset rate of MACE in subjects whose CACS is measured within the range.

[0082] In step S46, the upper and lower limits of the 95% CI interval of the odds ratio and the CI difference value are calculated, thereby calculating the CI difference value corresponding to the CACS within that range.

[0083] In step S47, the CPU determines whether or not the end value exceeds MAX. If it is determined in step S47 that the end value does not exceed MAX, the process proceeds to step S48.

[0084] In step S48, the CPU stores a label in a predetermined storage area in the computer, etc. At this time, a label is stored in which the range of the CACS defined by the start value set in the process of step S42 and the end value set in the process of step S43, the odds ratio calculated in step S45, and the upper and lower limits of the 95% CI interval calculated in step S46 and the CI difference value are associated with each other.

[0085] In step S49, the CPU increments the value of the variable n by one.

[0086] After the process of step S49, the process returns to step S43, and a new end value is set. In this manner, the processes of steps S43 to S49 are repeatedly executed. Such repetition is performed until it is determined in step S47 that the end value has exceeded MAX.

[0087] If it is determined in step S47 that the end value exceeds MAX, the process proceeds to step S50.

[0088] In step S50, the CPU replaces the end value with MAX and stores the label. At this time, the label is stored in which the CACS range defined by the start value set in the process of step S42 and the end value MAX corresponds to the odds ratio calculated in step S45, the upper and lower limits of the 95% CI interval calculated in step S46, and the CI difference value.

[0089] In step S51, the CPU sets the starting value to 25 times the value of m (=25×m).

[0090] In step S52, the CPU determines whether the start value is equal to or greater than MAX.

[0091] In step S53, the CPU sets the value of the variable n to the value obtained by adding 1 to the value of the variable m (=m+1).

[0092] In step S54, the CPU increments the value of the variable m by one.

[0093] After the process of step S54, the process returns to step S43, and a new end value is set. Then, the processes of steps S44 to S49 are executed. This repetition is performed until it is determined in step S47 that the end value has exceeded MAX, and if it is determined in step S47 that the end value has exceeded MAX, the processes of steps S50 to S54 are executed. In this way, each process is executed until it is determined in step S52 that the start value is equal to or greater than MAX.

[0094] If it is determined in step S52 that the start value is equal to or greater than MAX, the 25 multiple label creation process ends.

[0095] For example, if MAX is 1120, when the 25 multiple label creation process is executed, the following labels will be stored for CACS ranges: 1 to 25, 1 to 50, 1 to 75, ..., 1 to 1120, 25 to 50, 25 to 75, 25 to 100, ...25 to 1120, 50 to 75, 50 to 100, 50 to 125, ..., 50 to 1120, ..., 1100 to 1120.

[0096] In this way, the start value is set to 1 or a multiple of N (N is a natural number), the end value is set to a multiple of N, and multiple labels are created that combine multiple end values ​​with each of the multiple start values.

[0097] In this manner, the 25-multiple label creation process is carried out.

[0098] (First pattern creation process) Next, a description will be given of details of the first pattern creation process in step S21 in Fig. 7. Fig. 9 is a flowchart illustrating an example of the first pattern creation process.

[0099] In step S71, the CPU sets a variable i to 1, and sets a variable j to 1.

[0100] In step S72, the CPU sets the value of the variable i to a starting value.

[0101] In step S73, the CPU acquires, from among the labels created in the process of step S20, the label with the j-th smallest CI difference value among the labels having the same start value. For example, if the value of variable j is 1 and the start value is set to 1 in step S72, the label with the smallest CI difference value is acquired from the labels with a start value of 1. That is, the label with the smallest CI difference value is acquired from the labels with CACS ranges of 1 to 25, 1 to 50, 1 to 75, ..., 1 to 1120.

[0102] In step S74, the CPU determines whether the CI difference value of the label acquired in step S73 is equal to or greater than 10 (threshold value). If it is determined in step S74 that the CI difference value is not equal to or greater than 10, the process proceeds to step S75.

[0103] In step S75, the CPU determines whether or not the end value is equal to or less than 900 (cutoff value). If it is determined in step S75 that the end value is equal to or less than 900, the process proceeds to step S76.

[0104] In step S76, the label obtained in step S73 is stored as the i-th label.

[0105] In step S77, the CPU sets the start value to the end value of the i-th label. For example, if the range of CACS of the label stored in step S76 is 1 to 125, the start value is set to 125.

[0106] In step S78, the CPU increments the value of the variable i by one.

[0107] After the process of step S78, the process returns to step S73, and a new label is obtained. For example, the label with the smallest CI difference value is obtained from the labels whose starting value is 125. That is, the label with the smallest CI difference value is obtained from the labels whose CACS ranges are 125 to 150, 125 to 157, 125 to 200, ..., 125 to 1120.

[0108] In this manner, the processes from step S73 to step S78 are repeatedly executed until it is determined in step S74 that the CI difference value is 10 or more, or until it is determined in step S75 that the end value is not 900 or less.

[0109] If it is determined in step S75 that the end value is not equal to or less than 900, the process proceeds to step S79. In step S79, the CPU increments the value of the variable j by one.

[0110] If it is determined in step S74 that the CI difference value is 10 or more, the process of step S80 is executed. In step S80, the CPU stores the i-th label with the end value as MAX. For example, if the value of variable i is 4 and the start value is 750, a label with a CACS range of 750 to 1120 is stored as the fourth label.

[0111] After the process of step S80, the first pattern creation process ends.

[0112] In this way, from among multiple labels created by the 25-multiple label creation process that have the same start value, one label whose CI difference value is below the threshold is selected, and the process of using the end value of the selected label as the start value of the next label is repeated to generate a first label pattern consisting of labels arranged consecutively in ascending order according to the range of measurement values.

[0113] By executing the first pattern creation process, for example, a label pattern is obtained that includes labels with CACS ranges of 1 to 125, 125 to 425, 425 to 750, and 750 to 1120. In this case, the label pattern includes four labels, and the CI difference value of each label, except for the fourth label (the final label), is always equal to or less than the threshold.

[0114] (MAX label creation process) Next, a detailed description will be given of the MAX label creation process in step S22 in Fig. 7. Fig. 10 is a flowchart illustrating an example of the MAX label creation process.

[0115] In step S101, the CPU sets 1 to a variable m.

[0116] In step S102, the CPU sets a starting value to one.

[0117] In step S103, the CPU sets the end value to MAX.

[0118] In step S104, the CPU extracts information on subjects whose CACS was measured within the range defined by the start value 1 and the end value MAX from the clinical data. As described above, prior to the execution of the learning label pattern creation process, clinical diagnoses were performed on approximately one thousand and several hundred subjects, and clinical data was obtained. Therefore, by the process of step S104, it is possible to calculate from the clinical data how many subjects had a CACS measured within the range, and how many of those subjects developed MACE.

[0119] In step S105, the CPU calculates the odds ratio of the onset of a disease (MACE in this example), thereby calculating a numerical value of the onset rate of MACE in subjects whose CACS is measured within the range.

[0120] In step S106, the upper and lower limits of the 95% CI interval of the odds ratio and the CI difference value are calculated, thereby calculating the CI difference value corresponding to the CACS within that range.

[0121] In step S107, the CPU stores a label in a predetermined storage area in the computer, etc. At this time, a label is stored in which the range of the CACS defined by the start value set in the process of step S102 and the end value set in the process of step S103, the odds ratio calculated in step S105, and the upper and lower limits of the 95% CI interval calculated in step S106 and the CI difference value are associated with each other.

[0122] In step S108, the CPU determines whether or not the starting value exceeds 875. If it is determined in step S108 that the starting value does not exceed 875, the process proceeds to step S109.

[0123] In step S109, the CPU sets the starting value to 25 times the value of the variable m (=m×25).

[0124] In step S110, the CPU increments the value of m by one.

[0125] After the process of step S110, the process returns to step S103, and the end value is set to MAX. After that, the processes of steps S104 to S106 are performed, and a new label is stored in step S107. At this time, a label is stored in which the range of the CACS defined by the start value set in the process of step S109 and the end value set in the process of step S103, the odds ratio calculated in step S105, and the upper and lower limits of the 95% CI interval calculated in step S106 are associated with the CI difference value.

[0126] Then, the processes of steps S108 to S110 and steps S103 to S107 are executed. Such repetition is performed until it is determined in step S108 that the starting value exceeds 875.

[0127] If it is determined in step S108 that the starting value exceeds 875, the MAX label creation process ends.

[0128] By executing the MAX label creation process, labels with CACS ranges of 1 to MAX (1120), 25 to MAX, 50 to MAX, . . . , and 1100 to 1120 are stored.

[0129] Fig. 11 is a diagram showing an example of a label created by executing the MAX label creation process. "col_name" in the figure indicates the range of CACS. In this example, "cacs1≦MAX", "cacs25≦MAX", "cacs50≦MAX", ... are shown, which indicate that the CACS ranges are 1 to MAX, 25 to MAX, 50 to MAX, ... respectively.

[0130] In the example of FIG. 11, the starting values ​​of the CACS range are either 1 or a multiple of 25, but the ending values ​​are all MAX.

[0131] Note that the label shown in Fig. 11 is also created by executing the 25 multiple label creation process in step S20 in Fig. 7. For this reason, the MAX label creation process in step S22 in Fig. 7 may be a process in which a label with an end value of MAX is selected from the labels obtained as a result of the 25 multiple label creation process.

[0132] (Final label confirmation process) Next, a detailed description will be given of the final label determination process in step S23 in Fig. 7. Fig. 12 is a flowchart illustrating an example of the final label determination process.

[0133] In step S131, the CPU obtains the start value of the final label of the label pattern created as a result of the first pattern creation process in step S21, and sets the obtained start value to a variable m.

[0134] For example, if a label pattern consisting of labels with CACS ranges of 1 to 125, 125 to 425, 425 to 750, and 750 to 1120 is obtained by the first pattern creation process, the label with a CACS range of 750 to 1120 (MAX) becomes the final label. In this case, in step S131, 750 is set to the variable m.

[0135] In step S132, the CPU sets the CI difference value of the final label of the label pattern created as a result of the first pattern creation process in step S21 to a variable p.

[0136] In step S133, the CPU acquires, from among the labels created as a result of the MAX label creation process in step S22, a label whose CACS range start value is the value of variable m plus 25 (=m+25). Note that the label created by the MAX label creation process is also referred to as a MAX label. In the above example, m+25 (=750+25)=775, so a MAX label whose CACS range is 775 to MAX is acquired.

[0137] In step S134, the CPU sets the CI difference value of the MAX label acquired in step S133 to a variable q.

[0138] In step S135, the CPU determines whether or not the value of the variable q is equal to or less than the value of the variable p. If it is determined in step S135 that the value of the variable q is equal to or less than the value of the variable p, the process proceeds to step S136.

[0139] In step S136, the CPU replaces the final label with the MAX label obtained in step S133.

[0140] In step S137, the CPU adds 25 to the current value of variable m to update the value of variable m.

[0141] After the process of step S137, the process returns to step S133, and the processes of steps S133 to S137 are executed. Such repetition is performed until it is determined in step S135 that the value of variable q is not equal to or less than the value of variable p.

[0142] If it is determined in step S135 that the value of the variable q is not equal to or less than the value of the variable p, the final label determination process ends.

[0143] (Another example of the final label confirmation process) The final label determination process may include, for example, the following processes (a) to (c).

[0144] (a) A MAX label having a start value greater than the start value of the final label of the label pattern generated as a result of the first pattern generation process in step S21 is obtained.

[0145] (b) Among the MAX labels obtained in (a), extract MAX labels having a CI difference value equal to or less than the CI difference value of the final label.

[0146] (c) Among the MAX labels extracted in (b), the one with the largest starting value is selected, and the final label is replaced with the selected MAX label.

[0147] In this way, the final label of the label pattern is replaced based on the comparison result between the CI difference value of the final label, which is a label whose end value is the maximum value of the measurement value among the labels included in the first label pattern by the first pattern creation process, and the CI difference value of the label created by the 25 multiple label creation process, whose end value is the maximum value of the measurement value and whose start value is the start value of the final label plus N.

[0148] In the final label determination process, if there is no MAX label having a start value larger than the start value of the final label and a CI difference value equal to or smaller than the CI difference value of the final label, the final label is not replaced. As described above, if the final label is not replaced, the learning label pattern creation process of FIG. 7 is terminated (step S24 of FIG. 7).

[0149] On the other hand, if the final label has been replaced, the second pattern creation process of step S25 in FIG. 7 is executed.

[0150] (Second pattern creation process) Next, a description will be given of details of the second pattern creation process in step S25 in Fig. 7. Fig. 13 is a flowchart illustrating an example of the second pattern creation process.

[0151] In step S151, the CPU sets the number of the final label to a variable k. For example, if the label pattern created as a result of the first pattern creation process in step S21 includes four labels, the value of the variable k becomes 4.

[0152] In step S152, the CPU sets the end value to the start value of the k-th label.

[0153] In step S153, the CPU determines whether the start value is less than 25.

[0154] In step S154, the CPU obtains the label with the smallest CI difference value among the labels having the same end value from among the labels created in the process of step S20. For example, consider a case where k is 4 and the CACS range of the fourth label is 750 to 1120. In this case, among the labels created in the 25 multiple label creation process, the CI difference values ​​of the labels with CACS ranges of 1 to 750, 25 to 750, 50 to 750, ..., 725 to 750 are checked. Then, of these labels, the label with the smallest CI difference value is obtained.

[0155] In step S155, the CPU stores the k-1-th label in a predetermined storage area or the like in the computer.

[0156] In step S156, the CPU decrements the value of the variable k by one.

[0157] In step S157, the CPU determines whether or not the value of the variable k is less than 1. If it is determined that the value of the variable k is not less than 1, the process returns to step S152, and the subsequent steps are repeatedly executed.

[0158] If it is determined in step S153 that the starting value is less than 25, the process proceeds to step S159. If it is determined in step S157 that the value of the variable k is less than 1, the process proceeds to step S158.

[0159] In step S158, the CPU sets a starting value to 1 and stores the (k-1)th label in a predetermined storage area or the like in the computer.

[0160] After the process of step S158, the second pattern creation process ends.

[0161] In this way, the start value of the final label replaced by the final label determination process is set as the end value of the next label, and the process of selecting the label with the smallest CI difference value from among the multiple labels with the same end value created by the 25 multiple label creation process is repeatedly performed, thereby generating a second label pattern in which the measurement value ranges are arranged consecutively in descending order.

[0162] By executing the second pattern creation process, for example, label patterns consisting of labels with CACS ranges of 800 to 1120, 550 to 800, 175 to 550, and 1 to 175 are obtained.

[0163] In this manner, the second pattern creation process is performed.

[0164] (Label pattern selection process) Next, a description will be given of details of the label pattern selection process in step S26 in Fig. 7. Fig. 14 is a flowchart illustrating an example of the label pattern selection process.

[0165] In step S171, the CPU calculates a total value G1 of the CI difference values ​​of the labels included in the first pattern.

[0166] In step S172, the CPU calculates a total value G2 of the CI difference values ​​of the labels included in the second pattern.

[0167] In step S173, the CPU determines whether or not the sum G1 is greater than the sum G2. If it is determined in step S173 that the sum G1 is greater than the sum G2, the process proceeds to step S174.

[0168] In step S174, the CPU selects the first pattern, which causes the label pattern of the first pattern to be output as the processing result of the learning label pattern creation processing.

[0169] On the other hand, if it is determined in step S173 that the sum G1 is not greater than the sum G2, the process proceeds to step S175.

[0170] In step S175, the CPU selects the second pattern, and the second label pattern is output as the processing result of the learning label pattern creation process.

[0171] In this way, either the first label pattern or the second label pattern is selected based on the comparison result between the sum of the CI difference values ​​of the labels included in the first label pattern and the sum of the CI difference values ​​of the labels included in the second label pattern.

[0172] In this manner, the label pattern selection process is performed.

[0173] (Effects of the second embodiment) In this way, the measured values ​​of continuous or discrete variables can be converted into ordinal variables to create a label pattern consisting of an appropriate number of labels, thereby improving the efficiency of machine learning. In addition, the label pattern is created so that the CI difference value of the labels included in the created label pattern is small, thereby improving the accuracy of the labels.

[0174] The inventors compared the prediction accuracy of a model (called the first prediction model) trained by machine learning using learning data in which the CACS values ​​were used as labels without being converted into ordinal variables, with the prediction accuracy of a model (called the second prediction model) trained by machine learning using learning data in which a label pattern consisting of labels in which the CACS values ​​were converted into ordinal variables according to this embodiment. The prediction accuracy is indicated by the AUC (Area Under the Curve), and the prediction accuracy of the first prediction model was 0.767 (0.739 to 0.797), whereas the prediction accuracy of the second prediction model was 0.808 (0.785 to 0.832).

[0175] This revealed that the prediction accuracy of the second prediction model was improved compared to the first prediction model.

[0176] <Third embodiment> Next, a third embodiment of the present invention will be described. For example, the learning label pattern may be generated by executing the following processes (k) to (n).

[0177] (k) Execute the 25 multiple label creation process described above with reference to FIG.

[0178] (l) The number of labels included in the label pattern is set as a condition. For example, the number of labels is set to 3 to 5.

[0179] (m) From the labels created by the 25 multiple label creation process, create all label patterns that are made up of labels that have the CACS range from 1 to MAX arranged consecutively and that meet the conditions. That is, all label patterns made up of three labels with consecutive CACS ranges, all label patterns made up of four labels with consecutive CACS ranges, and all label patterns made up of five labels with consecutive CACS ranges are created.

[0180] (n) Among the label patterns created in (m), label patterns including a label whose starting value exceeds a cutoff value are eliminated.

[0181] (o) The sum of the CI difference values ​​of the labels contained in each label pattern remaining after the processes in (n) is calculated, and the label pattern with the smallest sum is output as the final created label pattern.

[0182] Fig. 15 is a diagram showing an example of a label pattern created by the above-mentioned process (m). For simplicity, the following description will be given on the assumption that three label patterns have been created, but in reality, many label patterns are created. In the example of Fig. 15, label pattern 101, label pattern 102, and label pattern 103 are shown.

[0183] Label pattern 101 contains five labels, label pattern 102 contains three labels, and label pattern 103 contains four labels.

[0184] In addition, descriptions such as "CACS1-150" shown in the figure indicate the range of CACS for each label (1 to 150 in this example), and descriptions such as "95% CI difference value 3" indicate the CI difference value for each label (3 in this example). In all of label patterns 101 to 103, the ranges of measurement values ​​(CACS) for each label included in the label pattern are consecutive in ascending order. That is, the end value of the range of measurement values ​​for the first label is the start value of the range of measurement values ​​for the second label, the end value of the range of measurement values ​​for the second label is the start value of the range of measurement values ​​for the third label, and so on.

[0185] In label pattern 101, the starting value of the fifth label is 950, which exceeds the cutoff value of 900. For this reason, label pattern 101 is excluded in process (n). On the other hand, label pattern 102 and label pattern 103 do not contain any labels whose starting value exceeds the cutoff value, so they are not excluded.

[0186] The sum of the CI difference values ​​of the labels included in label pattern 102 ("95% CI width sum" in the figure) is 30, and the sum of the CI difference values ​​of the labels included in label pattern 103 is 34. Therefore, label pattern 103 is output in the process of (o).

[0187] The label pattern may be created in this manner.

[0188] In the second and third embodiments, the label patterns are described as learning data used in machine learning of a model that predicts the onset of MACE, which is a disease, based on CACS, which is a measurement value, but the present invention is not necessarily limited to this. For example, the present embodiment can also be applied when generating label patterns as learning data used in machine learning of a model that predicts the onset of other diseases, based on other measurement values.

[0189] <Fourth embodiment> Next, a fourth embodiment will be described. In this embodiment, model parameters of a model for predicting disease onset are machine-learned using learning data generated by the learning data generation method described in the first to third embodiments, and disease onset is predicted using the model parameters.

[0190] (Configuration of disease onset prediction system) 16 is a block diagram showing an example of the configuration of a disease onset prediction system according to this embodiment. The disease onset prediction system 210 shown in the figure includes a control unit 220, a storage unit 230, a communication unit 241, an input unit 242, and an output unit 243.

[0191] (Control unit) The control unit 220 includes a measurement value receiving unit 221 , a prediction unit 222 , a result output unit 223 , and a learning unit 229 .

[0192] (Measurement value reception section) The measurement value receiving unit 221 receives a measurement value (e.g., CACS) obtained as a result of an examination on a subject, patient, etc. who is the subject of disease onset prediction. The measurement value is input to the control unit 220 via the input unit 242, for example.

[0193] (Prediction section) The prediction unit 222 executes prediction by a predetermined prediction model using the model parameters stored in the storage unit 230. That is, the prediction model is a prediction model that predicts the onset of the disease using model parameters trained by an arbitrary machine learning algorithm using the learning data generated by the learning data generation method described in the first to third embodiments.

[0194] In this example, the prediction unit 222 predicts the incidence rate of MACE using the measurement values ​​received by the measurement value receiving unit 221 as explanatory variables.

[0195] (Result output section) The result output unit 223 outputs the prediction result by the prediction unit 222. The prediction result is output as an image, sound, or the like from the output unit and presented to the user, for example.

[0196] (Learning Department) The learning unit 229 generates model parameters by performing machine learning on the learning data in the storage unit 230 using an arbitrary machine learning algorithm. Here, the learning data includes the learning data generated by the learning data generation method described in the first to third embodiments. In the case of machine learning using tabular data, machine learning algorithms such as XGBOOST, RandomForest, and SVM are particularly suitable. Other machine learning algorithms may also be adopted.

[0197] (Storage part) The storage unit 230 is configured, for example, by a semiconductor memory device or the like, and stores data. In this example, the storage unit 230 stores learning data and model parameters. Here, the learning data includes learning data generated by the learning data generation method described in the first to third embodiments. The model parameters are weighting coefficients obtained by machine learning in the learning unit 229.

[0198] The learning unit 229 of the control unit 220 may not be provided. That is, the process related to machine learning may be executed by another device, the model parameters may be generated, and the generated model parameters may be stored in the storage unit 230. In this case, it is not necessary for the storage unit 230 to store the learning data.

[0199] (Communications Department) The communication unit 241 is an interface for connecting the disease onset prediction system 210 to a network. The specific configuration of the network does not limit the present exemplary embodiment, but as an example, a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination of these networks can be used.

[0200] (Input section) The input unit 242 accepts various inputs to the disease onset prediction system 210. The specific configuration of the input unit 242 is not limited to this exemplary embodiment, but as an example, the input unit 242 may be configured to include input devices such as a keyboard and a touchpad. The input unit 242 may also be configured to include a data scanner that reads data via electromagnetic waves such as infrared rays and radio waves, and a sensor that senses the environmental state.

[0201] (Output section) The output unit 243 is a functional block that outputs the processing results by the disease onset prediction system 210. Although the specific configuration of the output unit 243 does not limit this exemplary embodiment, as an example, it is configured by a display, a speaker, a printer, etc., and displays various processing results by the disease onset prediction system 210 on a screen or outputs them as sounds or figures.

[0202] (Example of display screen) Fig. 17 is a diagram showing an example of a screen displayed on the display of the output unit. Screen 260 shown in the figure displays "Input CSV file:", and in area 261 below it, displays "ID", "Sex", "Height", "Weight", ..., "Agaston Score", .... Each of these items, "ID", "Sex", "Height", "Weight", ..., "Agaston Score", ..., is input as a measured value. Here, the item "Agaston Score" means CACS. Note that, for convenience, "ID", "Sex", etc. are also considered as measured values.

[0203] Fig. 18 is a diagram showing an example of a screen for inputting measurement values. The input form screen shown in Fig. 18 includes input fields corresponding to the items shown in area 261. "Agaston Score," which stands for CACS, is also entered in input field 272 at this time. The screen shown in Fig. 18 is displayed for each of the subjects and patients who are the targets of disease onset prediction.

[0204] In addition, for items in Fig. 18 where "Select" is displayed in the input field, the corresponding option is to be input from a pull-down menu displayed corresponding to the item. Also, in Fig. 18, each of the numerical values ​​displayed in the input fields represents a numerical range that can be input, and the actual measured values ​​of the subjects and patients who are the targets of disease onset prediction are to be input into these input fields.

[0205] After inputting the measurement values ​​on the screen of Fig. 18, disease onset is predicted by pressing button 262 of Fig. 16. Note that after inputting the measurement values ​​for multiple people, predictions for multiple people may be made at once by pressing button 262. The prediction results are displayed in area 263 ("Results") on screen 260.

[0206] Fig. 19 is a diagram showing an example of a screen 260 on which prediction results are displayed. In the example of Fig. 19, the prediction results of the incidence rate of MACE for each of four subjects / patients with "ID" of "1" to "4" are displayed, and numerical values ​​corresponding to each measurement value of each of the four subjects / patients are displayed in an area 261. Here, for convenience, the numerical values ​​corresponding to each measurement value of each of the four subjects / patients are all the same, but in reality, numerical values ​​corresponding to each measurement value of each person are displayed.

[0207] In the area 263 of Fig. 19, the prediction results are displayed as "high", "medium", and "low", respectively. That is, the subject with ID 1 is predicted to have a high incidence of MACE, the subject with ID 2 is predicted to have a medium incidence of MACE, and the subjects with IDs 3 and 4 are predicted to have a low incidence of MACE.

[0208] In other words, subjects / patients with ID 1 have a high risk of developing MACE, and subjects / patients with ID 3 and ID 4 have a low risk of developing MACE. The prediction results "high", "medium", and "low" may be displayed in different colors.

[0209] The items of the measurement values ​​to be input are not limited to the examples described above with reference to Figures 17 to 19, but may include at least an item that is closely related to the onset of the disease to be predicted. For example, when predicting the incidence rate of MACE, it may include at least CACS.

[0210] In this way, the disease onset prediction system 210 receives input of measurement values ​​including the patient's CACS, and predicts the onset of MACE for the patient using a prediction model based on the input measurement values.

[0211] According to the fourth embodiment, it is possible to easily predict the incidence rate of a given disease by simply inputting the measured values ​​of a subject or patient.

[0212] <Example of software implementation> The above-mentioned learning data generation method can be realized by a program for causing a computer to function. In this case, the above-mentioned program is executed by a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory). An example of such a computer is shown in FIG. 20.

[0213] The computer 500 includes at least one processor 501 and at least one memory 502. The memory 502 stores a program 520 for causing the computer 500 to operate as the reservation communication control device 80. In the computer 500, the processor 501 reads and executes the program 520 from the memory 502, thereby executing each process of the learning data generation method.

[0214] The processor 501 may be, for example, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a microcontroller, or a combination of these.

[0215] As the memory 502, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these can be used.

[0216] The computer 500 may further include a RAM (Random Access Memory) for expanding the program 520 during execution and for temporarily storing various data. The computer 500 may further include a communication interface for transmitting and receiving data to and from other devices. The computer 500 may further include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0217] Furthermore, the program 520 can be recorded on a non-transitory tangible recording medium 530 that is readable by the computer 500. For example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit can be used as the recording medium 530. The computer 500 can obtain the program 520 via the recording medium 530.

[0218] The program 520 can be transmitted via a transmission medium. For example, a communication network or a broadcast wave can be used as such a transmission medium. The computer 500 can also obtain the program 520 via such a transmission medium.

[0219] According to each aspect of the present invention described above, by achieving the above-mentioned effects, it is possible to contribute to the achievement of Goal 9 of the Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote inclusive and sustainable industrialization and innovation."

[0220] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

[0221] 〔summary〕 A training data generation method according to a first aspect of the present invention is a method for generating training data used in machine learning of a predictive model for predicting the onset of a disease, and includes a label generation step of generating a plurality of labels that correspond a range of measurement values, defined by a start value and an end value, to a numerical value related to the incidence rate of the disease, from clinical data in which measurement values ​​obtained as continuous variables or discrete variables are associated with the presence or absence of the onset of a specified disease, and a label pattern generation step of generating a label pattern consisting of labels in which the range of measurement values ​​is arranged consecutively, by extracting the labels generated by the processing of the label generation step with reference to the numerical value related to the incidence rate of the disease.

[0222] A learning data generation method according to a second aspect of the present invention is characterized in that, in the above-mentioned first aspect, the numerical value relating to the incidence rate of the disease is a CI difference value, which is the difference between the upper and lower limits of a confidence interval for the odds ratio of the onset of the disease within the range of the measurement value relative to a predetermined comparison target measurement value.

[0223] A third aspect of the present invention relates to a training data generation method according to the second aspect, wherein in the label generation step, a plurality of labels are generated, each including the CI difference value for a 95% confidence interval of the odds ratio.

[0224] A training data generation method according to a fourth aspect of the present invention is the same as that of the third aspect, except that a first variable related to the range of the measurement value, a second variable related to a cutoff value of the measurement value, and a threshold value of the CI difference value are set, and in the label generation step, a plurality of labels having the range of the measurement value corresponding to the set first variable are generated, and in the label pattern generation step, the labels generated by the label generation step are extracted based on a comparison result between the CI difference value of the label and the threshold value so that the starting value is equal to or less than the cutoff value of the measurement value specified by the second variable.

[0225] A learning data generation method according to a fifth aspect of the present invention is characterized in that, in the above-mentioned fourth aspect, the first variable is a natural number N, and in the label generation step, the start value is set to 1 or a multiple of N (N is a natural number), the end value is set to a multiple of N, and multiple labels are created by combining multiple end values ​​for each of the multiple start values.

[0226] In the training data generation method according to a sixth aspect of the present invention, in the above-mentioned fifth aspect, the label pattern generation step includes a first pattern generation step of generating a first label pattern consisting of labels arranged consecutively in ascending order over the range of the measurement values ​​by repeating a process of selecting one label, the CI difference value of which is equal to or less than a threshold value, from among a plurality of labels having the same starting value, which are labels created by the processing of the label generation step, and setting the end value of the selected label as the start value of the next label.

[0227] A training data generation method according to a seventh aspect of the present invention is, in the above sixth aspect, characterized in that the label pattern generation step includes a final label replacement step of replacing the final label of the label pattern based on a comparison result between a CI difference value of a final label, which is a label having an end value of the maximum value of the measurement value, among the labels included in the first label pattern created by the first pattern generation step, and a CI difference value of a label, which is a label having an end value that is the maximum value of the measurement value and a start value obtained by adding N to the start value of the final label, among the labels created by the label generation step; and, if the final label of the label pattern is replaced by the final label replacement step, The method further includes a second pattern generation step of generating a second label pattern in which the ranges of the measurement values ​​are arranged consecutively in descending order by repeatedly executing a process of selecting a label with the smallest CI difference value from among the plurality of labels having the same end value created by the process of the label generation step, using the start value of the final label replaced by the final label replacement step as the end value of the next label, and a pattern selection step of selecting either the first label pattern or the second label pattern based on a result of comparing a sum of the CI difference values ​​of the labels included in the first label pattern and a sum of the CI difference values ​​of the labels included in the second label pattern.

[0228] A training data generation method according to an eighth aspect of the present invention is the same as any one of the fourth to sixth aspects, except that a condition related to the number of labels included in the label pattern is further set, and if the number of labels included in the label pattern generated as a result of the processing of the label pattern generation step does not satisfy the condition, the first variable, the second variable, and / or the threshold value of the CI difference value are reset, and / or a method for calculating the odds ratio is changed, and a plurality of the labels are generated.

[0229] A learning data generation method according to a ninth aspect of the present invention, in any one of the above-mentioned first to eighth aspects, is characterized in that the measurement value is CACS (Coronary Artery Calcification Score), and the disease is MACE (Major Adverse Cardiovascular Events).

[0230] A program according to a tenth aspect of the present invention causes a computer to execute a learning data generation process for generating learning data used in machine learning of a predictive model for predicting the onset of a disease, the learning data generation process including a label generation step of generating a plurality of labels that correspond a range of measurement values, defined by a start value and an end value, to a numerical value related to the incidence rate of the disease from clinical data in which measurement values ​​obtained as continuous variables or discrete variables are associated with the presence or absence of the onset of a specified disease, and a label pattern generation step of generating a label pattern consisting of labels in which the range of measurement values ​​is arranged consecutively by extracting the labels generated by the processing of the label generation step with reference to the numerical value related to the incidence rate of the disease.

[0231] The predictive model of aspect 11 of the present invention predicts the onset of the disease using model parameters trained by an arbitrary machine learning algorithm using training data generated by the training data generation method of any of aspects 1 to 9 above.

[0232] A prediction system according to aspect 12 of the present invention accepts input of measurements including a patient's CACS, and predicts the onset of MACE for the patient based on the input measurements using the prediction model described in aspect 12 above. [Explanation of symbols]

[0233] 210 Disease onset prediction system 220 Control section 221 Measurement Receiver 222 Prediction Department 223 Result output section 229 Learning Department 230 Storage section

Claims

1. A method for generating learning data used in machine learning of a prediction model for predicting the onset of a disease, comprising: a label generating step of generating a plurality of labels in which a range of the measurement values ​​defined by a start value and an end value is associated with a numerical value relating to the incidence rate of the disease, based on clinical data in which the measurement values ​​obtained as continuous variables or discrete variables are associated with the presence or absence of the onset of a predetermined disease; A label pattern generation step of generating a label pattern consisting of labels in which the ranges of the measurement values ​​are consecutively arranged by extracting the labels generated by the processing of the label generation step with reference to a numerical value related to the incidence rate of the disease. Training data generation method.

2. The numerical value relating to the incidence rate of the disease is a CI difference value as the difference between the upper and lower limits of a confidence interval for the odds ratio of the onset of the disease in the range of the measurement value relative to a predetermined comparison target measurement value. The training data generating method according to claim 1 .

3. In the label generation step, A plurality of labels are generated that include the CI difference values ​​for the 95% confidence intervals of the odds ratios. The training data generating method according to claim 2 .

4. A first variable relating to the range of the measurement value, a second variable relating to a cutoff value of the measurement value, and a threshold value of the CI difference value are set; In the label generation step, A plurality of labels having ranges of the measurement values ​​corresponding to the set first variable are generated; In the label pattern generation step, A label generated by the process of the label generation step is extracted based on a comparison result between the CI difference value of the label and the threshold value so that the starting value is equal to or less than a cutoff value of the measurement value specified by the second variable. The training data generating method according to claim 3 .

5. the first variable is a natural number N, In the label generation step, The start value is set to 1 or a multiple of N (N is a natural number), the end value is set to a multiple of N, and a plurality of labels are created by combining a plurality of end values ​​with each of the plurality of start values. The training data generating method according to claim 4 .

6. The label pattern generating step includes: A first pattern generation step of selecting one label whose CI difference value is equal to or less than a threshold value from among a plurality of labels having the same start value, which are generated by the processing of the label generation step, and repeating a process of setting the end value of the selected label as the start value of the next label, thereby generating a first label pattern made up of labels in which the range of the measurement values ​​is consecutively arranged in ascending order. The training data generating method according to claim 5 .

7. The label pattern generating step includes: a final label replacement step of replacing a final label of the label pattern based on a comparison result between a CI difference value of a final label, which is a label having an end value of the maximum value of the measurement value, among the labels included in the first label pattern created by the first pattern generation step, and a CI difference value of a label, which is a label having an end value of the maximum value of the measurement value and a start value obtained by adding N to the start value of the final label, among the labels created by the label generation step; When the final label of the label pattern is replaced by the final label replacement step, a second pattern generation step of generating a second label pattern in which the ranges of the measurement values ​​are arranged consecutively in descending order by repeatedly executing a process of selecting a label having the smallest CI difference value from among the plurality of labels having the same end value generated by the process of the label generation step, using the start value of the final label replaced by the final label replacement step as an end value of the next label; and a pattern selection step of selecting either the first label pattern or the second label pattern based on a comparison result between a sum of CI difference values ​​of labels included in the first label pattern and a sum of CI difference values ​​of labels included in the second label pattern. The training data generating method according to claim 6 .

8. A condition regarding the number of labels included in the label pattern is further set, When the number of labels included in the label pattern generated as a result of the processing of the label pattern generation step does not satisfy the condition, the first variable, the second variable, and / or the threshold value of the CI difference value are reset, and / or a calculation method of the odds ratio is changed; A plurality of such labels are generated. The training data generating method according to claim 4 .

9. The measurement is the Coronary Artery Calcification Score (CACS), The disease is Major Adverse Cardiovascular Events (MACE). The training data generating method according to claim 1 .

10. On the computer, A process for generating learning data used in machine learning of a prediction model for predicting the onset of a disease, comprising: a label generating step of generating a plurality of labels in which a range of the measurement values ​​defined by a start value and an end value is associated with a numerical value relating to the incidence rate of the disease, based on clinical data in which the measurement values ​​obtained as continuous variables or discrete variables are associated with the presence or absence of the onset of a predetermined disease; Executing a learning data generation process including a label pattern generation step of generating a label pattern consisting of labels in which the ranges of the measurement values ​​are consecutively arranged by extracting the labels generated by the processing of the label generation step with reference to the numerical values ​​related to the incidence rate of the disease. program.

11. A prediction model that predicts onset of the disease using model parameters trained by an arbitrary machine learning algorithm using training data generated by the training data generation method according to claim 1.

12. Accepting input of measurements including the patient's CACS; Based on the input measurements, the predictive model of claim 11 predicts the onset of MACE for the patient. Prediction system.

Images