Model training device and model training method

Abstract

To improve accuracy of a model.SOLUTION: A model training device according to the present embodiment comprises a training unit, an evaluation unit, a grouping unit, and a model training information specification unit. The training unit learns a dataset including medial images as learning data to obtain an initial learning model. The evaluation unit evaluates the initial learning model by using global metrics to acquire abnormal datasets to be outliers from among a plurality of datasets used for the evaluation. The grouping unit groups the plurality of abnormal datasets acquired by the evaluation unit by using local metrics to obtain a plurality of abnormal dataset groups. The model training information specification unit specifies model training information for each of the abnormal dataset groups obtained by the grouping unit.SELECTED DRAWING: Figure 1B

Description

[Technical Field]

[0001] The embodiments disclosed in this specification and the drawings relate to a model training apparatus and a model training method. [Background technology]

[0002] With the development of medical image processing devices such as X-ray devices, CT (Computed Tomography) devices, and ultrasound diagnostic devices, automatic segmentation and interpretation technologies for medical images have also been developed. In particular, in recent years, by combining this with deep learning (DL) technology, it has become possible to train models for image segmentation using clinical datasets, and then apply the trained models to automatically perform image processing such as medical image segmentation. For example, each medical image of an actual example can be considered as a dataset.

[0003] Deep learning involves investigating model frameworks, establishing sufficient training datasets, and performing large-scale computations. In general, it is believed that the more training data there is, the better performance the model will have, and the more heterogeneous input data the model can be trained on.

[0004] However, for example, in the field of medical image segmentation, when training a model, it is necessary to use examples of medical images that have been clinically labeled, such as segmented. However, since it is not easy to obtain clinical datasets with high-quality labeling, it is very important to accurately estimate the number of training data sets required to achieve the target accuracy in training.

[0005] Currently, there are several methods for estimating the amount of training data, including a method in which experts in the relevant field determine the number of samples based on their experience, a method in which the total number of samples is estimated based on statistical methods, and a method in which the amount of data required is predicted based on the learning curve of the model.

[0006] These methods, which are determined artificially by experts, often lead to subjective results. Furthermore, statistical methods and methods that predict using model learning curves require a large number of clinical medical image examples for statistical or testing purposes. These examples often differ significantly between medical images depending on clinical conditions. For example, in medical images from a group of healthy individuals, organ segmentation results are often relatively ideal, whereas in medical images from a group of diseased patients, organ segmentation results are often less ideal.

[0007] Therefore, even if the amount of training data is estimated, it is difficult to obtain a highly accurate model using the above method. In addition, it is necessary to solve the problem of what kind of training data to collect.

[0008] Further research is needed to determine what types of datasets need to be collected to improve model performance, how many additional datasets need to be collected, and how much progress can be made. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] Special Publication No. 2020-512143 [Patent Document 2] Japanese Patent Application Publication No. 2020-064609 Summary of the Invention [Problem to be solved by the invention]

[0010] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to improve the accuracy of the model. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0011] The model training device according to this embodiment includes a training unit, an evaluation unit, a grouping unit, and a model training information identification unit. The training unit obtains an initial training model by training a dataset including medical images as training data. The evaluation unit evaluates the initial training model using global metrics to acquire abnormal datasets that are outliers from among the multiple datasets used for the evaluation. The grouping unit obtains multiple abnormal dataset groups by grouping the multiple abnormal datasets acquired by the evaluation unit using local metrics. The model training information identification unit identifies model training information for each of the abnormal dataset groups obtained by the grouping unit. [Brief explanation of the drawings]

[0012] [Figure 1A] FIG. 1A is a block diagram showing an example of the configuration of a model training device according to the first embodiment. [Figure 1B] FIG. 1B is a block diagram showing an example of the functional configuration of a processing circuit of the model training device according to the first embodiment. [Figure 2] FIG. 2 is a schematic diagram showing a learning curve for evaluating a model using global metrics as a comparative example. [Figure 3] FIG. 3 is a schematic diagram showing an example of the distribution of abnormal data sets. [Figure 4A] FIG. 4A is a diagram showing an example of sub-region divisions in the first embodiment. [Figure 4B] FIG. 4B is a diagram showing an example of sub-region divisions in the first embodiment. [Figure 4C] FIG. 4C is a diagram showing an example of sub-region divisions in the first embodiment. [Figure 4D] FIG. 4D is a diagram showing an example of sub-region divisions in the first embodiment. [Figure 5A] FIG. 5A is a schematic diagram showing evaluation of the division results of the sub-regions in the first embodiment. [Figure 5B]FIG. 5B is a schematic diagram showing an evaluation of the division result of the sub-regions in the first embodiment. [Figure 5C] FIG. 5C is a schematic diagram showing evaluation of the division results of the sub-regions in the first embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of abnormal data set grouping in the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of the characteristics of the abnormal data set group in the first embodiment. [Figure 8A] FIG. 8A is a schematic diagram showing a prediction of a learning curve for each abnormal data set group in the first embodiment. [Figure 8B] FIG. 8B is a schematic diagram showing a prediction of a learning curve for each abnormal data set group in the first embodiment. [Figure 9] FIG. 9 is a flowchart for explaining the processing of the model training device according to the first embodiment. [Figure 10] FIG. 10 is a block diagram showing an example of the functional configuration of a processing circuit of a model training device according to the second embodiment. [Figure 11A] FIG. 11A is a diagram showing an example of a lesion area partition in the second embodiment. [Figure 11B] FIG. 11B is a diagram showing an example of a lesion area partition in the second embodiment. [Figure 11C] FIG. 11C is a diagram showing an example of a lesion area partition in the second embodiment. [Figure 11D] FIG. 11D is a diagram showing an example of a lesion area partition in the second embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of abnormal data set grouping in the second embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of the characteristics of the abnormal data set group in the second embodiment. [Figure 14] FIG. 14 is a flowchart illustrating the processing of the model training device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, a model training device and a model training method according to the present embodiment will be described with reference to the drawings.

[0014] The model training device according to this embodiment is composed of multiple functional modules and can be installed as software on a device having a CPU (Central Processing Unit) and memory, such as an independent computer, or can be distributed across multiple devices. It is realized by a processor executing each functional module of the model training device stored in memory. It can also be realized in the form of hardware as a circuit capable of executing each function of the model training device. The circuit that realizes the model training device can send, receive, and collect data via a network such as the Internet. The model training device according to this embodiment can also be directly installed in a medical image processing device, such as a CT device or a magnetic resonance imaging device, as part of the medical image processing device.

[0015] In the following description, a data set of medical images consisting of multiple pixels is referred to as a single dataset, and a dataset is a unit of sample used for training data, sometimes referred to as a sample in the following description. The format and configuration of a dataset vary depending on the type of medical image processing device used and the imaging site. For ease of explanation, however, we will use as an example abdominal stereoscopic images (3D abdominal images) collected by a device capable of 3D stereoscopic scanning, such as a CT device or magnetic resonance imaging device. A dataset is constructed for each 3D abdominal image, and the liver is the target of interest for the examination. Note that other targets of interest may also be used, and depending on the target of interest and the equipment used, other types of images, such as two-dimensional images or ultrasound scan images, may also be used.

[0016] (First embodiment) First, the first embodiment will be described with reference to FIGS.

[0017] 1A is a block diagram showing an example of the configuration of a model training device 100 according to the first embodiment. For example, as shown in FIG. 1A, the model training device 100 includes an input interface 110, an output interface 120, a storage circuitry 130, and a processing circuitry 140.

[0018] FIG. 1B is a block diagram showing an example of the functional configuration of the processing circuitry 140 of the model training device 100 according to the first embodiment. As shown in FIG. 1B, the processing circuitry 140 executes a training function 10, an evaluation function 20, a grouping function 30, and a model training information identification function 40. Here, the training function 10, the evaluation function 20, the grouping function 30, and the model training information identification function 40 are examples of a training unit, an evaluation unit, a grouping unit, and a model training information identification unit, respectively. Here, the processing functions executed by the training function 10, the evaluation function 20, the grouping function 30, and the model training information identification function 40, which are components of the processing circuitry 140, are recorded in the storage circuitry 130, for example, in the form of a computer-executable program. The processing circuitry 140 is a processor that reads and executes each program from the storage circuitry 130 to realize the function corresponding to each program. The storage circuitry 130 also stores a learning program, which will be described later.

[0019] The term "processor" used in the above description refers to a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), or a field programmable gate array (FPGA)). If the processor is a CPU, for example, the processor realizes its function by reading and executing a program stored in the memory circuit 130. On the other hand, if the processor is an ASIC, for example, the program is directly embedded in the processor circuit instead of storing the program in the memory circuit 130. Note that each processor in this embodiment is not limited to being configured as a single circuit, but may be configured as a single processor by combining multiple independent circuits to realize its function. Furthermore, multiple components in FIG. 1B may be integrated into a single processor to realize its function.

[0020] First, the processing executed by the training function 10 of the model training device 100 according to the first embodiment will be described.

[0021] For example, the training function 10 trains a model for performing liver segmentation on abdominal images (medical images of the abdomen) by learning from multiple existing data sets as learning data. In liver segmentation, for example, when medical image segmentation is performed on an abdominal image to distinguish the positions of organs, an image region representing the liver, which is the region to be detected, is identified as the organ segmentation result. Furthermore, the obtained model is processed by an evaluation function 20, a grouping function 30, and a model training information identification function 40, which will be described later, to identify model training information for training the model, and the training function 10 trains the model by learning learning data based on the model training information.

[0022] The term "training data" as used herein is broadly defined and includes a training set for training and generating a model and a test set for testing (evaluating) the generated model. The training set and the test set each include multiple datasets. A known deep learning method, such as establishing a ground truth (GT) from multiple datasets in the training set and performing supervised learning, is used to train the model. That is, the training function 10 collects a training set and a test set, each including multiple datasets of medical images of the abdomen, establishes a ground truth (GT) from the multiple set data in the training set, and trains the model. Of the multiple datasets in the training set, a dataset for which a ground truth has been established represents a dataset in which image labeling (in the first embodiment, this refers to correct liver segmentation) has been performed correctly. When using the generated model, an unlabeled medical image is used as the input to the model, and a medical image labeled with the organ segmentation results is output from the model.

[0023] The existing dataset used in the training function 10 is not particularly limited and may be a plurality of example datasets pre-stored in the model training device 100, with the number and nature of the datasets being arbitrary as long as a training set and a test set are formed for model training. The existing dataset used in the training function 10 may also be a single dataset. That is, the training set includes at least one dataset.

[0024] Next, the processing executed by the evaluation function 20 of the model training device 100 according to the first embodiment will be described.

[0025] The evaluation function 20 uses global metrics to evaluate the model generated by the training function 10, thereby acquiring an outlier dataset from the multiple datasets in the test set used for evaluation as an abnormal dataset. Specifically, while the training function 10 gradually inputs the multiple datasets in the training set to generate a model, the evaluation function 20 generates a learning curve for the model by testing the model generated at different stages using the multiple datasets in the test set. The evaluation function 20 then acquires multiple abnormal datasets from the test set by calculating global metrics for each of the multiple datasets in the test set.

[0026] Global metrics are metrics that evaluate the overall output of a model (e.g., labeling of medical images) and are also called global evaluation metrics. In the field of medical image segmentation, metrics that measure the similarity between two sets, i.e., metrics that measure the similarity between the model output (segmentation result) and the correct segmentation result of a test set, from a global perspective, are commonly used. For example, in the field of 3D medical image segmentation, commonly used global metrics include the dice coefficient (dice similarity coefficient) and IOU (Intersection Over Union: a detection evaluation function). Further global metrics include overlap-based metrics and volume-based metrics. For the calculation of global metrics, please refer to the literature: Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool (Taha and Hanbury BMC Medical Imaging (2015) 15:29. DOI 10.1186 / s12880-015-0068-x). For the sake of convenience, it is assumed here that the evaluation function 20 evaluates the model using only the Dice coefficient as the global metric, and obtains multiple abnormal data sets from the test set used for evaluation based on the Dice coefficient.

[0027] As a comparative example, in known techniques, global metrics are often used to monitor the growth of a model during its learning process, for example, to fit the model's learning curve. FIG. 2 is a schematic diagram showing a learning curve for evaluating a model using global metrics as a comparative example. In FIG. 2, the horizontal axis indicates the number of data sets as the number of learning data samples (training set) for training the model, and the vertical axis indicates the dice value calculated by evaluating the model. As an example, the maximum number of data sets in the training set is set to 150, and the maximum number of data sets in the test set is set to 50.

[0028] In Figure 2, the line graph L1 represents the learning curve from the training set, and each node on the graph represents the dice value obtained by a model trained using the number of datasets in the training set corresponding to the node on the horizontal axis. The dotted curve represents a logarithmic curve fit to the line graph L1, which gives the trend of the learning curve. As can be seen from this trend, the more datasets (i.e., the number of samples) in the training set used to train the model, the better the model.

[0029] Also, in Figure 2, line graph L2 is a learning curve generated based on the test set, and the nodes above it represent the dice values ​​obtained by testing the models generated at each node on line graph L1 with a fixed number of datasets in the test set. The dotted curve is a logarithmic curve fit to line graph L2, which allows us to obtain the trend of the learning curve. When using a test set containing multiple datasets, such as a learning curve, the dice value for the entire model can be calculated using the average dice value for each dataset. In general, when testing a model, the closer the dice value on the learning curve is to 1, the more accurate the model's results are considered to be.

[0030] As can be seen from Figure 2, the dice value of the test set is lower than the dice value of the training set. By inputting the test set of training data into the model, a dice value is calculated that compares the similarity between the model output and the correct segmentation result of the test set, thereby enabling testing of the model. The above is a general usage of global metrics for model testing.

[0031] However, global metrics can also be used to evaluate a single dataset. In this embodiment, global metrics are used to evaluate a single dataset among the multiple datasets in the test set, thereby making it possible to confirm the differences and distribution of the individual datasets in the test set. As a result, the evaluation function 20 identifies, among the datasets in the test set, a portion of the datasets in which the difference between the model output result and the correct segmentation result obtained when using global metrics is relatively large as an abnormal dataset. For example, a dice value threshold is set in advance, and the evaluation function 20 inputs a certain dataset among the multiple datasets in the test set into a model, thereby calculating a dice value based on the model output result and the correct segmentation result of the dataset, and designates a dataset whose calculated dice value is below the set dice value threshold as an abnormal dataset.

[0032] FIG. 3 is a schematic diagram showing an example of the distribution of abnormal datasets. For ease of explanation, a test set containing 50 datasets will be described as an example. These 50 datasets are numbered 1 to 50, and as shown in FIG. 3, the horizontal axis represents the dataset number, and the vertical axis represents the dice value of the corresponding dataset. The dice value indicates the similarity between the segmentation result output from a model in which a dataset with a certain number is input, and the correct segmentation result of the dataset. For example, the first point Q on the horizontal axis in FIG. 3 is the dataset numbered 1, and its dice value is 0.8. In this way, all datasets are plotted as points on the graph, resulting in a dotted distribution diagram as shown in FIG. 3, with each point representing one dataset.

[0033] Here, if the threshold for dice values ​​is "0.8", any dice value less than "0.8" is an outlier. Hereinafter, a data set of outliers will be referred to as an abnormal data set. In the example shown in FIG. 3, dice values ​​below the dotted line indicating the threshold "0.8" are outliers, and data sets of outliers are abnormal data sets. For example, in FIG. 3, there are a total of 20 abnormal data sets O1 to O20.

[0034] As described above, the evaluation function 20 calculates the dice value of each data set in the test set and compares the result with a threshold to obtain a plurality of abnormal data sets.

[0035] Here, it is assumed that the evaluation function 20 performs evaluation using all data sets in the test set, but it may also perform evaluation using only a portion of the multiple data sets in the test set.

[0036] Next, the processing executed by the grouping function 30 of the model training device 100 according to the first embodiment will be described.

[0037] The grouping function 30 uses local metrics to group the plurality of abnormal data sets obtained by the evaluation function 20 to obtain a plurality of abnormal data set groups.

[0038] Local metrics are metrics for evaluating local areas of a model's output results (e.g., labeling of medical images) and are also called local evaluation metrics. Local metrics commonly used in the field of medical image segmentation are matching metrics related to local contours. Specifically, local metrics are local contour matching metrics that indicate the difference between the local contour of the output result obtained by inputting a dataset into a model and the local contour of the correct segmentation result of the dataset. Examples of local metrics include spatial distance metrics such as HD (Hausdorff Distance) and MHD (Mahalanobis Distance), which indicate the distance between two local contours. Both of these metrics can locally evaluate the error of a model. The grouping function 30 calculates the local metrics and compares them with a set criterion to obtain local evaluations such as local over-segmentation or local under-segmentation.

[0039] Returning to the description of FIG. 1B, in the first embodiment, the grouping function 30 includes a division function 31, a sub-region partition function 32, a local metric calculation function 33, and an abnormal data set grouping function .

[0040] The division function 31 divides (segments) the medical image corresponding to the abnormal data set acquired by the evaluation function 20 so as to distinguish the positional relationship between the detection target region to be detected and other regions adjacent to the detection target region in the medical image. Specifically, the division function 31 performs low-resolution multi-organ division for each abnormal data set. Here, since a 3D medical image captured with the liver as the detection target is taken as an example, the division function 31 divides the 3D volume data including the liver portion into multiple organs according to the above-mentioned multi-organ division method, and acquires segmented images that distinguish the positions of the multiple organs.

[0041] Next, the subregion partitioning function 32 partitions the boundary region of the target liver into multiple subregions according to multiple adjacent other parts. Specifically, the subregion partitioning function 32 partitions the boundary of the liver adjacent to different organs into multiple subregions.

[0042] The criteria for partitioning are determined according to the type of adjacent organ that may affect the detection target, for example, the liver may be affected by the adjacent organs such as the stomach, pancreas, duodenum, heart, diaphragm, spleen, colon, muscle, esophagus, etc. Therefore, the liver can be partitioned by each influencing organ. It is also possible to select and partition several important adjacent organs from among the influencing organs.

[0043] 4A to 4D are diagrams showing examples of subregion divisions in the first embodiment. In these diagrams, the three-dimensional volume data is divided into multiple slices, and the division of the subregions and calculation of local metrics may be performed on each slice, after which the volume data may be reconstructed from the divided slices.

[0044] The boundary region surrounded by a dotted line in Fig. 4A is the boundary region where the liver and heart are adjacent, and the subregion dividing function 32 divides this boundary region as subregion R1. The boundary region surrounded by a dotted line in Fig. 4B is the boundary region where the liver and stomach are adjacent, and the subregion dividing function 32 divides this boundary region as subregion R2. The boundary region surrounded by a dotted line in Fig. 4C is the boundary region where the liver and kidney are adjacent, and the subregion dividing function 32 divides this boundary region as subregion R3. The boundary region surrounded by a dotted line in Fig. 4D is the boundary region where the liver and pancreas are adjacent, and the subregion dividing function 32 divides this boundary region as subregion R4. In the example shown in Figures 4A to 4D, four subregions R1, R2, R3, and R4 are partitioned using the heart, stomach, kidneys, and pancreas adjacent to the liver. Also, in Figures 4A to 4D, only one slice is shown for each subregion, and the partitioning result is volume data containing slices of the same type.

[0045] Furthermore, the border of the liver may be completely defined by setting the subregion R4 at a border other than the subregions R1, R2, and R3.

[0046] Next, the local metrics calculation function 33 calculates local metrics of the abnormal dataset for each subregion. Here, a case where HD, which indicates the distance between two local contours, is calculated as the local metric, will be described with reference to FIGS. 5A to 5C. FIGS. 5A to 5C are schematic diagrams illustrating evaluation of subregion segmentation results in the first embodiment. FIG. 5A shows the correct segmentation result (true value) of the liver of a certain abnormal dataset O (e.g., one of the abnormal datasets O1 to O20 in FIG. 3), and the hatched area in FIG. 5A is the segmented liver portion. Meanwhile, FIG. 5B shows the liver segmentation result output by inputting image data of the same abnormal dataset O into a model generated by the training function 10, and the dotted hatched area in FIG. 5B is the liver portion in the segmentation result output to the model. FIG. 5C shows the difference between the segmentation of subregion R1 in FIGS. 5A and 5B. That is, in sub-region R1, there is a difference between the true value of the abnormal data set O and the model output result, as shown in the part surrounded by a dotted circle in Fig. 5C, and the local metrics calculation function 33 calculates the sum of distances indicated by HD at the location of this difference as the local metrics of the abnormal data set O. In addition, the local metrics calculation function 33 performs similar calculations for the other sub-regions, and finds the local metrics of each sub-region for each abnormal data set.

[0047] Next, the abnormal dataset grouping function 34 groups multiple abnormal datasets based on the sub-areas partitioned by the sub-area partitioning function 32 and the local metrics calculated by the local metrics calculation function 33. Specifically, the abnormal dataset grouping function 34 groups abnormal datasets that satisfy predetermined conditions into one group.

[0048] For example, in the example of liver segmentation described above, the abnormal dataset grouping function 34 groups abnormal datasets with the same evaluation result in the same subregion into one group. Specifically, the abnormal dataset grouping function 34 determines whether each subregion is over-segmented or under-segmented by comparing the local metrics of the subregion with a threshold. Then, for each subregion, the abnormal dataset grouping function 34 groups abnormal datasets that include over-segmented subregions into the same group, and groups abnormal datasets that include under-segmented subregions into the same group.

[0049] When the multiple abnormal datasets are the abnormal datasets O1 to O20 shown in FIG. 3 , the abnormal dataset grouping function 34 compares the absolute value |D| of the local metric D of each subregion for each abnormal dataset, where D is the local metric calculated by the local metric calculation function 33, with the set over-segmentation threshold T1 and under-segmentation threshold T2. Here, if the comparison result is D > 0 and |D| > T1, the abnormal dataset grouping function 34 determines the evaluation result of the subregion as over-segmentation. If the comparison result is D < 0 and |D| > T2, the abnormal dataset grouping function 34 determines the evaluation result of the subregion as under-segmentation. As described above, by individually evaluating the model division results for each subregion for each abnormal dataset, the multiple abnormal datasets can be divided into multiple groups, and the abnormal datasets for each group include subregions of the same category with the same evaluation. Moreover, the over-segmentation threshold T1 and the under-segmentation threshold T2 may be the same value or may be different values.

[0050] Fig. 6 is a diagram showing an example of abnormal dataset grouping in the first embodiment. When the multiple abnormal datasets are the abnormal datasets O1 to O20 shown in Fig. 3, for example, grouping as shown in Fig. 6 is performed. In Fig. 6, grouping is performed for four sub-regions R1, R2, R3, and R4, and the abnormal datasets O1 to O20 are divided into eight groups.

[0051] For example, if the abnormal dataset grouping function 34 determines that the subregion R1 in the medical images of the abnormal datasets O7, O9, O11, and O16 is all over-segmented, the group including the abnormal datasets O7, O9, O11, and O16 that contain the over-segmented subregion R1 is designated as "Group 1." For example, if the abnormal dataset grouping function 34 determines that the subregion R1 in the medical images of the abnormal datasets O6, O12, and O13 is all under-segmented, the group including the abnormal datasets O6, O12, and O13 that contain the under-segmented subregion R1 is designated as "Group 2."

[0052] For example, if the abnormal dataset grouping function 34 determines that the sub-region R2 in the medical images of the abnormal datasets O1, O3, and O5 is all over-segmented, the group including the abnormal datasets O1, O3, and O5 that contain the over-segmented sub-region R2 is called "Group 3." For example, if the abnormal dataset grouping function 34 determines that the sub-region R2 in the medical images of the abnormal datasets O17 and O14 is all under-segmented, the group including the abnormal datasets O17 and O14 that contain the under-segmented sub-region R2 is called "Group 4."

[0053] For example, if the abnormal dataset grouping function 34 determines that the subregion R3 in the medical images of the abnormal datasets O2 and O4 is both over-segmented, the group including the abnormal datasets O2 and O4 that includes the over-segmented subregion R3 will be called "Group 5." For example, if the abnormal dataset grouping function 34 determines that the subregion R3 in the medical images of the abnormal datasets O8 and O18 is both under-segmented, the group including the abnormal datasets O8 and O18 that includes the under-segmented subregion R3 will be called "Group 6."

[0054] For example, if the abnormal dataset grouping function 34 determines that the subregion R4 in the medical images of the abnormal datasets O15 and O10 is both over-segmented, the group including the abnormal datasets O15 and O10, which includes the under-segmented subregion R4, will be called "Group 7." For example, if the abnormal dataset grouping function 34 determines that the subregion R4 in the medical images of the abnormal datasets O19 and O20 is both under-segmented, the group including the abnormal datasets O19 and O20, which includes the under-segmented subregion R4, will be called "Group 8."

[0055] 6 is merely an example, and the present embodiment is not limited thereto. For example, the same abnormal data set may be divided into two groups simultaneously if it contains two sub-regions with abnormal local metrics. Furthermore, multiple sub-regions may be combined and grouped, and if the local metrics of the abnormal data set satisfy a predetermined condition, they can be grouped together.

[0056] Next, the processing executed by the model training information identifying function 40 of the model training device 100 according to the first embodiment will be described.

[0057] The model training information identification function 40 identifies, for each abnormal dataset group grouped by the grouping function 30, model training information that the training function 10 uses to train a model.

[0058] The grouping function 30 separates the originally scattered datasets into multiple abnormal dataset groups with common characteristics. The model training information identification function 40 collects features for each abnormal dataset group and identifies these features as model training information, thereby outputting the identified model training information from the output interface 120. The features of the abnormal dataset groups identified by the model training information identification function 40 may be grouping rules or common features obtained by analyzing the multiple datasets in the group after grouping. These features of the abnormal dataset groups may be image features, anatomical features, or pathological features. That is, the model training information identification function 40 identifies image features, anatomical features, or pathological features for each abnormal dataset group as model training information.

[0059] FIG. 7 is a diagram showing an example of the features of the abnormal data set groups output by the model training information identification function 40 in the first embodiment. As shown in the table in FIG. 7, the features of the medical images in each of the abnormal data set groups "Group 1" to "Group 8" are analyzed individually. FIG. 7 illustrates examples of imaging features and pathological features. For example, the abnormal data set group "Group 1" collectively has imaging features I1, I2, and I3 and pathological features P1 and P2, while the abnormal data set group "Group 2" collectively has imaging features I4, I5, and I6 and pathological feature P3. Here, a description of the features of the other groups is omitted.

[0060] The types of features are not limited to those listed in Fig. 7, and for example, the radiological features may be acquisition protocols, imaging artifacts (e.g., metal artifacts in CT data, dynamic artifacts in MR), breath-holding conditions, partial volume effects (slice thickness), etc. For example, the pathological features may be liver fat, iron deposits, fibrosis, tumors, etc.

[0061] The model training apparatus 100 can perform targeted learning when generating a model specialized for a specific application using training data including the abnormal dataset group based on the above-mentioned features identified as model training information by the model training information identifying function 40. For example, the training function 10 can train a new model based on the features of the abnormal dataset group by collecting datasets having the same features and combining the newly collected dataset with the abnormal dataset group. In this case, the model can output more accurate results for medical images having these features.

[0062] The model training information identification function 40 can also identify model training information by predicting the number of datasets required to train a model or the accuracy of the model for each abnormal dataset group. For example, for each abnormal dataset group, the model training information identification function 40 tests a model (corresponding to an initial learning model) generated by the training function 10 using a set of datasets included in the abnormal dataset group as learning data (test set), thereby fitting a learning curve to the model. Based on the fitted learning curve, the model training information identification function 40 can identify model training information by predicting the number of datasets (the number of learning data to be collected) required to build a model corresponding to the characteristics of the abnormal dataset group or the accuracy of the model. Figures 8A and 8B are schematic diagrams showing predicted learning curves for each abnormal dataset group in the first embodiment.

[0063] FIG. 8A shows the learning curves when testing (evaluating) models at different stages (different numbers of datasets in the training set) generated by the training function 10 shown in FIG. 2 using "Group 1" in the table of FIG. 7 as the test set. As shown in FIG. 8A, the dotted curve is obtained by fitting the learning curve of "Group 1." According to the trend of the curve obtained by fitting, when the number of datasets (i.e., the number of samples) used for model training reaches 70, the Dice value is approximately 0.75, which is a relatively high accuracy. Furthermore, since the dataset used for testing includes features such as image features I1, I2, and I3 and pathological features P1 and P2, to train a model with relatively high accuracy (Dice value reaching 0.75) for features I1, I2, I3, P1, and P2, at least 70 datasets are required for the training set. The model training information identification function 40 identifies this ideal number of data sets as model training information and outputs the identified model training information from the output interface 120, so that the training function 10 can collect additional learning data (data sets) based on the model training information and train the model by learning the learning data.

[0064] FIG. 8B shows the learning curves obtained when testing (evaluating) models at different stages (different numbers of datasets in the training set) generated by the training function 10 shown in FIG. 2 using "Group 2" in the table of FIG. 7 as the test set. As shown in FIG. 8B, the dotted curve is obtained by fitting the learning curve for "Group 2." The trend of the curve obtained by fitting indicates that when the number of datasets used for model training reaches 35, the Dice value is approximately 0.8, which is a relatively high accuracy. Furthermore, since the dataset used for testing includes features such as image features I4, I5, and I6 and pathological feature P3, to train a model with relatively high accuracy (Dice value reaching 0.8) for features I4, I5, I6, and P3, the training set must contain at least 35 datasets. The model training information identification function 40 identifies this ideal number of data sets as model training information and outputs the identified model training information from the output interface 120, so that the training function 10 can collect additional learning data (data sets) based on the model training information and train the model by learning the learning data.

[0065] The model training device 100 can also group training sets similarly using the model training information output by the model training information identification function 40 as grouping rules, and train a model using the grouped training sets. In this case, the training function 10 collects learning data corresponding to the characteristics of multiple abnormal dataset groups based on the model training information for each abnormal dataset group, and generates multiple learning models corresponding to the characteristics of the multiple abnormal dataset groups, respectively. This allows the model training device 100 to obtain a highly accurate model based on the characteristics of the corresponding groups.

[0066] Additionally, if the training sets are also grouped in the same way, the corresponding outlier dataset groups may be used as test sets to test the corresponding models, thereby predicting the number of datasets required for training each group, and performing supplementary collection, etc.

[0067] Furthermore, the global metrics and local metrics used in the above description are not limited to the examples in the above description, and different global metrics and local metrics can be used for different types of medical images and different detection targets. Any global metrics and local metrics can be used as long as they can evaluate the variability of the model output results of medical images globally or locally.

[0068] The following describes the processing performed by the model training device 100 according to the first embodiment. Fig. 9 is a flowchart for explaining the processing performed by the model training device 100 according to the first embodiment.

[0069] First, the training function 10 collects a training set and a test set, each of which contains multiple data sets of medical images of the abdomen, establishes a true value (GT) from the multiple set data of the training set (step S901), and trains a model (step S902).

[0070] In steps S901 and S902, the training function 10 gradually inputs multiple data sets from the training set to train the model, while the evaluation function 20 tests the model generated at different stages using multiple data sets from the test set to generate a learning curve for the model.The evaluation function 20 then calculates global metrics (e.g., dice values ​​in this embodiment) for each of the multiple data sets in the test set, thereby acquiring multiple abnormal data sets that are outliers from the multiple data sets in the test set (step S903).

[0071] Next, in the grouping function 30, the segmentation function 31 performs low-resolution multi-organ segmentation on the multiple abnormal datasets acquired by the evaluation function 20 (step S904). Subsequently, the subregion partitioning function 32 sets the liver as a detection target region in the medical images included in the multiple abnormal datasets, and partitions each boundary region between the detection target region and multiple regions adjacent to the detection target region into multiple subregions (step S905). Then, the local metrics calculation function 33 calculates local metrics for each subregion for the multiple abnormal datasets (step S906). Next, the abnormal dataset grouping function 34 obtains multiple abnormal dataset groups by grouping the multiple abnormal datasets based on the subregions partitioned by the subregion partitioning function 32 and the local metrics calculated by the local metrics calculation function 33. That is, the abnormal dataset grouping function 34 divides each abnormal dataset into multiple groups (step S907).

[0072] Next, the model training information identification function 40 identifies model training information for the training function 10 to train a model for each abnormal dataset group grouped by the grouping function 30. For example, for each abnormal dataset group grouped by the grouping function 30, the model training information identification function 40 identifies the features of each abnormal dataset group as model training information and outputs the identified model training information from the output interface 120 (step S908). Also, for example, for each abnormal dataset group, the model training information identification function 40 identifies the number of training data (the number of datasets) required to generate a model corresponding to the features of the abnormal dataset group by testing the model using a set of datasets included in the abnormal dataset group as a test set, and outputs the identified model training information from the output interface 120 (step S909). Then, the training function 10 additionally collects training data (datasets) based on the model training information and trains the model by learning the training data (step S910).

[0073] As explained above, in the model training device 100 according to the first embodiment, the evaluation function 20 acquires the datasets used for model evaluation using global metrics, the grouping function 30 groups the datasets using local metrics, and the model training information identification function 40 obtains model training information more suited to the detection target. This allows for the provision of an accurate model through additional collection of training data. Therefore, the model training device 100 according to the first embodiment enables targeted and highly efficient data collection, reducing the time and cost required for labeling medical images. Furthermore, in the model training device 100 according to the first embodiment, the model training information identification function 40 identifies the characteristics of each abnormal dataset group and the number of required training data as model training information. The training function 10 then collects additional training data (datasets) based on the model training information and trains the model by learning the training data. This improves the accuracy of the trained model.

[0074] (Second embodiment) The second embodiment will be described with reference to Figures 10 to 14. The model training device 100 according to the second embodiment differs from the first embodiment mainly in the grouping method used by the grouping function. In the following, the second embodiment will be described focusing on the differences from the first embodiment, and explanations that overlap with the first embodiment will be omitted.

[0075] 10 is a block diagram showing an example of the functional configuration of a processing circuitry 140a of a model training device 100 according to the second embodiment. As shown in FIG. 10, the processing circuitry 140a executes a training function 10a, an evaluation function 20a, a grouping function 30a, and a model training information identification function 40a.

[0076] First, the processing executed by the training function 10a of the model training device 100 according to the second embodiment will be described.

[0077] For example, the training function 10a trains a model that extracts liver features from abdominal images (medical images of the abdomen) and divides the image into subsegments by learning from a plurality of existing data sets as learning data. The feature extraction and subsegmentation can extract, for example, an image region representing the liver, which is the region to be detected, from the abdominal image, and divide the abdominal image into subsegments, such as the liver, which is the region to be detected, and organs adjacent to the liver, lung lobes, blood vessels, nodules, etc. Here, for example, the training function 10a extracts an image region representing the liver, which is the region to be detected, and divides the subsegments, such as organs adjacent to the liver, and blood vessels and nodules within the liver, using the brightness value distribution of the abdominal image.

[0078] Next, the processing executed by the evaluation function 20 of the model training device 100 according to the second embodiment will be described.

[0079] The evaluation function 20 uses global metrics to evaluate the model generated by the training function 10a, thereby acquiring multiple outlier datasets from the multiple datasets of the test set used for evaluation as abnormal datasets. Specifically, while the training function 10 gradually inputs multiple datasets from the training set to generate a model, the evaluation function 20 generates a learning curve for the model by testing the model generated at different stages using multiple datasets from the test set. Then, the evaluation function 20 acquires multiple abnormal datasets from the test set by calculating global metrics for each of the multiple datasets in the test set. That is, the method of acquiring the abnormal datasets is the same as in the first embodiment. Therefore, a detailed description of the evaluation function 20 will be omitted.

[0080] Next, the processing executed by the grouping function 30a of the model training device 100 according to the second embodiment will be described.

[0081] The grouping function 30a obtains a plurality of abnormal data set groups by grouping the plurality of abnormal data sets obtained by the evaluation function 20a using local metrics.

[0082] Specifically, in the second embodiment, the grouping function 30a includes a division function 31a, a special region identification function 35, a local metrics calculation function 33a, and an abnormal data set grouping function 34a.

[0083] The division function 31a divides the medical image corresponding to the abnormal data set acquired by the evaluation function 20 so as to distinguish between regions having different image features within the detection target region to be detected. Specifically, the division function 31a performs low-resolution feature extraction and sub-segment division for each abnormal data set. Here, a 3D medical image captured with the liver as the detection target is used as an example, so the division function 31a divides the 3D volume data including the liver region according to the above-described feature extraction and sub-segment division method, and acquires a divided image in which sub-segment regions such as blood vessels and nodules are divided into multiple regions as features within the liver.

[0084] The special region identifying function 35 identifies a region having a specific characteristic as a special region among regions having different image characteristics. For example, a region suspected of being a lesion can be identified as a special region.

[0085] 11A to 11D are diagrams showing examples of lesion area divisions in the second embodiment. As shown in FIGS. 11A to 11D, an area having characteristics of a nodule in the liver (block-shaped shaded area in the diagram) may be set as a special area. Furthermore, differences in the characteristics of the special areas can be used to distinguish between multiple special areas.

[0086] 11A to 11D, nodules are divided into multiple lesion areas based on their characteristics. In the example shown in FIG. 11A, a block-shaped shadow is located inside the liver and has not spread to blood vessels. The special region identifying function 35 identifies the special region representing such a block-shaped shadow as lesion area P1. In the example shown in FIG. 11B, a block-shaped shadow is located inside the liver and has entered blood vessels. The special region identifying function 35 identifies the special region representing such a block-shaped shadow as lesion area P2. In FIG. 11C, a block-shaped shadow protrudes outside the liver and has spread to adjacent organs (e.g., the pancreas). The special region identifying function 35 identifies the special region representing such a block-shaped shadow as lesion area P3. In FIG. 11D, the block-shaped shadow has spread to lymph nodes. The special region identifying function 35 identifies the special region representing such a block-shaped shadow as lesion area P4. The special regions shown in FIGS. 11A to 11D are merely examples, and distinctions can be made according to other rules.

[0087] The local metric calculation function 33a calculates a local metric for each special region. Similarly, here, the local metric is HD, which indicates the distance between two local contours.

[0088] Next, the abnormal dataset grouping function 34a groups the abnormal datasets based on the special area identified by the special area identification function 35 and the local metrics calculated by the local metrics calculation function 33a. Specifically, the abnormal dataset grouping function 34a groups abnormal datasets that satisfy predetermined conditions into one group.

[0089] For example, the abnormal dataset grouping function 34a groups abnormal datasets with the same segmentation evaluation results in the same type of subregion into one group. The abnormal dataset grouping function 34a analyzes the image, anatomical, or pathological features of a special region based on the local metrics of the special region, and groups abnormal datasets in the special region that have the same image, anatomical, or pathological features into the same group. Specifically, the abnormal dataset grouping function 34a compares the local metrics of each special region with a threshold to determine whether the special region is over-segmented or under-segmented. Then, for each special region, the abnormal dataset grouping function 34a groups abnormal datasets in which an over-segmented special region exists into the same group, and groups abnormal datasets in which an under-segmented special region exists into the same group.

[0090] For example, the abnormal dataset grouping function 34a compares the absolute value |D| of the local metric D of each special region for each abnormal dataset, where D is the local metric calculated by the local metric calculation function 33a, with the set over-segmentation threshold T3 and under-segmentation threshold T4. If the comparison result is D > 0 and |D| > T3, the abnormal dataset grouping function 34a determines the evaluation result of the special region as over-segmentation. If the comparison result is D < 0 and |D| > T4, the abnormal dataset grouping function 34a determines the evaluation result of the special region as under-segmentation. As described above, by individually evaluating the model segmentation results for each special region for each abnormal dataset, multiple abnormal datasets can be divided into multiple groups, and each abnormal dataset in each group includes special regions of the same category with the same evaluation. Furthermore, the over-segmentation threshold T3 and the under-segmentation threshold T4 may be the same value or different values.

[0091] Fig. 12 is a diagram showing an example of abnormal dataset grouping in the second embodiment. When the multiple abnormal datasets are the abnormal datasets O1 to O20 shown in Fig. 3, for example, grouping as shown in Fig. 12 is performed. In Fig. 12, grouping is performed based on three lesion sites P1, P2, and P3, and the abnormal datasets O1 to O20 are divided into six groups.

[0092] For example, if the abnormal dataset grouping function 34a determines that the medical images of the abnormal datasets O5, O10, O14, O17, and O20 all contain a lesion site P1 that is a special region and that the division of the lesion site P1 is all over-segmented, the abnormal dataset grouping function 34a designates the group including the abnormal datasets O5, O10, O14, O17, and O20 in which the over-segmented lesion site P1 exists as “Group 1.” For example, if the abnormal dataset grouping function 34a determines that the medical images of the abnormal datasets O1, O6, and O9 all contain a lesion site P1 that is a special region and that the division of the lesion site P1 is all under-segmented, the abnormal dataset grouping function 34a designates the group including the abnormal datasets O1, O6, and O9 in which the over-segmented lesion site P1 exists as “Group 2.”

[0093] For example, if the abnormal dataset grouping function 34a determines that the medical images of the abnormal datasets O7, O11, and O13 all contain a lesion site P2 that is a special region and that the division of the lesion site P2 is all over-segmented, the abnormal dataset grouping function 34a classifies the group including the abnormal datasets O7, O11, and O13 in which the over-segmented lesion site P2 exists as "Group 3." For example, if the abnormal dataset grouping function 34a determines that the medical images of the abnormal datasets O16 and O18 all contain a lesion site P2 that is a special region and that the division of the lesion site P2 is all under-segmented, the abnormal dataset grouping function 34a classifies the group including the abnormal datasets O16 and O18 in which the over-segmented lesion site P2 exists as "Group 4."

[0094] For example, if the abnormal dataset grouping function 34a determines that the medical images of the abnormal datasets O2, O3, O4, and O8 all contain a lesion site P3 that is a special region and that the division of the lesion site P3 is all over-segmented, the abnormal dataset grouping function 34a classifies the groups including the abnormal datasets O2, O3, O4, and O8 in which the over-segmented lesion site P3 exists as "group 5." For example, if the abnormal dataset grouping function 34a determines that the medical images of the abnormal datasets O12, O15, and O19 all contain a lesion site P3 that is a special region and that the division of the lesion site P3 is all under-segmented, the abnormal dataset grouping function 34a classifies the groups including the abnormal datasets O12, O15, and O19 in which the over-segmented lesion site P3 exists as "group 6."

[0095] 12 is merely an example, and the present embodiment is not limited thereto. For example, the same abnormal data set may contain two lesion sites and be divided into two groups at the same time, or the abnormal data set may not contain any lesion sites or may not be included in either group because there is no over-segmentation or under-segmentation. Furthermore, multiple feature regions may be combined and grouped, and if the local metrics of the abnormal data set satisfy a predetermined condition, they can be grouped together.

[0096] Next, the process executed by the model training information identifying function 40a of the model training device 100 according to the second embodiment will be described.

[0097] The model training information identifying function 40a identifies model training information for the training function 10a to train a model for each abnormal dataset group grouped by the grouping function 30a.

[0098] By grouping using the grouping function 30a, the originally scattered datasets are divided into multiple abnormal dataset groups with common characteristics, and the model training information identification function 40a collects the characteristics of each abnormal dataset group and identifies these characteristics as model training information, thereby outputting the identified model training information from the output interface 120.

[0099] FIG. 13 is a diagram showing an example of the features of the abnormal data set groups output by the model training information identification function 40a in the second embodiment. As shown in the table in FIG. 13, the features of the medical images in each of the abnormal data set groups, "Group 1" to "Group 6," are analyzed individually. FIG. 13 illustrates examples of imaging features, anatomical features, and pathological features. For example, the abnormal data set group "Group 1" as a whole has imaging features I1, I2, and I3, anatomical features A1 and A2, and pathological features P1 and P2, while the abnormal data set group "Group 2" as a whole has imaging features I4, I5, and I6, anatomical features A3 and A4, and pathological feature P3. Here, a description of the features of the other groups is omitted.

[0100] The types of features are not limited to those listed in FIG. 13 . For example, the imaging features may be the acquisition protocol, imaging artifacts (e.g., metal artifacts in CT data, dynamic artifacts in MR), breath-holding conditions I1, I2, I3, partial volume effect (slice thickness), etc. For example, the anatomical features may be the location, size, etc. For example, the pathological features may be the pathological classification and stage of HCC (Hepatocellular Carcinoma), LI-RADS (Liver Imaging Reporting and Data System; imaging diagnosis of hepatocellular carcinoma), TNM (Tumor; size and spread of cancer), N (Nodes; presence or absence of lymph node metastasis), M (Metastasis; presence or absence of distant metastasis), etc.

[0101] Based on the above characteristics identified as model training information by the model training information identifying function 40a, the model training apparatus 100 can perform targeted learning when generating a model specialized for a specific application using training data including the above abnormal dataset group. For example, the training function 10 can collect datasets with the same characteristics based on the characteristics of the abnormal dataset group and combine the newly collected dataset with the above abnormal dataset group to train a new model. In this case, the model can output more accurate results for medical images with these characteristics.

[0102] The model training information identification function 40a can also identify model training information by predicting the number of datasets required to train a model or the accuracy of the model for each abnormal dataset group.

[0103] The model training device 100 can also group training sets similarly using the model training information output by the model training information identification function 40a as a grouping rule, and train a model using the grouped training sets. In this case, the training function 10 collects learning data corresponding to the characteristics of multiple abnormal dataset groups based on the model training information for each abnormal dataset group, and generates multiple models corresponding to the characteristics of the multiple abnormal dataset groups, respectively. This allows the model training device 100 to obtain highly accurate models based on the corresponding grouping characteristics.

[0104] Additionally, if the training sets are also grouped in the same way, the corresponding outlier dataset groups may be used as test sets to test the corresponding models, thereby predicting the number of datasets required for training each group, and performing supplementary collection, etc.

[0105] The following describes the processing performed by the model training device 100 according to the second embodiment. Fig. 14 is a flowchart for explaining the processing performed by the model training device 100 according to the second embodiment.

[0106] First, the training function 10a collects a training set and a test set, each of which contains multiple data sets of medical images of the abdomen, establishes a true value (GT) from the multiple set data of the training set (step S1401), and trains a model (step S1402).

[0107] In steps S1401 and S1402, the training function 10a gradually inputs multiple data sets from the training set to train the model, while the evaluation function 20a tests the model generated at different stages using multiple data sets from the test set to generate a learning curve for the model.The evaluation function 20a then calculates global metrics (e.g., dice values ​​in this embodiment) for each of the multiple data sets in the test set, thereby acquiring multiple abnormal data sets that are outliers from the multiple data sets in the test set (step S1403).

[0108] Next, in the grouping function 30a, the division function 31a performs low-resolution feature extraction and subsegmentation on the multiple abnormal datasets acquired by the evaluation function 20, thereby dividing the medical images included in the multiple abnormal datasets so as to distinguish between the detection target region to be detected and multiple regions in the medical image that represent features within the detection target region (step S1404). Next, the special region identification function 35 identifies, among the multiple regions in the medical images included in the multiple abnormal datasets, a region (special region) having specific features as a lesion region (step S1505). Then, the local metrics calculation function 33a calculates local metrics for each lesion region for the multiple abnormal datasets (step S1406). Next, the abnormal dataset grouping function 34a obtains multiple abnormal dataset groups by grouping the multiple abnormal datasets based on the lesion region identified by the special region identification function 35 and the local metrics calculated by the local metrics calculation function 33a. That is, the abnormal data set grouping function 34a divides each abnormal data set into a plurality of groups (step S1407).

[0109] Next, the model training information identification function 40a identifies model training information for the training function 10a to train a model for each abnormal dataset group grouped by the grouping function 30a. For example, for each abnormal dataset group, the model training information identification function 40a tests a model using a set of datasets included in the abnormal dataset group as a test set, thereby identifying the number of training data (the number of datasets) required to generate a model corresponding to the characteristics of the abnormal dataset group and the characteristics of the dataset group as model training information, and outputs the identified model training information from the output interface 120 (step S1408). Then, the training function 10a supplements and collects training data according to the number of training data and the characteristics of the dataset group, and re-trains the model by re-learning the training data (step S1409). That is, the initial learning model is retrained.

[0110] When retraining the initial learning model, the training function 10a can combine the supplementary collected learning data with the original learning data to train the model again as learning data. After combining the supplementary collected learning data with the original learning data, the features output to the model training information identification function 40a can be used as grouping rules to perform the same grouping as the abnormal data set of the training set described above, so that the learning data of each group can be used to individually train models corresponding to each feature.

[0111] As explained above, in the model training device 100 according to the second embodiment, the evaluation function 20a acquires the datasets used for model evaluation using global metrics, and the grouping function 30a groups the datasets using local metrics to obtain targeted and highly efficient model training information. This enables more accurate estimation of the quantity of training data collected, reducing the time and cost required for labeling medical images. Furthermore, in the model training device 100 according to the second embodiment, the model training information identification function 40a identifies the characteristics of each abnormal dataset group and the number of required training data as model training information. The training function 10a then collects additional training data (datasets) based on the model training information and trains the model by learning the training data, thereby improving the accuracy of the trained model.

[0112] In addition, local regions may be set according to other influencing factors of medical image segmentation, such as image change density, signal intensity, structural distortion, imaging conditions, etc., and the technical effects described above can be achieved by grouping abnormal data sets using local metrics for the local regions.

[0113] The components of each device in the above-described embodiments are conceptual and functionally independent, and do not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to that illustrated, and all or part of the devices can be functionally or physically distributed and integrated in any unit corresponding to various loads, usage conditions, etc. Furthermore, all or any part of the processing and functions performed in each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware based on hardwired logic.

[0114] The model training device and model training method described in the above-described embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. This program can be distributed via a network such as the Internet. This program can also be recorded on a computer-readable non-transitory recording medium such as a hard disk, floppy disk (FD), CD-ROM, MO, or DVD, and can be read from the recording medium and executed by a computer.

[0115] According to at least one of the embodiments described above, the accuracy of the model can be improved.

[0116] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications are possible without departing from the spirit of the invention. These embodiments and modifications are included within the scope and spirit of the invention, as well as within the scope of the invention described in the claims and their equivalents. [Explanation of symbols]

[0117] 10 Training Function 20 Evaluation Function 30 Grouping Function 40 Model training information identification function 100 Model Training Device

Claims

1. a training unit that obtains an initial learning model by learning a dataset including medical images as learning data; an evaluation unit that evaluates the initial learning model using global metrics to acquire an outlier dataset from among a plurality of datasets used in the evaluation as an abnormal dataset; a grouping unit that obtains a plurality of abnormal dataset groups by grouping the plurality of abnormal datasets obtained by the evaluation unit using local metrics; a model training information identification unit that identifies model training information for each of the abnormal dataset groups generated by the grouping unit; A model training device comprising:

2. the local metrics include local contour matching metrics or spatial distance metrics; The model training device of claim 1 .

3. The grouping unit a dividing unit that divides the medical image corresponding to the abnormal data set so that a positional relationship between a detection target portion to be detected and other portions adjacent to the detection target portion in the medical image can be distinguished; a sub-region dividing unit that divides a boundary region of the detection target region into a plurality of sub-regions according to the plurality of adjacent other regions; a local metrics calculation unit that calculates the local metrics for each of the sub-regions; an abnormal data set grouping unit that groups the plurality of abnormal data sets based on the sub-areas defined by the sub-area defining unit and the local metrics calculated by the local metrics calculating unit; The model training device of claim 1 , comprising:

4. the local metrics are local contour matching metrics; The abnormal data set grouping unit for each subregion, determining whether the subregion is over-segmented or under-segmented by comparing a local metric of the subregion to a threshold; For each sub-region, abnormal data sets including sub-regions of over-segmentation are grouped together, and abnormal data sets including sub-regions of under-segmentation are grouped together. The model training device according to claim 3 .

5. The grouping unit a division unit that divides the medical image corresponding to the abnormality dataset so as to distinguish regions having different image features in a detection target part that is a detection target; a special region identifying unit that identifies a region having a specific image feature as a special region among the regions; a local metrics calculation unit that calculates the local metrics for each of the special regions; an abnormal dataset grouping unit that groups the plurality of abnormal datasets based on the special region identified by the special region identifying unit and the local metrics calculated by the local metrics calculating unit; The model training device of claim 1 , comprising:

6. the local metrics are local contour matching metrics; The abnormal data set grouping unit analyzing imaging, anatomical, or pathological features of the special region based on the local metrics of the special region; Abnormal datasets of the special region having the same imaging, anatomical or pathological features are grouped together; The model training device according to claim 5 .

7. The model training information identification unit identifies imaging features, anatomical features, or pathological features for each of the abnormal dataset groups as the model training information. The model training device of claim 1 .

8. The model training information identification unit: For each of the abnormal dataset groups, fitting a learning curve to the model by testing the initial learning model using a set of datasets included in the abnormal dataset group as a test set; Identifying the model training information by predicting the number of learning data to be collected to build a model corresponding to the characteristics of the abnormal dataset group or the accuracy of the model based on the fitted learning curve. The model training device of claim 1 .

9. The training unit collects supplementary learning data based on the model training information and re-learns the learning data, thereby training the initial learning model. The model training device of claim 1 .

10. the training unit collects, for each of the abnormal dataset groups, learning data corresponding to the characteristics of the plurality of abnormal dataset groups based on the model training information, and generates a plurality of learning models corresponding to the characteristics of the plurality of abnormal dataset groups, respectively. The model training device of claim 1 .

11. a training step of obtaining an initial learning model by learning a dataset including medical images as learning data; an evaluation step of evaluating the initial learning model using global metrics to obtain an outlier dataset as an abnormal dataset from among a plurality of datasets used in the evaluation; a grouping step of obtaining a plurality of abnormal dataset groups by grouping the plurality of abnormal datasets obtained in the evaluation step using local metrics; a model training information identification step of identifying model training information for each of the abnormal dataset groups obtained in the grouping step; A model training method in a model training device comprising:

Images