Information processing device and learning method

Abstract

To provide a technique that can suppress a reduction in a recognition accuracy by a recognition device in a learning of the recognition device (learning model) based on machine learning.SOLUTION: An information processing apparatus according to the present invention includes a first acquisition unit that acquires learning image data, correct region image data representing a correction region of a first region included in the learning image data, and correction data relating to predetermined recognition performed by a learning model, a second acquisition unit that acquires estimation region image data representing an estimation region of the first region for the learning image data, and a learning unit that causes the learning model to be learned by using a first teacher data set configured to include the learning image data, the correction region image data, and the correct data, and a second teacher data set configured to include the learning image data, the estimation region image data, and the correct data.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The disclosure of this specification relates to an information processing apparatus and a learning method.

Background Art

[0002] In image recognition by machine learning, in some cases, target image data and image data representing the region of an object depicted in the target image data are input to a recognizer for recognition of the target image data. Image data representing the region of an object depicted in this target image data often uses the recognition result by another recognizer. Non-Patent Document 1 discloses a technique for training a recognizer based on machine learning by inputting CT image data, which is target image data, and image data representing the region of the pancreas segmented by another recognizer so that the region of pancreatic cancer can be recognized.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when the accuracy of the region segmented by another recognizer is low, the training of the recognizer based on machine learning may not proceed smoothly, and the recognition accuracy by the recognizer may decrease.

[0005] The purpose of this disclosure is to provide a technology that can suppress the decrease in recognition accuracy by a machine learning-based recognizer (learning model) during its training. [Means for solving the problem]

[0006] The information processing apparatus according to the present invention is an information processing apparatus that takes image data and region image data relating to a first region contained in the image data as input and trains a learning model that performs a predetermined recognition of the image data, comprising: a first acquisition unit that acquires training image data, ground truth region image data representing the ground truth region of the first region contained in the training image data, and ground truth data relating to the recognition; and a second acquisition unit that acquires estimated region image data representing the estimated region of the first region for the training image data. Input to the aforementioned learning model The aforementioned training image data, Input to the aforementioned learning model The aforementioned correct region image data and, Regarding the aforementioned recognition A first training dataset comprising the aforementioned ground truth data. The learning model is trained using the above method. , Furthermore, input to the learning model The aforementioned training image data, Input to the aforementioned learning model The estimated region image data and, Regarding the aforementioned recognition A second training data set comprising the aforementioned correct answer data. To Using, Trained using the first training dataset described above It includes a learning unit that trains the aforementioned learning model. [Effects of the Invention]

[0007] According to the disclosures herein, it is possible to suppress the decrease in recognition accuracy by a machine learning-based recognizer (learning model) during its training. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram showing an example of the functional configuration of an information processing device according to the first embodiment. [Figure 2] A diagram showing an example of the hardware configuration of an information processing device according to the first embodiment. [Figure 3] A diagram showing an example of the processing procedure of the information processing device according to the first embodiment. [Figure 4] A diagram showing an example of the processing procedure of the information processing device according to the first embodiment. [Figure 5] A diagram illustrating an image according to the first embodiment. [Figure 6] A diagram showing an example of the processing procedure of the information processing device according to the second embodiment. [Figure 7] A diagram showing an example of the functional configuration of an information processing device according to the third embodiment. [Figure 8] A diagram showing an example of the processing procedure of the information processing device according to the third embodiment. [Modes for carrying out the invention]

[0009] Embodiments of the information processing apparatus disclosed herein will be described below with reference to the drawings. The same or equivalent components, members, and processes shown in each drawing will be denoted by the same reference numerals, and redundant explanations will be omitted as appropriate. Furthermore, some components, members, and processes will be omitted from the drawings as appropriate.

[0010] The present invention will be described below using abdominal tissue depicted in abdominal CT image data acquired by an X-ray computed tomography (X-ray CT) scanner as an example. However, the present invention is applicable not only to abdominal tissue but to any structure. It is also applicable to tomographic images acquired by magnetic resonance imaging (MRI) scanners, positron emission tomography (PET) scanners, and ultrasound scanners, as well as ordinary images taken by cameras. Furthermore, the present invention is applicable whether the input image is a two-dimensional or three-dimensional image. Moreover, the present invention is applicable to images of general objects. It should be noted that the embodiments of the present invention are not limited to the embodiments described below.

[0011] <First Embodiment> (overview) In this embodiment, a learning method for a learning model for segmenting the region of a pancreatic tumor, which is a second region, will be described using, as inputs, abdominal CT image data, which is image data in which an object is depicted, and pancreatic region image data, which is region image data of the pancreas that is the region of the first region. Hereinafter, the learning model will be referred to as a recognizer.

[0012] First, the information processing apparatus according to this embodiment uses a first teacher data set including abdominal CT image data, pancreatic correct region image data representing the region of the true pancreas in the abdominal CT image data, and pancreatic tumor correct region image data representing the region of the true pancreatic tumor to train the recognizer. Subsequently, the information processing apparatus further trains the recognizer using a second teacher data set including abdominal CT image data, pancreatic estimated region image data representing the estimation result of the region of the pancreas in the abdominal CT image data, and pancreatic tumor correct region image data. Here, the abdominal CT image data, the pancreatic correct region image data, the pancreatic estimated region image data, and the pancreatic tumor correct region image data are, respectively, examples of learning image data, correct region image data of the first region, estimated region image data of the first region, and correct data for the learning image data.

[0013] In this embodiment, an example of using a convolutional neural network (CNN) as a recognizer will be described. The CNN in this embodiment is, for example, one that segments the region of an object from image data such as U-Net.

[0014] (Functional Configuration) Hereinafter, the functional configuration of the information processing apparatus 100 according to this embodiment will be described with reference to FIG. 1. As shown in the figure, the information processing apparatus 100 includes a first acquisition unit 110, an estimation unit 120, a second acquisition unit 130, and a learning unit 140. Further, the information processing apparatus 100 according to this embodiment includes an external storage device 70.

[0015] The storage device 70 is an example of a computer-readable storage medium and is a large-capacity storage device typified by a hard disk drive (HDD) or a solid state drive (SSD). The memory device 70 holds abdominal CT image data, ground truth region image data of the pancreas, and ground truth region image data of pancreatic tumors. The memory device 70 also holds a CNN that has been trained to segment the pancreatic region from abdominal CT image data. Hereafter, the CNN trained to segment the pancreatic region from abdominal CT image data will be referred to as the pancreatic segmentation CNN. The pancreatic segmentation CNN held in the memory device 70 may be data that holds the CNN's layer configuration and parameters (weights, biases, etc.), or it may be data that holds only the parameters.

[0016] Ground truth region image data is image data that represents the area of ​​the target tissue depicted in an abdominal CT image, and is annotated, for example, by a physician or radiographer. In ground truth region image data, for example, the voxel value of a voxel included in the area of ​​the target tissue is represented as 1, and the value of other voxels is represented as 0.

[0017] The storage device 70 may be configured as a function of the information processing device 100.

[0018] The first acquisition unit 110 acquires multiple abdominal CT image data, multiple ground truth region image data of the pancreas, and multiple ground truth region image data of pancreatic tumors corresponding to each of the multiple abdominal CT image data from the storage device 70. The first acquisition unit 110 then transmits the abdominal CT image data to the estimation unit 120 and transmits the abdominal CT image data, the ground truth region image data of the pancreas, and the ground truth region image data of pancreatic tumors to the learning unit 140.

[0019] The estimation unit 120 first receives abdominal CT image data from the first acquisition unit 110 and acquires the pancreatic segmentation CNN from the storage device 70. Based on the acquired data related to the pancreatic segmentation CNN, the estimation unit 120 constructs a CNN that can perform estimation processing (described later). Then, the estimation unit 120 inputs the abdominal CT image data into the pancreatic segmentation CNN and generates estimated region image data of the pancreas as a processing result.

[0020] Estimated region image data is image data that represents the area of ​​the target tissue depicted in abdominal CT image data. In estimated region image data, for example, the voxel value of the area of ​​the target tissue is represented as 1, and the value of the other voxels is represented as 0.

[0021] The second acquisition unit 130 acquires estimated pancreatic region image data from the estimation unit 120 and transmits it to the learning unit 140.

[0022] The learning unit 140 receives multiple abdominal CT image data, multiple ground truth region image data of the pancreas corresponding to each of the multiple abdominal CT image data, and multiple ground truth region image data of pancreatic tumors from the first acquisition unit 110. It also receives estimated region image data of the pancreas corresponding to each of the multiple abdominal CT image data from the second acquisition unit 130. Next, the learning unit 140 trains a recognition system using a first training dataset consisting of multiple abdominal CT image data and the ground truth region image data of the pancreas and pancreatic tumors corresponding to each of them. This recognition system is a CNN that takes abdominal CT image data and pancreatic region image data as input and is trained to segment the pancreatic tumor region. In other words, the learning unit 140 trains the CNN using abdominal CT image data, pancreatic region image data, and pancreatic tumor region image data as a set of training data. Next, the learning unit 140 uses a second training dataset, which consists of multiple abdominal CT image data and corresponding estimated pancreatic region image data and ground truth region image data of pancreatic tumors, to further train the CNN trained on the first training dataset. Additional training refers to updating the parameters of the recognition system (CNN) again, using the parameters of the recognition system obtained by training with the first training dataset as initial values. At this time, some parameters may be fixed, and only the remaining parameters may be updated. Then, the learning unit 140 completes the series of training processes. Next, the learning unit 140 stores the CNN trained using the second training dataset in the memory device 70. Hereafter, the CNN trained by the learning unit 140 to segment the pancreatic tumor region will be referred to as the pancreatic tumor segmentation CNN.

[0023] Here, the image data according to this embodiment will be explained using Figure 5. Figure 5(a) shows abdominal CT image data 510. In this embodiment, abdominal CT image data 510 is a three-dimensional image cropped to include the pancreatic region 512. In the abdominal CT image data 510, for example, the pancreatic region 512 and the pancreatic tumor region 511 are depicted. Figure 5(b) shows ground truth region image data 520 of the pancreatic tumor, which represents the ground truth region 521 of the pancreatic tumor, which is the true pancreatic tumor region in the abdominal CT image data 510. Also, Figure 5(c) shows the ground truth region image data 530 of the true pancreas in the abdominal CT image data 510, and Figure 5(d) shows the estimated region image data 540 of the pancreas generated by the estimation unit 120. The ground truth region image data 530 and the estimated region image data 540 of the pancreas represent the ground truth region 532 and the estimated region 542 of the pancreas, respectively. The estimated pancreatic region 542 is the result of estimating (segmenting) the pancreatic region 512 in the abdominal CT image data 510. Here, the correct pancreatic region 532 and the estimated pancreatic region 542 do not necessarily coincide, and differences may occur between the two. In the example shown in the figure, a difference occurs between the correct pancreatic region 532 and the pancreatic tumor region 511 near where the pancreatic tumor region 511 is located. If the recognition system is trained using only the second training dataset, which contains estimated region image data that differs from the true pancreatic region, errors in the estimated pancreatic region image data may have adverse effects such as stagnation of learning, potentially reducing the accuracy of the recognition system. On the other hand, when achieving fully automated pancreatic cancer segmentation from abdominal CT image data, the estimated pancreatic region image data is input as pancreatic region image data during the inference of the pancreatic cancer segmentation CNN. Therefore, if the recognition system is trained using only the first training dataset, which contains correct pancreatic region image data, the accuracy of the recognition system may decrease due to differences in region image data between training and inference. Therefore, in this embodiment, by using the first training dataset and the second training dataset, the characteristics of pancreatic region image data are efficiently trained in the recognition system, thereby suppressing a decrease in the accuracy of the recognition system.

[0024] (Hardware configuration) Next, the hardware configuration of the information processing device 100 will be described using Figure 2. The information processing device 100 has the configuration of a known computer (information processing device). The hardware configuration of the information processing device 100 includes a CPU 201, main memory 202, magnetic disk 203, display memory 204, monitor 205, mouse 206, and keyboard 207.

[0025] The CPU (Central Processing Unit) 201 primarily controls the operation of each component. The main memory 202 stores control programs executed by the CPU 201 and provides a workspace for program execution by the CPU 201. The magnetic disk 203 stores programs for implementing various application software, including the OS (Operating System), device drivers for peripheral devices, and programs for processing described later. By the CPU 201 executing programs stored in the main memory 202, magnetic disk 203, etc., the functions (software) of the information processing device 100 shown in Figure 1 and the processing shown in the flowchart described later are realized.

[0026] The display memory 204 temporarily stores display data. The monitor 205 is, for example, a CRT monitor or an LCD monitor, and displays images, text, etc., based on the data from the display memory 204. The mouse 206 and keyboard 207 are used for pointing input and character input, respectively, by the user. Each of the above components is connected to each other via a common bus 208 so that they can communicate with one another.

[0027] The CPU 201 corresponds to an example of a processor or control unit. In addition to the CPU 201, the information processing device 100 may have at least one of a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array). Alternatively, the CPU 201 may be replaced with at least one of a GPU or an FPGA. The main memory 202 and magnetic disk 203 correspond to an example of memory or storage device.

[0028] (Processing procedure) Next, the processing procedure of the information processing device 100 according to this embodiment will be explained with reference to Figure 3.

[0029] (Step S300) In step S300, the first acquisition unit 110 acquires multiple abdominal CT image data, multiple ground truth region image data of the pancreas and multiple ground truth region image data of pancreatic tumors corresponding to each of the multiple abdominal CT image data from the storage device 70. The first acquisition unit 110 then transmits the multiple abdominal CT image data to the estimation unit 120 and transmits the multiple abdominal CT image data, multiple ground truth region image data of the pancreas and multiple ground truth region image data of pancreatic tumors to the learning unit 140.

[0030] (Step S310) In step S310, the estimation unit 120 acquires a pancreatic segmentation CNN from the storage device 70 that has been trained to segment the pancreatic region from abdominal CT image data. Next, the estimation unit 120 inputs the abdominal CT image data received from the first acquisition unit 110 into the pancreatic segmentation CNN, estimates the pancreatic region depicted in the abdominal CT image data, and generates estimated pancreatic region image data. At this time, the estimation unit 120 may perform any post-processing on the estimated pancreatic region image data. Any image processing is a process to refine the segmented pancreatic region, such as binarization, maximum connected component acquisition, morphology processing, or graph cut segmentation. Other processes such as resolution conversion may also be performed. Then, the second acquisition unit 130 acquires the estimated pancreatic region image data from the estimation unit 120 and transmits it to the learning unit 140.

[0031] (Step S320) In step S320, the learning unit 140 receives multiple abdominal CT image data, multiple ground truth region image data of the pancreas corresponding to each of the multiple abdominal CT image data, and multiple ground truth region image data of pancreatic tumors from the first acquisition unit 110. The learning unit 140 also receives multiple estimated region image data of the pancreas corresponding to each of the multiple abdominal CT image data from the second acquisition unit 130. In this embodiment, a teaching dataset containing multiple abdominal CT image data, multiple ground truth region image data of the pancreas, and multiple ground truth region image data of pancreatic tumors is defined as the first teaching dataset. A teaching dataset containing multiple abdominal CT image data, multiple estimated region image data of the pancreas, and multiple ground truth region image data of pancreatic tumors is defined as the second teaching dataset. The learning unit 140 uses the first and second teaching datasets to train a pancreatic tumor segmentation CNN and saves the trained pancreatic tumor segmentation CNN in the storage device 70. The processing procedure in step S320 will be explained in detail below with reference to Figure 4.

[0032] (Step S421) In step S421, the learning unit 140 performs a training process on the pancreatic tumor segmentation CNN using a first training dataset containing ground truth region image data of the pancreas, and updates the parameters of the CNN. The parameters of the CNN are typically: These include the kernel weights and biases of the convolutional layer, and the offset and scale coefficients of the batch normalization layer. In this embodiment, the pancreatic tumor segmentation CNN receives abdominal CT image data and ground truth region image data of the pancreas corresponding to the abdominal CT image data as input, and is trained to segment the pancreatic tumor region. Therefore, the learning unit 140 treats the abdominal CT image data, the ground truth region image data of the pancreas, and the ground truth region image data of the pancreatic tumor as a set of training data and updates the parameters of the pancreatic tumor segmentation CNN.

[0033] In this step, multiple training data (batches) are selected from the first training dataset, and the parameters of the pancreatic tumor segmentation CNN are updated using known methods such as backpropagation. At this time, known loss functions such as Mean Squared Error (MSE) or Dice Coefficient Loss can be used. Similarly, known optimization methods such as Stochastic Gradient Descent (SGD) or Adam can be used. Other hyperparameters such as learning rate and batch size can also be set to appropriate values.

[0034] (Step S422) In step S422, the learning unit 140 determines whether the training termination conditions for the pancreatic tumor segmentation CNN using the first training dataset have been met, and then determines the next step. The training termination conditions are typically, but not limited to, reaching a predetermined number of epochs since the start of training or reaching a predetermined value in the loss function. For example, the accuracy of a dataset other than the first training dataset (validation dataset) may reach a predetermined value. Multiple termination conditions may also be combined. In this embodiment, the training termination condition is that the number of epochs since the start of training reaches a predetermined upper limit of the number of epochs (maximum number of epochs). If the training termination conditions are met, the learning unit 140 proceeds to step S423; otherwise, it returns to step S421.

[0035] (Step S423) In step S423, the learning unit 140 performs additional learning on the pancreatic tumor segmentation CNN trained using the first training dataset, using a second training dataset containing estimated pancreatic region image data. The learning process is performed using a known method, similar to that in step S421. At this time, hyperparameters such as the loss function, optimization method, learning rate, and batch size may be the same as those used when training the pancreatic tumor segmentation CNN with the first training dataset, or some of them may be modified.

[0036] (Step S424) In step S424, the learning unit 140 determines whether the training completion conditions for the pancreatic tumor segmentation CNN using the second training dataset have been met, and then determines the next step. The training completion conditions are known conditions, such as those exemplified in step S422. Note that the same completion conditions as in step S422 may be used, or different completion conditions may be used. If the training completion conditions are met, the learning unit 140 saves the pancreatic tumor segmentation CNN trained by the above training process to the storage device 70 and terminates step S320; otherwise, it returns to step S423.

[0037] Using the learning method described above, a pancreatic tumor segmentation CNN can be generated that segments the pancreatic tumor region using abdominal CT image data and pancreatic region image data contained in the abdominal CT image as input. This trained pancreatic tumor segmentation CNN (recognizer) can then be used for recognition processing (inference processing). As an example of recognition processing, Figure 1 describes a method for segmenting the region of a pancreatic tumor and generating estimated region image data of the pancreatic tumor using a recognition unit (not shown). In the information processing device 100, the recognition unit first acquires a pancreatic segmentation CNN from the storage device 70. Next, the recognition unit receives abdominal CT image data to be recognized from the first acquisition unit 110 and pancreatic region image data included in the abdominal CT image data from the second acquisition unit 130. Subsequently, the recognition unit inputs the abdominal CT image data and the pancreatic region image data into the pancreatic tumor segmentation CNN and segments the region of the pancreatic tumor. Then, as a result of segmenting the pancreatic tumor region, the recognition unit generates estimated region image data of the pancreatic tumor and stores it in the storage unit 70. The recognition unit may be configured in the information processing device 100 or it may be implemented as a function of another device.

[0038] (effect) As described above, the information processing device 100 according to this embodiment trains a recognition device using a first training dataset containing ground truth region image data of the pancreas, and then further trains the recognition device using a second training dataset containing estimated region image data of the pancreas. By training in this stepwise manner, even if there are errors in the pancreatic region held in the estimated region image data of the pancreas, features from the pancreatic region image data can be efficiently learned, thereby suppressing a decrease in the accuracy of the pancreatic tumor segmentation CNN.

[0039] (Variations) In the above explanation, the pancreas and pancreatic tumors in abdominal CT image data were used as examples, but the present invention is applicable to any object in any image data. In other words, it is applicable to medical image data other than CT, as well as image data captured by cameras.

[0040] In the above explanation, an image recognition task was given as an example in which a second region (pancreatic tumor) is segmented using target image data (abdominal CT image data) and region image data of a first region (pancreas) as input. However, the applicable image recognition tasks are not limited to this. For example, it could be image classification of the target image data, or class classification of the first or second region. In this case, the information processing device obtains labels corresponding to the classification target (data indicating the class to which the target image data, the first region, or the second region belongs) as ground truth data corresponding to the target image data and the ground truth region image data of the first region, and trains the recognition device. Alternatively, it could be an image recognition task that identifies the location of the first or second region. In the case of segmentation of the first or second region, ground truth region image data corresponding to the first or second region to be classified is obtained as ground truth data corresponding to the target image data and the ground truth region image data of the first region, and the recognition device is trained. For bounding box detection or landmark detection, the correct coordinate data corresponding to the first or second region to be classified is obtained as ground truth data, and the recognizer is trained using this data. Furthermore, this method can be applied to any image recognition task as long as the input consists of the target image data and the region image data of the first region. Multiple first and second regions may also exist.

[0041] In the above description, the estimation unit 120 was configured as a function of the information processing device 100, but it may also be configured as a function of an external device. In this case, in step S310, the information processing device 100 uses the second acquisition unit 130 to acquire estimated pancreatic region image data from an external device having the function of the estimation unit 120, or from an external storage device storing estimated pancreatic region image data.

[0042] In the above explanation, the ground truth region image data and estimated region image data were binary image data represented by voxel values ​​of 0 or 1. However, this representation format is just an example, and any format capable of representing the region of the target tissue is acceptable. For example, an image that represents the likelihood of being a region of the target tissue using continuous values. The data may be data, or it may be image data representing multiple target organizations in a multi-level representation. Furthermore, the representation formats of the ground truth region image data and the estimated region image data may be the same, or they may be different.

[0043] In the above explanation, a deep learning-based recognizer such as a CNN was used as an example of a recognizer, but it is not limited to this. For example, a classification tree such as a Support Vector Machine (SVM) or Random Forest may be used as a recognizer. In this case, the learning method should be an appropriate method depending on the recognizer. Furthermore, the recognizer in the estimation unit 120 is not limited to the machine learning-based methods described above, but any image processing-based method may be used. For example, a method based on alignment with an organ probability atlas or a threshold-based method may be used.

[0044] In the above explanation, the example given was the use of annotated data by physicians or radiologists as the ground truth region image data. However, the ground truth region image data is not limited to this; any data with higher reliability than the estimated region image data can be used. For example, the ground truth region image data may be generated by a first recognition system that can be expected to perform highly accurate region estimation, and the estimated region image data may be generated by a second recognition system that is computationally simpler than the first recognition system. In this case, the effects disclosed herein can be enjoyed even without annotated region image data by physicians or other professionals. It should be noted that when performing a series of inference processes from pancreatic region segmentation to pancreatic tumor region segmentation, it may be difficult to use a recognition system that can be expected to perform highly accurate region estimation due to computational cost constraints (computation time and computational resources). In such cases, applying the learning method disclosed herein makes it possible to reduce the adverse effects of low accuracy of region image data generated by a simple recognition system.

[0045] In the above explanation, the example given was the use of the same abdominal CT image data as input to the pancreatic segmentation CNN in the estimation unit 120 and the pancreatic tumor segmentation CNN in the learning unit 140. However, different image data may be used. For example, different smoothing filters may be applied to each input, or a smoothing filter may be applied to only one of them. Alternatively, abdominal CT image data with different resolutions may be used as inputs. Furthermore, image data with different contrast enhancement states and alignment may be used as inputs, or image data from different modalities may be used.

[0046] <Second Embodiment> (overview) In the first embodiment, a recognition system (learning model) was trained stepwise using a first training dataset containing ground truth region image data of the pancreas and a second training dataset containing estimated region image data of the pancreas. In the second embodiment, a method for constructing a third training dataset based on the first and second training datasets and training the recognition system using the third training dataset will be described. In this case, the information processing device according to this embodiment constructs the third training dataset while changing the ratio of training data contained in the first training dataset to training data contained in the second training dataset when predetermined conditions are met. More specifically, at the start of each learning step, which is an epoch (an example of predetermined conditions), the third training dataset is constructed so that the number of training data samples taken from the first training dataset gradually decreases according to the number of epochs since the start of learning. Conversely, the third training dataset is constructed so that the proportion of training data contained in the second training dataset in the third training dataset gradually increases as the number of epochs increases.

[0047] (Functional Configuration) The configuration of the information processing apparatus according to this embodiment is the same as that of the information processing apparatus 100 according to the first embodiment. Hereinafter, with reference to Figure 1, the functional configuration of the information processing apparatus according to this embodiment will be described, omitting any overlap with that of the information processing apparatus according to the first embodiment. The storage device 70, the first acquisition unit 110, the estimation unit 120, and the second acquisition unit 130 are the same as in the first embodiment, and therefore their description will be omitted.

[0048] The learning unit 140 receives multiple abdominal CT image data, multiple ground truth region image data of the pancreas corresponding to each of the multiple abdominal CT image data, multiple ground truth region image data of pancreatic tumors, and multiple estimated region image data of the pancreas, similar to the first embodiment. Next, the learning unit 140 constructs a third training dataset based on a first training dataset containing ground truth region image data of the pancreas and a second training dataset containing estimated region image data of the pancreas, and performs training on the pancreatic tumor segmentation CNN. In this embodiment, the learning unit 140 constructs the third training dataset at the start of an epoch such that the number of training data samples taken from the first training dataset gradually decreases according to the number of epochs. After completing a series of training processes, the learning unit 140 saves the trained pancreatic tumor segmentation CNN to the storage device 70.

[0049] (Hardware configuration) The hardware configuration of the information processing device 100 according to this embodiment is the same as that of the first embodiment, so a description will be omitted.

[0050] (Processing procedure) Next, the processing procedure of the information processing device 100 in this embodiment will be described using Figure 3. In the following description, parts that overlap with the description of the information processing device 100 in the first embodiment will be omitted.

[0051] (Steps S300, S310) Steps S300 and S310 are the same as steps S300 and S310 in the first embodiment, so their description is omitted.

[0052] (Step S320) In step S320, the learning unit 140 receives from the first acquisition unit 110 a plurality of abdominal CT image data, a plurality of ground truth region image data of the pancreas corresponding to each of the plurality of abdominal CT image data, and a plurality of ground truth region image data of pancreatic tumors. The learning unit 140 also receives from the second acquisition unit 130 a plurality of estimated region image data of the pancreas corresponding to each of the plurality of abdominal CT image data. In this embodiment, similar to the first embodiment, a teaching dataset containing a plurality of abdominal CT image data, a plurality of ground truth region image data of the pancreas, and a plurality of ground truth region image data of pancreatic tumors is defined as the first teaching dataset. A teaching dataset containing a plurality of abdominal CT image data, a plurality of estimated region image data of the pancreas, and a plurality of ground truth region image data of pancreatic tumors is defined as the second teaching dataset. The learning unit 140 constructs a third training dataset based on the first teaching dataset and the second teaching dataset. Then, the learning unit 140 trains the pancreatic tumor segmentation CNN while sequentially updating the third training dataset, and saves the trained pancreatic tumor segmentation CNN in the memory device 70. Figure 6 illustrates the processing procedure of step S320.

[0053] (Step S621) In step S621, the learning unit 140 constructs a third training dataset based on a first training dataset containing ground truth region image data of the pancreas and a second training dataset containing estimated region image data of the pancreas. In this embodiment, at the start of the epoch, the third training dataset is constructed. A third training dataset is constructed such that the number of training data points in the first training dataset relative to the master dataset gradually decreases in proportion to the number of epochs since the start of training. For example, let m1 be the total number of training data points in the first training dataset. The number of training data points m1' to sample from the first training dataset when constructing the third training dataset is determined by the following formula. m1' = m1 × (1 - n current / n maximum )

[0054] Here, n current and n maximum m1' represents the number of epochs since the start of learning and the upper limit of the number of epochs, respectively. The upper limit of the number of epochs is an example of the learning termination condition in step S623, which will be described later. In other words, as the number of epochs increases, the number of training data sampled from the first training dataset gradually decreases, and in the final epoch (the epoch in which the upper limit of the number of epochs is reached), m1'=0. On the other hand, the number of training data selected from the second training dataset when constructing the third training dataset does not change according to the epoch. That is, if the total number of training data contained in the second training dataset is m2, then the number of training data sampled from the second training dataset when constructing the third training dataset is always m2'=m2. As a result, the proportion of training data contained in the first training dataset and the second training dataset that constitute the third training dataset is m1 × (1-n current / n maximum ):m2. In this way, at the start of the epoch (an example of predetermined conditions), the third training dataset is constructed while changing the ratio of training data included in the first training dataset and training data included in the second training dataset.

[0055] (Step S622) In step S622, the learning unit 140 performs a training process on the pancreatic tumor segmentation CNN using the third training dataset and updates the parameters of the CNN. The training process is performed using a known method, similar to step S421 in the first embodiment. At this time, hyperparameters such as the loss function, optimization method, learning rate, and batch size are arbitrarily set to appropriate values.

[0056] (Step S623) In step S623, the learning unit 140 determines whether the conditions for terminating the learning of the pancreatic tumor segmentation CNN using the third training dataset have been met, and determines the next step. The learning termination conditions are known ones, such as those exemplified in step S422 of the first embodiment. If the learning termination conditions are met, the learning unit 140 saves the pancreatic tumor segmentation CNN learned by the above learning process to the storage device 70 and terminates step S320; otherwise, it returns to step S621.

[0057] (effect) As described above, the information processing device 100 according to this embodiment constructs a third training dataset based on a first training dataset containing ground truth region image data of the pancreas and a second training dataset containing estimated region image data of the pancreas, and trains the recognition device. By training the recognition device in this way, even if there are errors in the pancreatic region held in the estimated region image data of the pancreas, the recognition device can efficiently learn features from the pancreatic region image data, thereby suppressing a decrease in the accuracy of the pancreatic tumor segmentation CNN.

[0058] (Variations) The above explanation showed an example where the ratio of training data in the first training dataset to the training data in the second training dataset is changed at the start of each learning step, which is an epoch, but it is not limited to this. For example, the ratio may be changed on the condition that a predetermined stage within the learning epoch (such as the start or end of the epoch) is reached. The ratio may be changed on the condition that the learning evaluation metric (loss function or other metric) reaches a predetermined value. For example, if the condition is that the value of the loss function reaches a predetermined value, a third training dataset is constructed such that m1':m2'=1:1 at the start of learning, m1':m2'=1:2 when the first predetermined value is reached, and m1':m2'=1:3 when the second predetermined value is reached. In the above explanation, when constructing the third training dataset, the ratio with the number of training data included in the second training dataset was changed by gradually decreasing the number of training data sampled from the first training dataset, but this is not the only way. For example, the number of training data sampled from the first training dataset may be fixed at a predetermined number, and the number of training data sampled from the second training dataset may be gradually increased according to the number of epochs since the start of learning. Also, the number of training data sampled from the first or second training dataset does not have to be monotonically increased / decreased, and the number of samples may be increased or decreased during learning. Furthermore, not limited to the method described above, any method may be used to change the conditions or ratios for constructing the third training dataset, such as changing the ratio of training data contained in the first training dataset and the second training dataset during the learning process.

[0059] In the above explanation, the ratio of the number of training data points in the first and second training datasets that constitute the third training dataset was changed when certain conditions were met. However, it is not necessary to change the ratio. That is, in a series of training sessions, the third training dataset may be constructed using all the training data points contained in the first and second training datasets. Alternatively, the third training dataset may be constructed by randomly sampling from each dataset at a fixed ratio.

[0060] <Third Embodiment> (overview) In the first embodiment, a method was described in which a pancreatic tumor segmentation CNN was trained using a first training dataset containing ground truth region image data of the pancreas, and then further trained using a second training dataset containing estimated region image data of the pancreas. In this embodiment, an example is described in which a pancreatic tumor segmentation CNN trained using the first training dataset containing ground truth region image data of the pancreas is obtained externally, and then further trained using the second training dataset.

[0061] (Functional Configuration) The functional configuration of the information processing device 700 according to this embodiment will be described below with reference to Figure 7. As shown in the figure, the information processing device 700 is composed of a training data acquisition unit 710, a learning model acquisition unit 720, and a learning unit 730. The information processing device 700 according to this embodiment also includes an external storage device 70.

[0062] The storage device 70 holds abdominal CT image data, estimated pancreatic region image data, and ground truth region image data of pancreatic tumors. The storage device 70 also holds a pancreatic tumor segmentation CNN trained on a first training dataset that includes abdominal CT image data, ground truth region image data of the pancreas, and ground truth region image data of pancreatic tumors.

[0063] The training data acquisition unit 710 acquires multiple abdominal CT image data, multiple estimated pancreatic region image data corresponding to each of the multiple abdominal CT image data, and multiple ground truth region image data of pancreatic tumors from the storage device 70, and transmits them to the learning unit 730.

[0064] The learning model acquisition unit 720 acquires a pancreatic tumor segmentation CNN trained using a first training dataset containing ground truth region image data of the pancreas from the storage device 70, and transmits it to the learning unit 730.

[0065] The learning unit 730 receives multiple abdominal CT image data, multiple estimated pancreatic region image data corresponding to each of the multiple abdominal CT image data, and multiple ground truth region image data of pancreatic tumors from the training data acquisition unit 710. This set of image data is referred to as the second training dataset. The learning unit 730 also obtains the pancreatic tumor segmentation CNN trained on the first training dataset from the learning model acquisition unit 720. Next, the learning unit 730 further trains the pancreatic tumor segmentation CNN trained on the first training dataset using the second training dataset, which includes the estimated pancreatic region image data. After completing the training process, the learning unit 730 saves the pancreatic tumor segmentation CNN trained on the second training dataset to the storage device 70.

[0066] (Hardware configuration) The hardware configuration of the information processing device 700 according to this embodiment is the same as that of the first embodiment, so a description will be omitted.

[0067] (Processing procedure) Next, the processing procedure of the information processing device 700 according to this embodiment will be described using Figure 8.

[0068] (Step S800) In step S800, the training data acquisition unit 710 acquires multiple abdominal CT image data, multiple estimated pancreatic region image data, and multiple ground truth region image data of pancreatic tumors from the storage device 70, and transmits them to the learning unit 730.

[0069] (Step S810) In step S810, the learning model acquisition unit 720 acquires a pancreatic tumor segmentation CNN trained using a first training dataset containing ground truth region image data of the pancreas from the storage device 70, and transmits it to the learning unit 730.

[0070] (Step S820) In step S820, the learning unit 730 receives multiple abdominal CT image data, multiple estimated pancreatic region image data corresponding to each of the multiple abdominal CT image data, and multiple ground truth region image data of pancreatic tumors from the training data acquisition unit 710. This set of image data is referred to as the second training dataset. The learning unit 730 also obtains the pancreatic tumor segmentation CNN trained on the first training dataset from the training model acquisition unit 720. Subsequently, the learning unit 730 further trains the pancreatic tumor segmentation CNN using the second training dataset, which includes multiple abdominal CT image data, multiple estimated pancreatic region image data corresponding to the multiple abdominal CT images, and multiple ground truth region image data of pancreatic tumors. The training process is performed using known methods, similar to step S421 in the first embodiment. At this time, appropriate values ​​can be used for hyperparameters such as the loss function, optimization method, learning rate, and batch size.

[0071] (Step S830) In step S830, the learning unit 730 determines whether the conditions for terminating the learning of the pancreatic tumor segmentation CNN using the second training dataset have been met, and then determines the next step. The learning termination conditions are known ones, such as those exemplified in step S422 of the first embodiment. If the learning termination conditions are met, the learning unit 730 saves the pancreatic tumor segmentation CNN learned by the above learning process to the storage device 70 and terminates the series of processes; otherwise, it returns to step S820.

[0072] (effect) As described above, the information processing device 700 according to this embodiment further trains the recognizer, which has been trained using a first training dataset containing ground truth region image data of the pancreas, using a second training dataset containing estimated region image data of the pancreas. By training in this manner, even if there are errors in the pancreatic region held in the estimated region image data of the pancreas, features from the pancreatic region image data can be efficiently learned, thereby suppressing a decrease in the accuracy of the pancreatic tumor segmentation CNN. [Explanation of Symbols]

[0073] 100: Information Processing Device 110: First acquisition unit 120: Estimation unit 130: Second acquisition unit 140: Learning unit 700: Information Processing Device 710: Training data acquisition unit 720: Learning model acquisition unit 730: Learning unit

Claims

1. An information processing device that takes image data and region image data relating to a first region contained in the image data as input and trains a learning model that performs a predetermined recognition of the image data, A first acquisition unit that acquires training image data, correct region image data representing the correct region of the first region included in the training image data, and correct data relating to the recognition, A second acquisition unit acquires estimated region image data representing the estimated region of the first region for the training image data, A learning unit that trains the learning model using a first training dataset comprising the training image data to be input to the learning model, the ground truth region image data to be input to the learning model, and the ground truth data relating to the recognition, and further trains the learning model trained using the first training dataset using a second training dataset comprising the training image data to be input to the learning model, the estimation region image data to be input to the learning model, and the ground truth data relating to the recognition. An information processing device characterized by having the following features.

2. The information processing apparatus according to Claim 1, wherein the learning model is a model that recognizes any of the classes to which the image data belongs, the classes to which the first region contained in the image data belongs, and the classes to which the second region contained in the first region contained in the image data belongs, and the ground truth data is data indicating any of the classes to which the learning image data belongs, the classes to which the first region belongs, and the classes to which the second region of the learning image data belongs.

3. The information processing apparatus according to claim 1, characterized in that the correct data is correct region image data of the first region, or correct region image data of the second region for the training image data.

4. The information processing apparatus according to claim 1, characterized in that the ground truth data is ground truth coordinate data representing a position with respect to the first region, or ground truth coordinate data representing a position with respect to the second region with respect to the training image data.

5. The information processing apparatus according to any one of claims 1 to 4, further comprising a first region estimation unit that estimates a first region from the training image data, wherein the second acquisition unit acquires estimated region image data of the first region based on the estimation by the estimation unit.

6. The information processing apparatus according to claim 5, characterized in that the estimation unit receives image data as input and generates estimated region image data representing the estimated region of the first region included in the image data.

7. The information processing apparatus according to any one of claims 1 to 6, characterized in that the learning unit, when certain conditions are met, trains the learning model using a third training dataset constructed by including at least a portion of the training data included in the first training dataset and at least a portion of the training data included in the second training dataset, and further trains the learning model trained using the third training dataset using a training dataset obtained by changing the ratio of the number of training data included in the first training dataset and the number of training data included in the second training dataset in the third training dataset.

8. The information processing apparatus according to claim 7, characterized in that the learning unit trains the learning model trained using the third training dataset using a training dataset in which the proportion of training data included in the second training dataset included in the third training dataset is increased.

9. The information processing apparatus according to claim 7, characterized in that the predetermined condition is reaching a predetermined stage in learning, or the learning evaluation index reaches a predetermined value.

10. The information processing apparatus according to any one of claims 1 to 9, further comprising a recognition unit that performs a predetermined recognition process on target image data using a trained model learned by the learning process of the learning unit.

11. The information processing apparatus according to any one of claims 1 to 10, characterized in that the first region is a region of the pancreas.

12. The information processing apparatus according to any one of claims 2 to 4, characterized in that the second region is the region of a pancreatic tumor.

13. A learning method for a learning model that performs a predetermined recognition of image data by inputting image data and region image data relating to a first region contained in said image data, A first acquisition step of acquiring training image data, ground truth region image data representing the ground truth region of the first region included in the training image data, and ground truth data relating to the recognition, A second acquisition step of acquiring estimated region image data representing the estimated region of the first region for the training image data, The learning model is trained using a first training dataset comprising the training image data to be input to the learning model, the ground truth region image data to be input to the learning model, and the ground truth data related to the recognition, and further, the learning model is trained using the first training dataset to be input to the learning model A learning step in which the learning model trained using the first training dataset is trained using a second training dataset comprising training image data, estimated region image data to be input to the learning model, and the correct data relating to the recognition. A learning method characterized by having the following features.

14. A recognition method that performs a predetermined recognition process using a trained model trained by the learning method described in claim 13, The acquisition step involves obtaining the data to be recognized, A recognition step which involves performing a predetermined recognition process on the data to be recognized, A recognition method characterized by having the following features.

15. A program for a computer to execute the learning method described in claim 13 or the recognition method described in claim 14.

Images