Medical image processing device, liver segment division method and program

Abstract

Provided are a medical image processing device, a liver segment division method, and a program that are capable of dividing a liver into segments in a medical image accurately. A medical image processing device according to one aspect of the present disclosure is configured to: use a trained model, which is generated by executing machine learning using training data including first input data including a first image of a liver and portal vein branch labeling data in which a portal vein branch label is assigned to each portal vein branch corresponding to a liver segment for a portal vein region in the liver of the first image, to receive second input data of the same type as the first input data; use the trained model to assign a portal vein branch label to each image unit element of a second image region including at least the liver region of a second image included in the second input data; and divide the liver region included in the second input data into a plurality of liver segments on the basis of the portal vein branch label assigned to each image unit element of the second image region.

Description

[Technical Field]

[0001] The present disclosure relates to a medical image processing device, a liver segmentation method, and a program, and in particular to machine learning technology and image processing technology for handling medical images of areas including the liver. [Background technology]

[0002] The liver is divided into eight segments, S1 to S8, using the branching portal vein as an index: S1 is the caudate lobe, S2 is the left lateral posterior segment (dorsal lateral segment), S3 is the left lateral anterior segment (ventrolateral segment), S4 is the left medial segment (quadratic lobe), S5 is the right anterior inferior segment, S6 is the right posterior inferior segment, S7 is the right posterior superior segment, and S8 is the right anterior superior segment.

[0003] Segmenting the liver into its anatomical regions is important medically, and segmentation of the liver into S1 to S8 regions is required in various situations, such as when reporting the location of an abnormal mass in a radiological report, and it is necessary to properly identify liver regions on medical images.

[0004] Patent document 1 and non-patent document 1 describe a method for identifying the area controlled by each blood vessel as a liver segment by extracting blood vessels within the liver region and using a Voronoi diagram to identify the area controlled by each blood vessel that includes areas other than blood vessels within the liver region (such as the liver parenchyma).

[0005] Furthermore, Patent Document 2 describes a convolutional neural network (CNN) that uses deep learning to perform a classification task of vascular branches in the liver. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-033349 [Patent Document 2] International Publication No. 2020 / 203552 [Non-patent literature]

[0007] [Non-Patent Document 1] R Beichel et al., “Liver segment approximation in CT data for surgical resection planning”, Medical Imaging 2004: Image Processing. Edited by Fitzpatrick, J. Michael; Sonka, Milan, 2004, Proceedings of the SPIE, Volume 5370, pp. 1435-1446 Summary of the Invention [Problem to be solved by the invention]

[0008] The liver's segments S1 to S8 do not have clear physical or anatomical boundaries, and the positions at which the liver segments are divided vary greatly depending on the individual. Therefore, a method for automatically and uniquely dividing liver segments from medical images is desired.

[0009] Conventionally, the method of dividing the liver region into segments S1 to S8 is basically based on the labeled portal vein branches (subportal veins) within the liver. Specifically, the portal vein region is extracted from a medical image of the liver and the portal vein branches are labeled. Then, Voronoi division is performed based on the distance from the labeled portal vein branches, and the governing region is set based on the obtained results.

[0010] The portal vein is classified and labeled as portal vein branches S1 to S8 corresponding to the liver segments S1 to S8. For example, the region controlled by the S1 portal vein branch is the S1 liver segment, and there may be a one-to-one correspondence between the labels of the portal vein branches and the labels of the liver segments. In the case of Voronoi tessellation, the liver segment to which each voxel belongs is determined based on the criterion of which portal vein branch label region each voxel is closest to in the 3D image. The portal vein branch labels are associated with the eight portal vein branches S1 to S8 and are used to classify (divide) a predetermined image region into eight regions. Here, the portal vein branch labels classify the portal vein region as a predetermined image region into eight portal vein branch regions, and the liver segment to which each voxel belongs is determined based on the distance from the eight portal vein branch regions.

[0011] However, the boundary surfaces of the liver segments S1 to S8 are not simple. This makes it difficult for the doctor to uniquely set appropriate boundary surfaces for the liver segments. It is also difficult to automate the complex processing performed by the doctor.

[0012] Furthermore, some images captured by different modalities have low voxel density values ​​in the portal vein region, or the portal vein region is not clearly visible in the image. In particular, there are images in which the terminal portion of the portal vein is not visible. As a result, a method using Voronoi tessellation based on the identified portal vein region may not be able to accurately segment the liver segments. In other words, the accuracy of segmenting the liver segments using Voronoi tessellation varies depending on how the blood vessel (portal vein) is depicted in the image (see Figures 12 and 13).

[0013] To address this issue, a method using machine learning to generate a learning model for the task of segmenting liver segments can be considered. Specifically, a large number of data sets are prepared as learning data, each consisting of an input image and data for that image with the correct labels for each liver segment (S1 to S8). These data sets are then used for supervised learning. This generates a trained model that outputs the results of segmenting liver segments.

[0014] However, the above method requires doctors to label the input images by attaching correct labels for each liver segment. Preparing correct labels for each liver segment (S1 to S8) for a large number of images places a heavy burden on doctors. Furthermore, there are individual differences among doctors when labeling liver segments, making it difficult to compile uniform correct data (training data). To achieve the desired task using machine learning, a method is needed that reduces the workload of doctors and others when generating training data, and allows for the relatively easy preparation of a large amount of high-quality training data.

[0015] The present disclosure has been made in view of the above circumstances, and aims to provide a medical image processing device, a liver segmentation method, and a program that can accurately segment the liver from a medical image. [Means for solving the problem]

[0016] A medical image processing device according to one aspect of the present disclosure includes a processor and a storage device storing a program executed by the processor, the program including a trained model generated by performing machine learning using training data including first input data including a first image of a liver and portal vein branch labeling data in which a portal vein branch label is assigned to each portal vein branch corresponding to a liver segment for a portal vein region in the liver in the first image, the trained model being trained to accept input of the first input data and output a labeling result of a portal vein branch label for each image unit element of the first image region of the first image. Lu's The model has updated parameters, and the processor executes the instructions of the program to accept second input data, which is input data of the same type as the first input data and includes a second image of the liver, and uses the trained model to assign portal vein branch labels to each image unit element of a second image region of the second image, and divides the liver region included in the second input data into multiple liver segments based on the portal vein branch labels assigned to each image unit element of the second image region.

[0017] According to this embodiment, liver segments can be segmented accurately regardless of how blood vessels appear in the image to be processed. Furthermore, the portal vein branch labeling data used for learning to generate the trained model of this embodiment can be generated relatively easily without placing an excessive workload on doctors. Image unit elements in a three-dimensional image may be understood as voxels, and image unit elements in a two-dimensional image as pixels.

[0018] In a medical image processing device according to another aspect of the present disclosure, the first input data includes at least one of a CT (Computed Tomography) image of an area including the liver and a portal vein mask image in which the portal vein area is identified, and the first image may be a CT image or a portal vein mask image.

[0019] In the medical image processing device according to another aspect of the present disclosure, the first input data may include a CT image and a portal vein mask image.

[0020] In a medical image processing device according to another aspect of the present disclosure, the first input data may further include at least one of a liver mask image in which the liver region is identified, a vein mask image in which the vein region is identified, and an inferior vena cava mask image in which the inferior vena cava region is identified.

[0021] In a medical image processing device according to another aspect of the present disclosure, the first input data may include a portal vein mask image, a liver mask image, and a vein mask image.

[0022] In a medical image processing device according to another aspect of the present disclosure, the first image region may be the entire region of the first image, and the second image region may be the entire region of the second image.

[0023] In a medical image processing device according to another aspect of the present disclosure, the portal vein branch labels may be labels that classify the portal vein branches into eight classes corresponding to eight types of liver segments, S1 to S8.

[0024] In a medical image processing device according to another aspect of the present disclosure, the trained model may be constructed using a convolutional neural network.

[0025] In a medical image processing device according to another aspect of the present disclosure, the machine learning process for generating a trained model may include calculating a loss for a score map indicating the likelihood of a portal vein branch label output from the trained model, limited to portal vein regions to which a portal vein branch label is assigned in the portal vein branch labeling data corresponding to the first input data, and updating parameters of the trained model based on the calculated loss.

[0026] In a medical image processing device according to another aspect of the present disclosure, each of the first image and the second image may be a three-dimensional image.

[0027] In a medical image processing device according to another aspect of the present disclosure, the processor may be configured to label liver segment labels indicating liver segments based on portal vein branch labels assigned to each image unit element of the second image region.

[0028] In a medical image processing device according to another aspect of the present disclosure, the second input data may include a CT image of an area including the liver, and the processor may be configured to extract the liver area from the CT image included in the second input data and invalidate label information labeled for areas of the second image area other than the extracted liver area.

[0029] Invalidating label information includes, for example, the concepts of deleting label information or ignoring label information.

[0030] In a medical image processing device according to another aspect of the present disclosure, the processor may be configured to generate a liver segment segmentation image divided into liver segments by converting portal vein branch labels assigned to each image unit element of the second image region into liver segment labels.

[0031] A liver segment segmentation method according to another aspect of the present disclosure is a liver segment segmentation method in which a computer divides a liver region in an image into liver segments, using learning data including first input data including a first image of the liver and portal vein branch labeling data in which portal vein branches corresponding to liver segments in a portal vein region within the liver in the first image are labeled with portal vein branch labels. Machine Generating a learning model through machine learning and and updating parameters of the learning model based on the labeling result of the portal vein branch labels output by the learning model for each image unit element of the first image region of the first image, to generate a trained model. and The method includes accepting second input data, which is input data of the same type as the first input data and includes a second image related to the liver; assigning portal vein branch labels to each image unit element of a second image region of the second image using the trained model; and dividing the liver region included in the second input data into a plurality of liver segments based on the portal vein branch labels assigned to each image unit element of the second image region.

[0032] A program according to another aspect of the present disclosure is a program for operating a computer as a medical image processing device, and the program uses learning data including first input data including a first image of a liver and portal vein branch labeling data in which portal vein branches corresponding to liver segments are labeled with portal vein branch labels for portal vein regions within the liver in the first image. MachineThe program includes a trained model generated by machine learning, the trained model being trained to accept input of first input data and output a labeling result of a portal vein branch label for each image unit element of a first image region including at least a liver region of the first image. The program causes a computer to accept second input data, which is input data of the same type as the first input data and includes a second image related to the liver, assign a portal vein branch label to each image unit element of the second image region of the second image included in the second input data using the trained model, and divide the liver region included in the second input data into a plurality of liver segments based on the portal vein branch labels assigned to each image unit element of the second image region. [Effects of the Invention]

[0033] According to the present disclosure, it is possible to accurately segment the liver from a medical image. [Brief explanation of the drawings]

[0034] [Figure 1] FIG. 1 is a block diagram showing an example of an image processing device that performs processing to generate data for learning. [Figure 2] FIG. 2 is a block diagram showing an example of an information processing device that performs labeling of portal vein branches in a portal vein region. [Figure 3] FIG. 3 is a conceptual diagram showing an example of a training data set stored in the training data storage unit. [Figure 4] FIG. 4 is a conceptual diagram showing an overview of the learning phase when generating a trained model to be applied to the medical image processing apparatus according to the first embodiment. [Figure 5] FIG. 5 is a block diagram showing an example of the configuration of a learning device. [Figure 6] FIG. 6 is a flowchart showing the flow of the learning process performed by the learning device. [Figure 7] FIG. 7 is a conceptual diagram showing an overview of processing in the inference phase using the trained model of the first embodiment. [Figure 8]FIG. 8 is a block diagram showing the configuration of a medical image processing apparatus according to the first embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of a liver segment division method using the medical image processing apparatus according to the first embodiment. [Figure 10] FIG. 10 is a conceptual diagram showing an outline of the learning phase in the second embodiment. [Figure 11] FIG. 11 is a block diagram showing an overview of the inference phase using a trained model generated by the training method of the second embodiment. [Figure 12] FIG. 12 shows an example of an image illustrating an example of a liver segment division method using Voronoi division according to a comparative example. [Figure 13] FIG. 13 is a diagram showing a comparison between the processing result of liver segment segmentation based on Voronoi division according to a comparative example and the result of appropriate liver segment segmentation. DETAILED DESCRIPTION OF THE INVENTION

[0035] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings.

[0036] Overview of the First Embodiment Here, we will explain using an example of a CT image obtained by imaging an area including a patient's liver using a CT device. Creating labels for each section of the liver (correct labels for liver sections) on a CT image of an area including the liver is a heavy burden for the doctor, and there are problems with large individual differences in labeling.

[0037] On the other hand, labeling the portal vein region within the liver by type of portal vein branch region is not as burdensome for physicians as labeling liver segments, and there is less variability in the labeling results between physicians.

[0038] In light of this background, in the first embodiment of the present disclosure, machine learning is performed using, as one piece of training data, image data in which portal vein regions are labeled by type of portal vein branch. Furthermore, a trained model obtained as a result of the machine learning is used to realize liver segmentation (segmentation of liver segments). That is, the training data is data used to train a learning model 50 (described later) by machine learning. Furthermore, a trained model 650 is generated by training the learning model 50 by machine learning using the training data. That is, the trained model 650 is a model in which the parameters of the learning model 50 are optimized. The trained model 650 is applied to a medical image processing device 70 according to the first embodiment.

[0039] 《Preparing training data》 1 and 2 are block diagrams showing an example of a method for generating data for training. FIG. 1 shows an example of an image processing device 10 that performs processing to generate a liver mask image LM, a portal vein mask image PM, and a vein mask image HM from a CT image IM. FIG. 2 shows an example of an information processing device 40 that generates a portal vein branch label map PLM from the CT image IM, the liver mask image LM, the portal vein mask image PM, and the vein mask image HM. In this embodiment, the training data includes the CT image IM, the liver mask image LM, the portal vein mask image PM, and the vein mask image HM, as well as the portal vein branch label map PLM (see FIG. 3).

[0040] The CT image IM is a three-dimensional image reconstructed from three-dimensional data obtained by sequentially capturing two-dimensional slice images. The liver mask image LM, portal vein mask image PM, and venous mask image HM are also three-dimensional images. Note that the term "image" includes the meaning of image data.

[0041] The image processing device 10 is realized using computer hardware and software. Software is synonymous with a program. The image processing device 10 includes a processor 12 and a computer-readable medium 14, which is a non-transitory tangible entity. The form of the image processing device 10 is not particularly limited, and it may be a server, a workstation, or a personal computer.

[0042] The processor 12 includes a central processing unit (CPU). The processor 12 may also include a graphics processing unit (GPU). The computer-readable medium 14 includes a memory serving as a main storage device and a storage serving as an auxiliary storage device. The computer-readable medium 14 may be, for example, a semiconductor memory, a hard disk drive (HDD) device, a solid state drive (SSD) device, or a combination of these.

[0043] A plurality of programs including an image processing program, data, etc. are stored in the computer-readable medium 14. The processor 12 executes instructions of the programs stored in the computer-readable medium 14 to function as a liver extraction processing unit 15, a portal vein extraction processing unit 16, and a vein extraction processing unit 17.

[0044] The liver extraction processing unit 15 performs processing to extract a liver region from the input CT image IM. A liver mask image LM is generated by the liver extraction processing unit 15. The liver mask image LM is an image in which the liver region is identified, and may be, for example, a binary image in which the voxel value of the liver region in the CT image IM is "1" and the voxel value of other regions (non-liver regions) is "0".

[0045] The portal vein extraction processing unit 16 performs processing to extract a portal vein region from the input CT image IM. A portal vein mask image PM is generated by the portal vein extraction processing unit 16. The portal vein mask image PM is an image in which the portal vein region is identified, and may be, for example, a binary image in which the voxel value of the portal vein region in the CT image IM is "1" and the voxel value of other regions (non-portal vein regions) is "0."

[0046] The vein extraction processing unit 17 performs processing to extract vein regions from the input CT image IM. The vein extraction processing unit 17 generates a vein mask image HM. The vein mask image HM is an image in which vein regions are identified, and may be, for example, a binary image in which the voxel value of the vein regions in the CT image IM is "1" and the voxel value of other regions (non-vein regions) is "0."

[0047] The liver extraction processor 15, the portal vein extraction processor 16, and the vein extraction processor 17 may each be configured to extract the liver, portal vein, or vein regions using a trained model trained to generate a mask image from an input image using machine learning such as deep learning. A model that performs such an image recognition task is realized using, for example, a CNN such as V-net.

[0048] The image processing device 10 acquires a CT image IM from the image storage unit 20, and generates a liver mask image LM, a portal vein mask image PM, and a vein mask image HM corresponding to the CT image IM. The images generated by the image processing device 10 are linked (associated) with the original CT image IM and stored in the learning data storage unit 30.

[0049] 1 illustrates an example in which the image processing device 10 generates three types of mask images: a liver mask image LM, a portal vein mask image PM, and a vein mask image HM. However, the mask images generated by the image processing device 10 are not limited to these. For example, the image processing device 10 may generate other mask images, such as an inferior vena cava mask image in which the inferior vena cava region is identified. Furthermore, the image processing device 10 may be configured to generate only some of the multiple types of mask images illustrated in FIG. 1, for example, it may be configured to generate only the portal vein mask image PM.

[0050] The image storage unit 20 includes a large-capacity storage device that stores a large number of images, including CT images IM. The image storage unit 20 may be, for example, a DICOM (Digital Imaging and Communication in Medicine) server on a medical institution's network. The DICOM server is a server that operates according to the DICOM specifications. The DICOM server is a computer that stores and manages various data, including images captured using a CT device or other modalities, and is equipped with a large-capacity external storage device and a database management program. The image processing device 10 can acquire multiple CT images IM from the image storage unit 20 via a communication line (not shown).

[0051] The learning data storage unit 30 includes a large-capacity storage for storing data used for learning. The learning data storage unit 30 may be included in the image processing device 10. Alternatively, a part of the storage area of ​​the image storage unit 20 may be used as the learning data storage unit 30.

[0052] Next, an example of labeling the portal vein branches on the portal vein mask image PM generated by the image processing device 10 will be described.

[0053] FIG. 2 shows an example of an information processing device 40. In the information processing device 40, portal vein branches are labeled in the portal vein region of the portal vein mask image PM. This labeling operation is performed, for example, by a doctor Dr. using the information processing device 40. The information processing device 40 may be a computer including a processor 42 and a computer-readable medium 44, which is a non-transitory tangible object. The hardware configurations of the processor 42 and the computer-readable medium 44 may be similar to the corresponding elements of the processor 12 and the computer-readable medium 14 described in FIG. 1.

[0054] The information processing device 40 may be in the form of a server, a personal computer, a workstation, a tablet terminal, etc. For example, the information processing device 40 may be a viewer terminal for image interpretation, etc.

[0055] An input device 47 and a display device 48 are connected to the information processing device 40. The input device 47 is configured, for example, by a keyboard, a mouse, a multi-touch panel, other pointing devices, a voice input device, or an appropriate combination of these. The display device 48 is configured, for example, by a liquid crystal display, an organic electro-luminescence (OEL) display, a projector, or an appropriate combination of these.

[0056] The information processing device 40 can acquire data stored in the learning data storage unit 30 and display it on the display device 48. For example, the information processing device 40 displays a portal vein mask image PM on the display device 48 and accepts input of portal vein branch labels from the input device 47. The information processing device 40 is not limited to acquiring the portal vein mask image PM, but can also acquire a CT image IM, a liver mask image LM, a vein mask image HM, and the like, and display them on the display device 48.

[0057] The computer-readable medium 44 stores a plurality of programs, data, etc., including a program for labeling portal vein branches in the portal vein region of the portal vein mask image PM. The processor 42 functions as a portal vein branch labeling processing unit 46 by executing instructions of the programs stored in the computer-readable medium 44.

[0058] The portal vein branch labeling processing unit 46 generates a portal vein branch label map PLM based on information (label information) related to portal vein branch labels input by the doctor Dr to the input device 47. As described above, portal vein branch labels are labels for classifying a predetermined image region into eight regions. Here, the predetermined image region is the portal vein region. Therefore, the label information is information that identifies which portal vein branch each of multiple portal vein branch regions, which are multiple partial regions included in the portal vein region in the liver, corresponds to. The portal vein branch labeling processing unit 46 accepts input of the label information and generates a portal vein branch label map PLM, which serves as training data, based on the input label information.

[0059] The portal vein is classified into eight classes of portal vein branches, S1 to S8, corresponding to each of the liver segments S1 to S8. That is, the portal vein belonging to the S1 liver segment is classified as the S1 portal vein branch, the portal vein belonging to the S2 liver segment is classified as the S2 portal vein branch, and so on. Therefore, the label information defines portal vein branch labels S1 to S8 for classifying portal vein regions into eight portal vein branch regions corresponding to the liver segments. A table may be created that defines the correspondence between portal vein branch labels and liver segment labels. It is also possible to interpret the portal vein branch labels by simply replacing them with liver segment labels.

[0060] While checking images such as the portal vein mask image PM displayed on the display device 48, the user, a doctor Dr, uses the input device 47 to assign portal vein branch labels to each portal vein branch area in the image.

[0061] That is, the doctor Dr specifies the correspondence between each portal vein branch region and the portal vein branch label via the input device 47. In accordance with this input, the portal vein branch labeling processing unit 46 assigns a portal vein branch label to each portal vein branch region included in the portal vein region, thereby generating a portal vein branch label map PLMj. That is, the portal vein branch labeling processing unit 46 generates a portal vein branch label map PLM in which one classification label (portal vein branch label) of eight classes S1 to S8 is assigned to each portal vein branch region, which is a partial region of the portal vein region, in accordance with the information input via the input device 47. In the portal vein branch label map PLMj, the portal vein region is classified into eight classes by the portal vein branch label, and the portal vein branch region is colored differently for each portal vein branch label. The portal vein branch label map PLMj may be understood to have an image similar to a portal vein branch segmentation image.

[0062] Based on information input via the input device 47, the portal vein branch label map PLM is linked to the portal vein mask image PM from which it was generated, and is stored in the learning data storage unit 30. The portal vein branch label map PLM is also linked to the original CT image IM, and is stored in the learning data storage unit 30.

[0063] Although Figures 1 and 2 illustrate an example in which the image processing device 10 and the information processing device 40 are separate devices, it is also possible to realize the processing functions of the image processing device 10 and the processing functions of the information processing device 40 in a single computer.

[0064] 3 is a conceptual diagram showing an example of a training data set stored in the training data storage unit 30. The training data storage unit 30 stores a plurality of data sets, each of which is linked to a CT image IMj, a liver mask image LMj, a portal vein mask image PMj, a vein mask image HMj, and a portal vein branch label map PLMj. The subscript "j" represents an index number for distinguishing the plurality of data sets.

[0065] Here, a CT image IMj, a liver mask image LMj, a portal vein mask image PMj, and a vein mask image HMj are prepared as input data, and a portal vein branch label map PLMj is prepared as teacher (correct answer) data corresponding to the input data. In FIG. 3, a plurality of data sets in which the input data and the teacher data are linked are shown as the training dataset. Specifically, the training dataset is a collection of data including a plurality of data sets in which the input data and the portal vein branch label map PLMj corresponding to the input data are linked.

[0066] In this embodiment, an example is given in which a combination of four types of images, namely, a CT image IMj, a liver mask image LMj, a portal vein mask image PMj, and a vein mask image HMj, is used as input data. While using a combination of four types of images is one preferred embodiment, the combination of images used as input data is not limited to this example. The input data may include at least one of the CT image IMj and the portal vein mask image PMj.

[0067] 《Explanation of the learning phase》 FIG. 4 is a conceptual diagram showing an overview of the learning phase. In the learning phase, machine learning of the learning model 50 is performed based on input image data, and a trained model 650 is generated. The trained model 650 is applied to the medical image processing device 70 according to the first embodiment. The learning model 50 is configured using CNN. The learning model 50 may be configured using a neural network based on the V-net architecture, for example.

[0068] The learning model 50 is trained to output a portal vein branch label for a predetermined image region based on input image data (input image). As described above, the portal vein branch label is a label associated with eight portal vein branches S1 to S8 and used to classify the predetermined image region into eight regions. Here, the portal vein branch labels are used to classify all regions of the input image (all image regions) into eight classes as the predetermined image region. 4 receives a CT image IMj, a liver mask image LMj, a portal vein mask image PMj, and a vein mask image HMj as input images. The learning model 50 is trained to output a portal vein branch label for each voxel in the entire image region of the input image. Furthermore, the learning model 50 outputs a score indicating the probability of the portal vein branch label for each voxel in the entire image region of the input image. That is, the learning model 50 outputs a portal vein branch label and a score for each of all voxels included in the entire image region of the portal vein mask image PMj. A voxel is an example of an "image unit element" in this disclosure.

[0069] That is, the learning model 50 outputs a prediction map 52 indicating portal vein branch labels and scores. The prediction map 52 is a portal vein branch label score map in which a score indicating the likelihood of the portal vein branch label is assigned to each voxel in the entire image region. This score map is a probability map indicating which portal vein branch label, S1 to S8, each voxel is most likely to have, and may be a map in which portal vein branch labels are predicted for the entire region of the image (entire image region).

[0070] In this embodiment, the entire image region is classified into eight classes, from S1 portal vein branch to S8 portal vein branch. Therefore, the prediction map 52 output from the learning model 50 is a probability map for each portal vein branch label from S1 portal vein branch to S8 portal vein branch. Note that, for convenience of illustration, each image is shown as a two-dimensional slice cross-sectional image in FIG. 4, but the images actually handled are three-dimensional images.

[0071] Anatomically, a portal vein branch label is assigned to a partial region of the portal vein region. However, the learning model 50 assigns a score indicating the probability of a portal vein branch label to each voxel in the entire image region, including not only the portal vein region in the input image but also regions other than the portal vein region. On the other hand, when calculating loss between the prediction map 52 output from the learning model 50 and the portal vein branch label map PLMj, which is training data, the loss calculation is performed only for the portal vein region in the input image, and regions other than the portal vein region are ignored, so that information other than the portal vein region is not reflected in the loss.

[0072] In the portal vein branch label map PLMj of the training data, the portal vein region is labeled as the correct answer. Therefore, only the scores predicted for the voxels in the portal vein region in the prediction map 52 are reflected in the loss. On the other hand, the scores predicted for the voxels outside the portal vein region in the prediction map 52 are ignored and no loss is calculated. In this way, the loss between the prediction map 52 and the portal vein branch label map PLMj is calculated by limiting the target to the portal vein region only, and the parameters of the learning model 50 are updated based on the calculated loss. Note that the loss may also be referred to as an error.

[0073] By training the learning model 50 using a plurality of training data sets, the parameters of the learning model 50 are optimized, and a trained model is obtained as a result of the training.

[0074] According to the training method of this embodiment, the target region for loss calculation is limited to the portal vein region in the image. However, by using a large number of training data sets, images containing portal vein regions of various shapes can be trained. As a result, training that covers the entire liver region is performed, improving the prediction accuracy of labeling for each voxel.

[0075] The input data combining the CT image IMj, liver mask image LMj, portal vein mask image PMj, and vein mask image HMj is an example of "first input data" in the present disclosure. The portal vein mask image PMj is an example of "first image" in the present disclosure. The entire image region of the portal vein mask image PMj is an example of "first image region" in the present disclosure. The portal vein branch label map PLMj is an example of "portal vein branch labeling data" in the present disclosure. A data set including the CT image IMj, liver mask image LMj, portal vein mask image PMj, vein mask image HMj, and portal vein branch label map PLMj is an example of "learning data" in the present disclosure.

[0076] Example of learning device configuration 5 is a block diagram showing an example configuration of a learning device 60. The learning device 60 includes a processor 602, a computer-readable medium 604 that is a non-transitory tangible entity, a communication interface 606, and an input / output interface 608. The hardware configuration of the processor 602 and the computer-readable medium 604 may be similar to the corresponding elements of the processor 12 and the computer-readable medium 14 described in FIG. 1. The learning device 60 may take the form of a server, a personal computer, or a workstation.

[0077] The processor 602 is connected to the computer-readable medium 604, the communication interface 606, and the input / output interface 608 via a bus 610. An input device 614 and a display device 616 are connected to the bus 610 via the input / output interface 608.

[0078] The hardware configurations of input device 614 and display device 616 may be similar to the corresponding elements of input device 47 and display device 48 described in Fig. 2. Learning device 60 is connected to a communication line (not shown) via communication interface 606, and is communicatively connected to external devices such as learning data storage unit 30.

[0079] The computer-readable medium 604 stores a plurality of programs, data, etc., including a learning processing program 630 and a display control program 640. The processor 602 executes instructions of the learning processing program 630 to function as each of the processing units: a data acquisition unit 632, a learning model 50, a loss calculation unit 634, and an optimizer 635.

[0080] The data acquisition unit 632 acquires training data from the training data storage unit 30. The loss calculation unit 634 calculates the loss between the prediction map 52 and the portal vein branch label map PLM. The portal vein branch label map PLM is training data corresponding to the input data used to generate the prediction map 52. The loss calculation unit 634 calculates the loss by limiting the target to the portal vein region where a correct label exists in the portal vein branch label map PLM, and ignores the score values ​​of voxels in regions other than the portal vein region, and does not use them in the loss calculation. The loss calculation by the loss calculation unit 634 is performed using, for example, a loss function.

[0081] The optimizer 635 determines the amount of update for the parameters of the learning model 50 based on the loss calculated by the loss calculation unit 634, and performs the update process for the parameters of the learning model 50. The optimizer 635 updates the parameters based on an algorithm such as gradient descent. The parameters of the learning model 50 include the filter coefficients (weights of connections between nodes) of the filters used in processing each layer of the CNN, the biases of the nodes, and the like.

[0082] The learning device 60 acquires data from the learning data storage unit 30 and performs machine learning on the learning model 50. The learning device 60 can acquire (read) data and update parameters in units of mini-batches, each of which is a collection of multiple learning data sets. In this way, the learning device 60 generates a trained model 650.

[0083] <Example of learning method> 6 is a flowchart showing the flow of the learning process by the learning device 60. In step S102, the processor 602 acquires data from the learning data storage unit 30. Specifically, the processor 602 accepts input of learning data and acquires a learning data set from the learning data storage unit 30.

[0084] In step S104, the processor 602 generates a prediction map 52 of portal vein branch labels using the learning model 50. Specifically, the processor 602 inputs an image included in the input data (see FIG. 3) into the learning model 50, and uses the learning model 50 to generate a prediction map 52 of portal vein branch labels corresponding to the input data.

[0085] Next, in step S106, the processor 602 calculates a loss between the prediction map 52 and the portal vein branch label map PLM, limited to voxels in the portal vein region.

[0086] Then, in step S108, the processor 602 performs processing to update the parameters of the learning model 50 based on the calculated loss. The operations from step S102 to step S108 may be performed in mini-batch units.

[0087] In step S110, the processor 602 determines whether to terminate learning. The learning termination condition may be determined based on the loss value or the number of parameter updates. In a method based on the loss value, for example, the learning termination condition may be that the loss has converged within a specified range. In a method based on the number of updates, for example, the learning termination condition may be that the number of updates has reached a specified number.

[0088] If the determination result in step S110 is No, processor 602 returns to step S102 and continues the learning process. On the other hand, if the determination result in step S110 is Yes, processor 602 ends the flowchart of FIG.

[0089] A trained model is generated by performing the training method shown in the flowchart of Fig. 6. The training method performed using the training device 60 is understood as a method for generating a trained model.

[0090] <<Explanation of the inference phase>> FIG. 7 is a conceptual diagram showing an overview of processing in the inference phase using the trained model 650 of the first embodiment. As described above, the trained model 650 is a model obtained by updating the parameters of the trained model 50 as a result of training. The inference phase is a phase in which liver segments are inferred from newly input image data. Specifically, in the inference phase, a liver segment segment image LSs is generated for the newly input CT image IMs. The liver segment segment image LSs is generated based on a probability map of portal vein branch labels. Here, the probability map is a map similar to the prediction map 52 output by the learning model 50. That is, the probability map is also a score map of portal vein branch labels, and is a map in which a score indicating the likelihood of the portal vein branch label is assigned to each voxel included in the entire image region. The probability map is output from the trained model 650. Therefore, the accuracy of the probability map is improved compared to the prediction map 52. The liver segment image LSs is a segmentation image in which the liver region of the newly input data is divided into eight liver segments. The liver segment image LSs is generated based on the probability map. The trained model 650 generated by the training method of the first embodiment receives input of unknown input data of the same type as the input data used for training, and generates a score of the likelihood of each voxel in the image as a portal vein branch label. Here, the likelihood of a portal vein branch label is synonymous with the reliability of the portal vein branch label. Furthermore, the input data of the same type as the input data used for training is image data of an area including the liver, and includes CT images and multiple types of mask images (see the input data in FIG. 3). Furthermore, the unknown input data refers to new image data not used for training.

[0091] Specifically, Fig. 7 shows an example of input data of the same type as the input data used for learning (see Fig. 4). A combination of four types of images, namely, CT images IMs, liver mask images LMs, portal vein mask images PMs, and vein mask images HMs, is input to the trained model 650. The subscript "s" is attached to new image data not used for learning and image data obtained as a result of inputting the new image data into the trained model 650.

[0092] The liver mask image LMs, portal vein mask image PMs, and vein mask image HMs can be generated by performing liver extraction processing, portal vein extraction processing, and vein extraction processing, respectively, on the CT image IMs. These extraction processes can be performed by processing units similar to the liver extraction processing unit 15, portal vein extraction processing unit 16, and vein extraction processing unit 17 described in FIG. 1. Based on a portal vein branch label probability map, the portal vein branch label assigned to each voxel with the highest score is adopted. That is, multiple portal vein branch labels may be output for each voxel. Furthermore, a score is output for each of the multiple portal vein branch labels. When multiple portal vein branch labels are output, the portal vein branch label with the highest score among the multiple portal vein branch labels is adopted as the portal vein branch label for the voxel. When a voxel has only one portal vein branch label assigned to it, that portal vein branch label is adopted even if it has a low score. This makes it possible to classify all voxels contained in the entire image region of the input image into eight classes, and to generate a map in which each voxel in the entire image region is labeled with a portal vein branch label.

[0093] The trained model 650 performs label conversion, such as converting a portal vein branch label into a liver segment label corresponding to the portal vein branch label, according to the correspondence between the portal vein branch label and the liver segment label. In this way, the liver region can be divided into liver segments based on the portal vein branch label map. Note that label conversion includes the concept of replacing a portal vein branch label with a liver segment label, or treating a portal vein branch label as a liver segment label.

[0094] Then, only the liver region is extracted from the entire image region. This results in a liver segmentation image LSs, which divides the liver region into liver segments. The liver segmentation image LSs is a segmentation image in which the liver region is divided into regions by liver segment labels, or a segmentation image in which the liver region is divided into regions by portal vein branch labels that can be interpreted as liver segment labels.

[0095] The input data combining the CT images IMs, liver mask images LMs, portal vein mask images PMs, and vein mask images HMs is an example of "second input data" in the present disclosure. The portal vein mask images PMs are an example of "second images" in the present disclosure. The entire image region of the portal vein mask images PMs is an example of "second image regions" in the present disclosure.

[0096] In the first embodiment, four types of images are input to the learning model 50 as input data, and a portal vein branch label likelihood score is output for each voxel in the entire image region of the input image (see FIGS. 3 and 4). On the other hand, for example, if only liver mask images LMj are used as input data in the learning stage, the learning model may be designed to learn only the liver region in the image. In this case, the learning model 50 may be configured to calculate a portal vein branch label likelihood score only for voxels in the liver region, and not to calculate a portal vein branch label likelihood score for voxels outside the liver region. The prediction map 52 output from the learning model 50 is sufficient as long as it includes a portal vein branch label likelihood score for each voxel in at least the liver region in the image, and is not required to calculate a score for each voxel in the entire image region.

[0097] 8 is a block diagram showing the configuration of a medical image processing apparatus 70 according to the first embodiment. The medical image processing apparatus 70 includes a processor 702, a computer-readable medium 704 which is a non-transitory tangible entity, a communication interface 706, an input / output interface 708, and a bus 710. An input device 714 and a display device 716 are connected to the bus 710 via the input / output interface 708. Each of these elements corresponds to the processor 702 described in FIG. 6 02. Computer-readable medium 6 04. Communication interface 6 06. Input / output interface 6 08, Bus 6 10. Input device 6 14 and display device 6 16 may be similar to the corresponding elements.

[0098] The medical image processing device 70 may be in the form of a server, a personal computer, a workstation, a tablet terminal, etc. The medical image processing device 70 is connected to a communication line (not shown) via a communication interface 706, and is connected to an external device such as a DICOM server so as to be able to communicate with the external device.

[0099] The computer-readable medium 704 stores multiple programs, data, and the like, including a liver segment segmentation program 720 and a display control program 750. The processor 702 executes the instructions of the liver segment segmentation program 720 to function as the trained model 650 and the label conversion unit 724. The label conversion unit 724 converts portal vein branch labels into liver segment labels. That is, the label conversion unit 724 labels the liver segments based on the portal vein branch labels. The label conversion unit 724 may include a liver extraction processing unit 725 that extracts the liver region in the image and a label deletion processing unit 726 that deletes label information assigned to voxels other than the liver region. The processing algorithm of the liver extraction processing unit 725 may be the same as that of the liver extraction processing unit 15 described in FIG. 1. In this embodiment, label information for regions other than the liver region is deleted to invalidate the labels. However, this is not limited to this; other processing modes are also possible, such as masking or ignoring label information for regions other than the liver region.

[0100] The computer readable medium 704 may further include at least one of an organ recognition program 740 , a disease detection program 742 , and a report creation support program 744 .

[0101] The organ recognition program 740 includes a processing module for performing organ segmentation, and may include a lung segment labeling program, a blood vessel region extraction program, and a bone labeling program.

[0102] The disease detection program 742 includes a detection processing module corresponding to a specific disease. The disease detection program 742 may include, for example, at least one of a pulmonary nodule detection program, a pulmonary nodule characterization program, a pneumonia CAD (Computer Aided Diagnosis, Computer Aided Detection) program, a breast CAD program, a liver CAD program, a brain CAD program, and a colon CAD program.

[0103] The report creation support program 744 includes a trained document generation model that generates candidate findings corresponding to a target medical image.

[0104] Various processing programs such as the organ recognition program 740, the disease detection program 742, and the report creation support program 744 may be AI processing modules that include trained models that have been trained to obtain output for a desired task by applying machine learning such as deep learning.

[0105] An AI model for CAD can be constructed using, for example, various CNNs with convolutional layers. Input data to the AI ​​model includes, for example, medical images such as two-dimensional images, three-dimensional images, or video images, and output from the AI ​​model can be, for example, information indicating the location of a diseased area (lesion site) in the image, information indicating a class classification such as the name of the disease, or a combination of these.

[0106] AI models that handle time-series data, document data, and the like can be constructed using, for example, various recurrent neural networks (RNNs). Time-series data includes, for example, electrocardiogram waveform data. Document data includes, for example, written findings written by doctors.

[0107] The computer-readable medium 704 may further include a program that causes the processor 702 to function as the liver extraction processing unit 15, the portal vein extraction processing unit 16, and the vein extraction processing unit 17 described in Fig. 1. The processing functions of the medical image processing device 70 may be realized by a plurality of computers. Also, some or all of the processing functions of the medical image processing device 70 may be incorporated into the image processing device 10 described in Fig. 1.

[0108] <Example of liver segmentation method> FIG. 9 is a flowchart showing an example of a method for segmenting liver segments using the medical image processing apparatus 70 according to the first embodiment. In step S202, the processor 702 receives input of data including an image to be processed. When the data is input, in step S204, the processor 702 generates a segmentation image of portal vein branch labels. In other words, in step S204, a trained model 65 0 outputs a probability map of portal vein branch labels. As described above, the probability map is a map in which portal vein branch labels and scores are assigned to a predetermined image region. Specifically, the processor 702 uses the trained model 650 to assign a portal vein branch label and a score to each voxel in the entire image region of the input image, or in an image region that includes at least the liver region. If a portal vein branch label and a score are assigned only to each voxel in the liver region, the liver region becomes the predetermined image region described above. The entire image region includes both the portal vein region and regions other than the portal vein region. Furthermore, based on the portal vein branch label and the score, it is determined to which portal vein branch label a voxel corresponding to the predetermined image region belongs, and a portal vein branch label is assigned to each voxel. This results in a segmentation image in which the predetermined image region is classified by portal vein branch label.

[0109] In step S206, the processor 702 performs label conversion processing and divides the liver region into liver segments based on the portal vein branch labels assigned to each voxel.

[0110] In step S208, the processor 702 generates a liver segment segment image LSs. Specifically, the processor 702 performs visualization processing, such as color-coding each segment of the liver segment to clearly indicate the regions, to generate the liver segment segment image LSs. The generated liver segment segment image LSs can be displayed on the display device 716 and a viewer terminal (not shown).

[0111] After step S208, the processor 702 ends the flowchart of FIG.

[0112] Advantages of the First Embodiment According to the medical image processing apparatus 70 of the first embodiment, it is possible to accurately divide the liver region in the CT images IMs into liver segments, regardless of how blood vessels are captured in the CT images IMs.

[0113] <<Variation>> The input data used during learning may take various forms. For example, the input data used during learning may be a combination of three types of masks: a liver mask image LM, a portal vein mask image PM, and a vein mask image HM. Alternatively, the input data may be a combination of two types of masks including at least the liver mask image LM.

[0114] Only the CT image IM (only one type) may be used as input data for learning. In this case, the CT image is the "first image" in the present disclosure. Note that in this case, the "first image region" is also the entire image region of the portal vein mask image. In particular, the "first image region" is the entire image region of the portal vein mask image that is generated from the CT image and has the same image region as the image region of the CT image. Alternatively, only the portal vein mask image PM may be used as input data for learning. In the second embodiment, an example in which only the portal vein mask image PM is used as input data will be described below.

[0115] Second Embodiment Fig. 10 is a conceptual diagram showing an overview of the learning phase in the second embodiment. In Fig. 10, elements that are the same as or similar to elements shown in Fig. 4 and Fig. 5 are given the same reference numerals, and redundant explanations will be omitted.

[0116] 10, in the second embodiment, only one type of image, the portal vein mask image PMj, is used as input data to the learning model 50. Other processing is the same as in the first embodiment.

[0117] 《Preparing training data》 The learning data used in the second embodiment is prepared, for example, as follows.

[0118] First, the image processing device 10 described with reference to FIG. 1 or the like is used to perform extraction processing of the portal vein region on the CT image IMj, and a portal vein mask image PMj is generated as the extraction result.

[0119] 2, the doctor Dr assigns portal vein branch labels to the portal vein regions of the same CT image IMj, thereby labeling each portal vein branch region. As a result, a portal vein branch label map PLMj is generated as training data.

[0120] Then, the portal vein mask image PMj and the portal vein branch label map PLMj are linked to obtain a dataset of the portal vein mask image PMj and the portal vein branch label map PLMj. By performing the same process on a large number of CT images, a sufficient number of data sets are prepared for training.

[0121] <<Explanation of the learning process>> After the learning dataset is prepared, a learning process is performed using the learning device 60 described in FIG. 5 or the like. Specifically, when a portal vein mask image PMj is input to the learning model 50, learning is performed so that a labeling result of a portal vein branch label is output to each voxel of the entire image region, including both the portal vein region and regions other than the portal vein region. As in the first embodiment, in this learning process, regions other than the portal vein region are excluded from the target of loss calculation, and the loss is calculated only for the portal vein region, and the parameters of the learning model 50 are updated based on the calculated loss.

[0122] The portal vein mask image PMj in the second embodiment is an example of the "first input data" and the "first image" in the present disclosure.

[0123] <<Explanation of the inference phase>> Fig. 11 is a block diagram showing an overview of the inference phase using a trained model 650 generated by the training method of the second embodiment. In Fig. 11, elements that are the same as or similar to those shown in Fig. 7 and Fig. 8 are denoted by the same reference numerals, and redundant explanations will be omitted.

[0124] The configuration of the medical image processing apparatus according to the second embodiment may be the same as the configuration of the medical image processing apparatus 70 described with reference to FIG.

[0125] In the second embodiment, the trained model 650 is used, for example, as follows.

[0126] [Step 1] The processor 702 first extracts the portal vein region from CT images IMs including the liver obtained by imaging a patient using a CT device, and generates a portal vein mask image PMs as the extraction result.

[0127] [Step 2] The processor 702 inputs the portal vein mask image PMs into the trained model 650.

[0128] [Step 3] Using the trained model 650, the processor 702 assigns a portal vein branch label to each voxel in the entire region of the input portal vein mask image PMs. Also, in this embodiment, as in the first embodiment, the trained model 650 determines the portal vein branch label with the highest score among the eight classes of portal vein branch labels as the portal vein branch label for that voxel. Specifically, the trained model 650 can assign multiple portal vein branch labels to each voxel included in the entire image region of the portal vein mask image PMs. Furthermore, a score for each of the multiple portal vein branch labels is output. Regardless of how low the score indicating the probability predicted for each voxel is, the trained model 650 determines the portal vein branch label with the highest probability score among the eight classes of portal vein branch labels as the portal vein branch label for that voxel. In this way, portal vein branch labels are assigned to all voxels in the image. The map showing the portal vein branch label labeling results generated by the trained model 650 is called a portal vein branch label segmentation image 652.

[0129] [Step 4] Based on the portal vein branch label segmentation image 652 generated by the trained model 650, the processor 702 labels liver segments corresponding to the portal vein branch labels on the original CT images IMs.

[0130] [Step 5] The processor 702 extracts the liver region from the original CT image IMs. Labels assigned to regions other than the portal vein region are unnecessary. Therefore, the processor 702 deletes the labels assigned to regions other than the portal vein region. As a result, of the portal vein branch labels assigned to all image regions in the portal vein mask image PMs, only the labels assigned to the portal vein region remain.

[0131] [Step 5] Furthermore, the processor 702 performs post-processing such as fine-tuning the inference results as necessary. This post-processing includes, for example, filling in an isolated small region with the label of the surrounding large region, a so-called hole-filling process. A small region may be defined as, for example, an area with a predetermined volume or less. The medical image processing device 70 includes a processing unit that performs fine-tuning of the labeling results.

[0132] In this way, the liver region is divided into eight classes of liver segments, S1 to S8, and a liver segment segmented image LSs is generated based on the output data of the trained model 650. The liver segment segmented image LSs can be a segmentation image in which the liver region is classified by liver segment labels.

[0133] According to the second embodiment, the same effects as those of the first embodiment can be obtained.

[0134] Comparative Example Fig. 12 shows example images illustrating an example of a liver segmentation method using Voronoi division according to a comparative example. The image shown on the left of Fig. 12 is an example of a CT image from which a portal vein region has been extracted. The image shown in the center of Fig. 12 is an example of a vascular labeling diagram showing the portal vein labeled by a user specifying the branching points of the portal vein branches. The image shown on the right of Fig. 12 is an example of an image showing the results of segmenting the liver region using Voronoi division based on vascular labeling.

[0135] FIG. 13 is a diagram showing a comparison between the processing result of liver segment segmentation based on Voronoi division according to a comparative example and the result of appropriate liver segment segmentation.

[0136] The image shown on the left in Figure 13 is an example of an image showing the processing result of liver segmentation based on Voronoi division according to a comparative example, and the image shown on the right in Figure 13 is an example of an image showing the result of correct (correct) liver segmentation. The segmentation result based on Voronoi division does not correctly divide the circled S1 segment. This is because the blood vessels are not completely visible in the CT image, and the S1 portal vein branch cannot be correctly extracted during the blood vessel labeling stage.

[0137] In this regard, by using the trained model 650 generated using the training method described in the first and second embodiments of the present disclosure, liver segmentation can be performed with high accuracy regardless of the state of blood vessels in the image.

[0138] About the programs that run computers A program that causes a computer to realize the processing functions of each of the image processing device 10, information processing device 40, learning device 60, and medical image processing device 70 can be recorded on a computer-readable medium such as an optical disk, magnetic disk, semiconductor memory, or other tangible non-transitory information storage medium, and the program can be provided through this information storage medium.

[0139] In addition, instead of providing the program by storing it on such a tangible, non-transitory computer-readable medium, it is also possible to provide the program signal as a download service using a telecommunications line such as the Internet.

[0140] <<Hardware configuration of each processing unit>> The hardware structure of processing units that perform various processes, such as the liver extraction processing unit 15, portal vein extraction processing unit 16, and vein extraction processing unit 17 in the image processing device 10, the portal vein branch labeling processing unit 46 in the information processing device 40, the data acquisition unit 632, loss calculation unit 634, and optimizer 635 in the learning device 60, and the label conversion unit 724, liver extraction processing unit 725, and label deletion processing unit 726 in the medical image processing device 70, is, for example, various processors as shown below.

[0141] Various types of processors include CPUs, which are general-purpose processors that execute programs and function as various processing units, GPUs, which are processors specialized for image processing, programmable logic devices (PLDs), such as FPGAs (Field Programmable Gate Arrays), which are processors whose circuit configuration can be changed after manufacturing, and dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processors with circuit configurations designed specifically to execute specific processes.

[0142] A single processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types. For example, a single processing unit may be configured with multiple FPGAs, or a combination of a CPU and an FPGA, or a combination of a CPU and a GPU. Alternatively, multiple processing units may be configured with a single processor. A first example of multiple processing units configured with a single processor is a configuration in which one or more CPUs and software are combined to form a single processor, as typified by client or server computers, and this processor functions as multiple processing units. A second example is a configuration in which a processor is used to realize the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip, as typified by a system-on-chip (SoC). In this way, the various processing units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0143] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit made up of a combination of circuit elements such as semiconductor elements.

[0144] About types of medical images The technology of the present disclosure is not limited to CT images, but can also be applied to various medical images captured by various medical devices (modalities). The various medical images include MR images captured using an MRI (Magnetic Resonance Imaging) device, ultrasound images projecting human body information, positron emission tomography (PET) images captured using a PET device, and endoscopic images captured using an endoscope device. Images targeted by the technology of the present disclosure are not limited to three-dimensional images, but may also be two-dimensional images. In a configuration that handles two-dimensional images, the term "voxel" in the content described in each of the above embodiments is replaced with "pixel."

[0145] "others" The above-described embodiments of the present invention may be modified, added, or deleted as appropriate within the scope of the spirit of the present invention. The present invention is not limited to the above-described embodiments, and many modifications may be made by a person skilled in the art within the technical concept of the present invention. [Explanation of symbols]

[0146] 10 Image processing device 12 processors 14 Computer-Readable Medium 15 Liver extraction processing section 16 Portal vein extraction processing unit 17 Vein extraction processing unit 20 Image storage section 30 Learning data storage unit 40 Information processing equipment 42 processors 44 Computer-Readable Medium 46 Portal vein branch labeling processing section 47 Input Devices 48 Display device 50 Learning Models 52 Predictive Maps 60 Learning Device 70 Medical image processing device 602 processor 604 Computer-readable medium 606 Communication Interface 608 Input / Output Interface 610 Bus 614 Input Device 616 Display device 630 Learning Processing Program 632 Data Acquisition Unit 634 Loss calculation section 635 Optimizer 640 Display Control Program 650 pre-trained models 652 Portal vein branch label segmentation images 702 processor 704 Computer-Readable Medium 706 Communication Interface 708 Input / Output Interface 710 Bus 714 Input Device 716 Display device 720 Liver Segmentation Program 724 Label Conversion Unit 725 Liver Extraction Processing Unit 726 Label removal processing unit 740 Organ Awareness Program 742 Disease Detection Program 744 Report Writing Support Program 750 Display Control Program Dr. IM, IMj, IMs CT images HM, HMj, HMs vein mask images PM, PMj, PMs portal vein mask images LM, LMj, LMs liver mask images PLM, PLMj portal vein branch label map LSs liver segmentation image S102~S110 Learning method processing steps S202~S208: liver segment division method processing steps

Claims

1. A medical image processing device, a processor; a storage device that stores a program executed by the processor; Equipped with The program a trained model generated by performing machine learning using training data including first input data including a first image of the liver and portal vein labeling data in which a portal vein label is assigned to each portal vein branch corresponding to a liver segment for a portal vein region in the liver in the first image, the trained model is a model in which parameters of a training model trained to accept input of the first input data and output a labeling result of the portal vein branch label for each image unit element in a first image region of the first image have been updated; The processor: By executing the instructions of the program, accepting second input data of the same type as the first input data, the second input data including a second image relating to a liver; assigning the portal vein branch label to each image unit element in a second image region of the second image using the trained model; Dividing the liver region included in the second input data into a plurality of liver segments based on the portal vein branch labels assigned to each image unit element of the second image region. Medical imaging equipment.

2. the first input data includes at least one of a CT (Computed Tomography) image of a region including the liver and a portal vein mask image in which a portal vein region is identified; the first image is the CT image or the portal vein mask image; The medical image processing apparatus of claim 1 .

3. the first input data includes the CT image and the portal vein mask image; The medical image processing device according to claim 2 .

4. the first input data further includes at least one of a liver mask image in which a liver region is identified, a vein mask image in which a vein region is identified, and an inferior vena cava mask image in which an inferior vena cava region is identified; The medical image processing device according to claim 2 .

5. the first input data includes the portal vein mask image, the liver mask image, and the vein mask image; The medical image processing apparatus according to claim 4 .

6. the first image area is the entire area of ​​the first image, the second image area is the entire area of ​​the second image; The medical image processing apparatus of claim 1 .

7. The portal vein branch labels are labels that classify the portal vein branches into eight classes corresponding to the eight types of liver segments S1 to S8. The medical image processing apparatus of claim 1 .

8. The trained model is constructed using a convolutional neural network. The medical image processing apparatus of claim 1 .

9. The machine learning process for generating the trained model includes: calculating a loss for a score map indicating the likelihood of the portal vein branch label output from the learning model only for a portal vein region to which the portal vein branch label is assigned in the portal vein branch labeling data corresponding to the first input data, and updating parameters of the learning model based on the calculated loss; The medical image processing apparatus of claim 1 .

10. each of the first image and the second image is a three-dimensional image; The medical image processing apparatus of claim 1 .

11. The processor: performing labeling of liver segments with labels indicating the liver segments based on the portal vein branch labels assigned to each image unit element of the second image region; The medical image processing apparatus of claim 1 .

12. the second input data includes a CT image of a region including the liver; The processor: extracting a liver region from the CT image included in the second input data; invalidating label information labeled with respect to a region other than the extracted liver region in the second image region; The medical imaging device of claim 11 .

13. The processor: converting the portal vein branch labels assigned to each image unit element of the second image region into the liver segment labels, thereby generating a liver segment segmented image divided into the liver segments; The medical imaging device of claim 11 .

14. A liver segmentation method in which a computer divides a liver region in an image into liver segments, comprising: generating a learning model by performing machine learning using learning data including first input data including a first image of the liver and portal vein labeling data in which a portal vein label is assigned to each portal vein branch corresponding to the liver segment for a portal vein region in the liver in the first image; updating parameters of the learning model based on the labeling result of the portal vein branch label output by the learning model for each image unit element of a first image region of the first image, to generate a trained model; receiving second input data of the same type as the first input data, the second input data including a second image relating to a liver; assigning the portal vein branch label to each image unit element in a second image region of the second image using the trained model; Dividing the liver region included in the second input data into a plurality of liver segments based on the portal vein branch labels assigned to each image unit element of the second image region; A method for dividing liver segments, comprising:

15. A program for causing a computer to operate as a medical image processing device, a trained model generated by performing machine learning using training data including first input data including a first image of the liver and portal vein labeling data in which a portal vein label is assigned to each portal vein branch corresponding to a liver segment for a portal vein region in the liver in the first image, the trained model is a model in which parameters of a training model trained to accept input of the first input data and output a labeling result of the portal vein branch label for each image unit element in a first image region of the first image have been updated; The computer, receiving second input data of the same type as the first input data, the second input data including a second image relating to a liver; assigning the portal vein branch label to each image unit element in a second image region of the second image using the trained model; Dividing the liver region included in the second input data into a plurality of liver segments based on the portal vein branch labels assigned to each image unit element of the second image region; A program to make this happen.

Images