Medical image processing apparatus, method, and program

Abstract

To enable efficient and highly accurate segmentation. [Solution] The medical image processing apparatus according to the embodiment comprises a first acquisition unit, a second acquisition unit, a setting unit, and a processing unit. The first acquisition unit acquires information about the structure to be segmented. The second acquisition unit acquires information about the likelihood of the structure's presence in the medical image based on the information about the structure. The setting unit sets a segmentation region for performing structure segmentation on a part of the medical image based on the information about the likelihood of the structure's presence. The processing unit performs structure segmentation on the segmentation region of the medical image.

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to a medical image processing apparatus, method, and program.

Background Art

[0002] Conventionally, when analyzing diseases or the like using medical images, segmentation for detecting a target structure from the medical images has been performed. For example, segmentation for detecting a target structure from medical images is performed by a method based on a machine learning technique including deep learning, a method based on a known image processing technique such as Otsu's binarization method based on CT values, or the like. Such segmentation for detecting a target structure from medical images is important for measuring the volume of the structure and for performing detailed analysis in a subsequent stage.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to enable efficient performance of highly accurate segmentation. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problems. It is also possible to position, as other problems, the problems corresponding to the respective effects of each configuration shown in the embodiments described later.”

Means for Solving the Problems

[0005] The medical image processing apparatus according to the embodiment comprises a first acquisition unit, a second acquisition unit, a setting unit, and a processing unit. The first acquisition unit acquires information about a structure to be segmented. The second acquisition unit acquires information about the likelihood of the structure's presence in the medical image based on the information about the structure's presence. The setting unit sets a segmentation region for performing segmentation of the structure in a portion of the medical image based on the information about the likelihood of the structure's presence. The processing unit performs segmentation of the structure in the segmentation region of the medical image. [Brief explanation of the drawing]

[0006] [Figure 1] Figure 1 shows an example of the configuration of a medical image processing apparatus according to the first embodiment. [Figure 2] Figure 2 is a flowchart showing the processing procedure by the medical image processing apparatus according to the first embodiment. [Figure 3A] Figure 3A is a diagram illustrating an example of processing by the control function according to the first embodiment. [Figure 3B] Figure 3B is a diagram illustrating an example of processing by the control function according to the first embodiment. [Figure 4] Figure 4 shows an example of the segmentation results according to the first embodiment. [Figure 5A] Figure 5A is a diagram illustrating an example of processing by a medical image processing apparatus according to the first embodiment. [Figure 5B] Figure 5B is a diagram illustrating an example of processing by a medical image processing apparatus according to the first embodiment. [Figure 6A] Figure 6A shows an example of setting up a segmentation region using a probability map according to the first embodiment. [Figure 6B] Figure 6B shows an example of setting up a segmentation region using a probability map according to the first embodiment. [Figure 6C]Figure 6C shows an example of setting up a segmentation region using a probability map according to the first embodiment. [Figure 7] Figure 7 shows an example of a display according to the first embodiment. [Figure 8] Figure 8 shows an example of how the segmentation results according to the first embodiment are displayed. [Modes for carrying out the invention]

[0007] The embodiments of the medical image processing apparatus, method, and program will be described in detail below with reference to the drawings. However, the medical image processing apparatus, method, and program according to this application are not limited to the embodiments shown below.

[0008] (First Embodiment) Figure 1 shows an example configuration of a medical image processing device according to the first embodiment. For example, as shown in Figure 1, the medical image processing device 3 according to this embodiment is connected to the medical image diagnostic device 1 and the medical image storage device 2 via a network. Note that various other devices and systems may be connected to the network shown in Figure 1.

[0009] Medical imaging device 1 captures images of a subject and generates medical images. Then, medical imaging device 1 transmits the generated medical images to various devices on the network. For example, medical imaging device 1 may be an X-ray diagnostic device, an X-ray CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, an ultrasound diagnostic device, a SPECT (Single Photon Emission Computed Tomography) device, a PET (Positron Emission Computed Tomography) device, etc.

[0010] The medical image storage device 2 stores various medical images related to the subject. Specifically, the medical image storage device 2 receives medical images from the medical image diagnostic device 1 via a network and stores these medical images in its internal memory circuit. For example, the medical image storage device 2 can be implemented using computer equipment such as a server or workstation. Alternatively, for example, the medical image storage device 2 can be implemented using a PACS (Picture Archiving and Communication System) and store medical images in a format compliant with DICOM (Digital Imaging and Communications in Medicine).

[0011] The medical image processing device 3 performs various information processing on medical images collected from a subject. Specifically, the medical image processing device 3 receives medical images from the medical image diagnostic device 1 or the medical image storage device 2 via a network and performs various information processing using those medical images. For example, the medical image processing device 3 is implemented using computer equipment such as a server or workstation.

[0012] For example, the medical image processing device 3 includes a communication interface 31, an input interface 32, a display 33, a storage circuit 34, and a processing circuit 35.

[0013] The communication interface 31 controls the transmission and communication of various data between the medical image processing device 3 and other devices connected via the network. Specifically, the communication interface 31 is connected to the processing circuit 35 and transmits data received from other devices to the processing circuit 35, or transmits data received from the processing circuit 35 to other devices. For example, the communication interface 31 can be implemented by a network card, network adapter, NIC (Network Interface Controller), etc.

[0014] The input interface 32 receives input operations of various instructions and various information from the user. Specifically, the input interface 32 is connected to the processing circuit 35, converts the input operations received from the user into electrical signals, and transmits them to the processing circuit 35. For example, the input interface 32 is realized by a trackball, a switch button, a mouse, a keyboard, a touch pad that performs an input operation by touching an operation surface, a touch screen in which a display screen and a touch pad are integrated, a non-contact input interface using an optical sensor, a voice input interface, and the like. In the present specification, the input interface 32 is not limited to only those having physical operation components such as a mouse and a keyboard. For example, a processing circuit for electrical signals that receives an electrical signal corresponding to an input operation from an external input device provided separately from the apparatus and transmits this electrical signal to the control circuit is also included in the example of the input interface 32.

[0015] The display 33 displays various information and various data. Specifically, the display 33 is connected to the processing circuit 35 and displays various information and various data received from the processing circuit 35. For example, the display 33 is realized by a liquid crystal display, a CRT (Cathode Ray Tube) display, a touch panel, an LED (Light Emitting Diode) display, or the like.

[0016] The storage circuit 34 stores various data and various programs. Specifically, the storage circuit 34 is connected to the processing circuit 35, stores the data received from the processing circuit 35, or reads out the stored data and transmits it to the processing circuit 35. For example, the storage circuit 34 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, a hard disk, an optical disk, or the like.

[0017] The processing circuit 35 controls the entire medical image processing device 3. For example, the processing circuit 35 performs various processes in response to input operations received from the user via the input interface 32. For example, the processing circuit 35 receives data transmitted by other devices via the communication interface 31 and stores the received data in the storage circuit 34. Also, for example, the processing circuit 35 transmits the data received from the storage circuit 34 to other devices by sending it to the communication interface 31. Also, for example, the processing circuit 35 displays the data received from the storage circuit 34 on the display 33.

[0018] The configuration example of the medical image processing device 3 according to this embodiment has been described above. For example, the medical image processing device 3 according to this embodiment is installed in medical facilities such as hospitals and clinics and supports various diagnoses and the formulation of treatment plans performed by users such as doctors. Specifically, in the processing of segmentation of medical images, the medical image processing device 3 performs local segmentation based on information about structures desired by the user.

[0019] As mentioned above, machine learning-based methods and image processing-based methods are applied to segmentation of medical images, but these methods may not be able to efficiently perform highly accurate segmentation. For example, when performing segmentation for multiple different structures using deep learning, a huge amount of computational resources are required for training, and the inference during segmentation is processed separately for each structure, which takes time. Also, when performing segmentation of multiple structures using multi-class inference with deep learning, it may not be possible to extract independent regions within the same class (for example, fractured regions). Furthermore, when performing segmentation based on the pixel values ​​of medical images, it may not be possible to classify each structure in a series of structures with similar pixel values.

[0020] Therefore, in this embodiment, by performing local segmentation based on information about structures desired by the user, it becomes possible to efficiently perform highly accurate segmentation. Below, a medical image processing device 3 having such a configuration will be described in detail.

[0021] For example, as shown in Figure 1, in this embodiment, the processing circuit 35 of the medical image processing device 3 performs a control function 351, an acquisition function 352, a setting function 353, and a processing function 354. Here, the control function 351 is an example of a first acquisition unit and display control unit. The acquisition function 352 is an example of a second acquisition unit. The setting function 353 is an example of a setting unit. The processing function 354 is an example of a processing unit.

[0022] The control function 351 generates various GUIs (Graphical User Interfaces) and various display information in response to operations via the input interface 32 and controls them to be displayed on the display 33. For example, the control function 351 displays the results of processing by each function on the display 33. The control function 351 also acquires medical images of the subject from the medical image diagnostic device 1 or the medical image storage device 2 via the communication interface 31. Specifically, the control function 351 acquires medical images that include three-dimensional or two-dimensional morphological information. The control function 351 acquires CT images, ultrasound images, MRI images, X-ray images, etc., as the above-mentioned medical images.

[0023] Furthermore, the control function 351 acquires information about the structures to be segmented, but this process will be described in detail later.

[0024] The acquisition function 352 acquires information regarding the likelihood of the presence of structures in medical images, based on information about the structures. The processing performed by the acquisition function 352 will be described in detail later.

[0025] The setting function 353 sets a segmentation region for performing structural segmentation on a portion of the medical image, based on information regarding the likelihood of the existence of the structure. The processing performed by the setting function 353 will be described in detail later.

[0026] The processing function 354 performs segmentation of the structure on the segmentation region of the medical image. The processing performed by the processing function 354 will be described in detail later.

[0027] The processing circuit 35 described above is implemented, for example, by a processor. In this case, each of the processing functions described above is stored in the memory circuit 34 in the form of a program that can be executed by a computer. The processing circuit 35 then reads and executes each program stored in the memory circuit 34, thereby realizing the function corresponding to each program. In other words, the processing circuit 35, with each program read, has the processing functions shown in Figure 1.

[0028] Next, the processing procedure by the medical image processing device 3 will be explained using Figure 2, and then the details of each process will be described. Figure 2 is a flowchart showing the processing procedure by the medical image processing device 3 according to the first embodiment.

[0029] For example, as shown in Figure 2, in this embodiment, the control function 351 acquires a medical image of the subject from the medical image diagnostic device 1 or the medical image storage device 2 (step S101) and acquires information about the structure to be segmented (step S102). This process is realized, for example, by the processing circuit 35 calling and executing a program corresponding to the control function 351 from the storage circuit 34.

[0030] Next, the acquisition function 352 acquires information regarding the likelihood of existence of the target structure for segmentation in the medical image (step S103). This process is realized, for example, by the processing circuit 35 calling and executing a program corresponding to the acquisition function 352 from the storage circuit 34.

[0031] Next, the setting function 353 sets a segmentation region for a portion of the medical image (step S104). This process is achieved, for example, by the processing circuit 35 calling and executing a program corresponding to the setting function 353 from the storage circuit 34.

[0032] Next, the processing function 354 performs segmentation (step S105). This process is achieved, for example, by the processing circuit 35 calling and executing a program corresponding to the processing function 354 from the storage circuit 34.

[0033] Next, the acquisition function 352 determines whether or not the target structure is located outside the segmentation area (step S106). This process is achieved, for example, by the processing circuit 35 calling and executing a program corresponding to the acquisition function 352 from the storage circuit 34.

[0034] If the target structure is located outside the area (step S106, Yes), the acquisition function 352 returns to step S103 and executes the process.

[0035] On the other hand, if there are no target structures outside the area (step S106, No), the control function 351 displays the segmentation result (step S107). This process is achieved, for example, by the processing circuit 35 calling and executing a program corresponding to the control function 351 from the storage circuit 34.

[0036] The details of each process performed by the medical image processing device 3 are described below.

[0037] (Medical image acquisition process) As explained in step S101 of Figure 2, the control function 351 acquires a medical image containing three-dimensional or two-dimensional morphological information in response to a medical image acquisition operation via the input interface 32. For example, the control function 351 acquires a three-dimensional CT image (volume data). Note that the medical image acquired here is not limited to a three-dimensional CT image; any medical image that is subject to segmentation may be acquired. Furthermore, the medical image in this embodiment includes raw data collected by the medical image diagnostic device 1, image data after reconstruction of the raw data, and images on which various image processing has been performed.

[0038] (Process for acquiring information about the target structure) As explained in step S102 of Figure 2, the control function 351 acquires information about the structures to be segmented from the acquired medical image. For example, the control function 351 acquires the structures to be segmented based on the operator's specified operation on the medical image. In such cases, the control function 351 generates a display image from the acquired medical image and displays the generated display image on the display 33.

[0039] The control function 351 acquires the structure corresponding to the specified position on the display image shown on the display 33 as the structure to be segmented. An example of processing by the control function 351 will be described below using Figures 3A and 3B. Figures 3A and 3B are diagrams illustrating an example of processing by the control function 351 according to the first embodiment.

[0040] For example, as shown in Figure 3A, the control function 351 generates a display image from the acquired three-dimensional CT image (volume data) and displays it on the display 33, and also displays a cursor C1 for specifying the structure to be segmented. Here, as shown in Figure 3A, the cursor C1 is accompanied by a segmentation region ROI 1 that follows the movement of the cursor C1. That is, when the operator moves the cursor C1 by operating the input interface 32, the segmentation region ROI 1 also moves in accordance with it.

[0041] The operator operates the input interface 32 to move the cursor C1 to the location of the structure to be segmented and performs an operation to determine the target structure (for example, a mouse click). The control function 351 receives the operations performed by the operator as described above and acquires the structure to be segmented. Furthermore, the processing function 354 acquires the segmentation region ROI 1 placed on the image by the operations performed by the operator as the area to be segmented. Note that although the area to be segmented is shown on a 2D display image in Figure 3A, segmentation is performed in 3D.

[0042] In other words, if segmentation region ROI1 is set by the operator, segmentation region ROI1 will be set as a segmentation region in step S104, regardless of the existence likelihood information obtained in step S103.

[0043] Here, the segmentation region ROI1 associated with cursor C1 can be arbitrarily changed in size, shape, orientation, etc., by the operator. For example, the control function 351 accepts mouse wheel operation by the operator and changes the size of the segmentation region ROI1 as shown in Figure 3B. The operator can set a region to be segmented within a medical image by changing the size of the segmentation region ROI1 to include the structure to be segmented, but the larger the segmentation region, the greater the load on the segmentation processing.

[0044] Therefore, in this embodiment, the segmentation region is composed of multiple sub-regions, and the sub-regions are arranged based on the likelihood of existence of the structure to be segmented, thereby controlling the segmentation region to not become excessively large.

[0045] In such cases, for example, as shown in Figure 3A, the segmentation region ROI1 is set to a size that includes a part of the structure to be segmented (liver), and is used in the structure specification operation. When the segmentation region ROI1 is set to the size shown in Figure 3A, the segmentation region ROI1 becomes a small region that constitutes a part of the entire segmentation region.

[0046] Furthermore, if the operator sets the segmentation region ROI1 at the position shown in Figure 3A, the region R1 shown in Figure 4 will be extracted during the segmentation process in step S105. Figure 4 is a diagram showing an example of the segmentation result according to the first embodiment.

[0047] (Processing to obtain information on likelihood of existence) As explained in step S103 of Figure 2, the acquisition function 352 acquires information regarding the likelihood of the structure's existence. Specifically, the acquisition function 352 acquires at least one of the following as information regarding the likelihood of the structure's existence: the structure's continuity, probability map, and anatomical feature points.

[0048] For example, when acquiring the continuity of a structure as information regarding the likelihood of existence, the acquisition function 352 acquires the continuity with the structure specified by the operator (indicated by cursor C1) or with the region extracted by the segmentation process. To give one example, the acquisition function 352 acquires the pixel values ​​of the specified structure (or the region extracted by the segmentation process) and the surrounding pixel values. In other words, the acquisition function 352 acquires pixel values ​​as information to determine whether or not the structures are connected (continuous or not) inside and outside the segmentation region.

[0049] Furthermore, for example, when acquiring a probability map as information regarding the likelihood of existence, the acquisition function 352 acquires the probability map output in deep learning-based segmentation. To give one example, the acquisition function 352 acquires the probability map output in the segmentation in step S105. Here, the probability map shows the class probability for each pixel. That is, the probability map of the structure targeted for segmentation is information indicating whether or not each pixel in the medical image is the target structure.

[0050] For example, the acquisition function 352 acquires a probability map represented by continuous values ​​in the range [0,1]. In addition to acquiring the probability map output in the process of step S105, the acquisition function 352 can also acquire a probability map by inputting the medical image acquired by the control function 351 into a trained model formed by deep learning.

[0051] Here, the acquisition function 352 can acquire at least one of the following as probability maps: the probability map of structures and the probability map of structures other than structures. In other words, the acquisition function 352 can acquire the probabilities of structures targeted for segmentation and the probabilities of structures other than those targeted for segmentation.

[0052] Furthermore, for example, when acquiring anatomical feature points as information regarding likelihood of existence, the acquisition function 352 detects anatomical feature points of the human body by performing image processing such as pattern recognition on the medical image acquired by the control function 351. The acquisition function 352 associates an identifier for uniquely identifying the anatomical feature point with each pixel corresponding to the anatomical feature point in the medical image and stores it. This makes it possible to estimate the positional relationships of organs within the medical image.

[0053] (Segmentation setup process) As explained in step S104 of Figure 2, the setting function 353 sets the segmentation region based on information regarding the likelihood of existence. Specifically, the setting function 353 sets the segmentation region for areas where the structure to be segmented is likely to exist. For example, as shown in Figure 4, if the operator sets a sub-region of the initial segmentation region and performs segmentation, and region R1 is extracted, the setting function 353 sets the segmentation region for areas that are likely to be connected to region R1. Here, the setting function 353 estimates areas that are likely to be connected to region R1 based on the information regarding the likelihood of existence of the structure obtained by the acquisition function 352.

[0054] For example, if the continuity of a structure is obtained as information regarding the likelihood of existence, the setting function 353 sets a segmentation region for the region adjacent to region R1. To give one example, the setting function 353 compares the pixel values ​​of region R1 with the surrounding pixel values ​​and sets a segmentation region in the direction of pixels that have pixel values ​​similar to the pixel values ​​of region R1.

[0055] Furthermore, if, for example, a probability map is obtained as information regarding the likelihood of existence, the setting function 353 sets the segmentation region in the direction that increases the probability of being in region R1.

[0056] Furthermore, if, for example, anatomical feature points are obtained as information regarding the likelihood of existence, the setting function 353 estimates the arrangement of organs corresponding to region R1 based on the anatomical feature points. Then, based on the estimated arrangement information, the setting function 353 sets segmentation regions for areas in the medical image where organs corresponding to region R1 are likely to exist.

[0057] Here, the setting function 353 can set the segmentation region using the classification results obtained in the segmentation process described later and the anatomical feature points. For example, the setting function 353 also sets the segmentation region based on the proximity of the result classified as liver to the anatomical feature points of the liver.

[0058] (Segmentation process) As explained in step S105 of Figure 2, the processing function 354 performs segmentation on the segmentation region set by the setting function 353. Specifically, the processing function 354 extracts the segmentation region from the medical image and performs segmentation by inputting the extracted segmentation region into a trained model constructed by deep learning.

[0059] Here, the pre-trained model described above is constructed to output multi-class classification results and segmentation results. For example, the pre-trained model described above may be constructed using a Mask R-CNN (Region-Convolutional Neural Network) that simultaneously performs classification of structure class names and pixel-level classification within an image. Note that the above example is merely one example, and the pre-trained model may be constructed using other methods. Furthermore, it is also possible to construct and use separate pre-trained models for classifying structure class names and for extracting regions at the pixel level.

[0060] (Structure detection process) As explained in step S106 of Figure 2, the acquisition function 352 determines whether or not the target structure is located outside the segmentation region. Specifically, the acquisition function 352 determines whether or not the target structure is connected outside the segmentation region based on information regarding the likelihood of existence. More specifically, the acquisition function 352 determines the connections between subregions in multiple subregions based on at least one of the following: the continuity of the structure, the distribution of the probability map of the structure, and anatomical feature points.

[0061] For example, if the continuity of a structure is obtained as information regarding the likelihood of existence, the acquisition function 352 determines that the target structure exists outside the segmentation region if the similarity between the pixel value of a pixel outside the region closest to the region where segmentation was performed and the pixel value of a pixel included in the region extracted by segmentation is greater than or equal to a threshold. On the other hand, if the similarity between the pixel value of a pixel outside the region and the pixel value of a pixel included in the region extracted by segmentation is less than a threshold, the acquisition function 352 determines that the target structure does not exist outside the segmentation region.

[0062] Furthermore, for example, if a probability map is obtained as information regarding the likelihood of existence, the acquisition function 352 determines that the target structure exists outside the segmentation region if the probability map value of the outermost pixel within the segmentation region is greater than or equal to a threshold. On the other hand, if the probability map value of the outermost pixel within the segmentation region is less than a threshold, the acquisition function 352 determines that the target structure does not exist outside the segmentation region.

[0063] Furthermore, for example, if anatomical feature points are obtained as information regarding the likelihood of existence, the acquisition function 352 determines whether or not the target structure is located outside the segmentation region based on the arrangement of the structure estimated from the anatomical feature points.

[0064] The acquisition function 352 performs the above-described determination process on the entire perimeter of the segmentation region. For example, if the segmentation region is a rectangle as shown in Figure 4, the acquisition function 352 performs the above-described determination process on each side of the segmentation region.

[0065] (Displaying segmentation results) As explained in step S107 of Figure 2, the control function 351 displays the segmentation processing results from the processing function 354 on the display 33. Specifically, the control function 351 displays the structures extracted by the processing shown in Figure 2 on the display 33. For example, the control function 351 highlights the extracted structures in the display image generated from the medical image.

[0066] Below, an example of processing by the medical image processing device 3 will be explained using Figures 5A and 5B. Figures 5A and 5B are diagrams illustrating an example of processing by the medical image processing device 3 according to the first embodiment. Here, Figures 5A and 5B show the processing after the segmentation processing shown in Figure 4. That is, Figures 5A and 5B show the processing after the operator has set the segmentation region ROI1 (a small region that constitutes a part of the entire segmentation region) and extracted region R1 from the segmentation region ROI1.

[0067] As shown in Figure 5A, once region R1 is extracted by the processing function 354, the acquisition function 352 determines whether or not there is a structure corresponding to region R1 outside the segmentation region ROI1. That is, the acquisition function 352 performs the process in step S106 of Figure 2. Here, the acquisition function 352 determines that there is a structure above the segmentation region ROI1 in the figure and acquires information regarding the likelihood of the existence of a structure in the region above the segmentation region ROI1.

[0068] The setting function 353 sets the segmentation region ROI2 shown in Figure 5A based on the likelihood of existence information obtained by the acquisition function 352. Here, the medical image processing device 3 can automatically perform segmentation targeting the segmentation region ROI2 set by the setting function 353, but as shown in Figure 5A, it can also receive instructions from the operator on whether or not to approve the segmentation region ROI2 as a segmentation region.

[0069] For example, as shown in the left diagram of Figure 5A, the control function 351 displays the segmentation region on the medical image and also displays a GUI for receiving instructions from the operator. Then, as shown in the right diagram of Figure 5A, when the segmentation region ROI2 is approved, the processing function 354 performs segmentation processing on the segmentation region ROI2 to extract the target structure within the segmentation region ROI2. As a result, as shown in the right diagram of Figure 5A, the region R1 corresponding to the target structure is further extracted.

[0070] As shown in the right-hand diagram of Figure 5A, once region R1 is further extracted, the acquisition function 352 determines again whether or not the target structure is outside the segmentation region. That is, the acquisition function 352 determines whether or not the structure corresponding to region R1 is outside the segmentation region ROI2. In this way, the medical image processing device 3 uses small regions to repeatedly perform the determination of the presence or absence of the target structure and the segmentation process, thereby suppressing the expansion of the segmentation target region and enabling efficient and highly accurate segmentation.

[0071] Figure 5A shows a case where the operator is asked to approve or reject the setting of a sub-region each time a sub-region is set. However, the embodiment is not limited to this, and the operator's approval may be sought after setting multiple sub-regions to extract the entire target structure.

[0072] For example, after region R1 is extracted by the processing function 354, the medical image processing device 3 sequentially performs the following steps, as shown in Figure 5B: setting up segmentation region ROI2 and performing segmentation processing within ROI2; setting up segmentation region ROI3 and performing segmentation processing within ROI3; and setting up segmentation region ROI4 and performing segmentation processing within ROI4. Subsequently, the medical image processing device 3 can be controlled to accept confirmation from the operator whether or not to approve these processes.

[0073] The processes described in Figures 5A and 5B are merely examples, and the medical image processing device 3 can perform various other processes. For example, an example in which a probability map is acquired as information regarding the likelihood of existence, and a segmentation region is set using the values ​​in the probability map, will be explained using Figures 6A to 6C. Figures 6A to 6C show an example of setting a segmentation region using a probability map according to the first embodiment.

[0074] For example, as shown in Figure 6A, when segmentation is performed in segmentation region ROI1, the setting function 353 can set segmentation region ROI2, segmentation region ROI3, etc., based on the gradient of the probability map M1 within segmentation region ROI1. In other words, the setting function 353 can set new segmentation regions (new sub-regions) in the direction of increasing probability in the gradient of the probability map M1 within segmentation region ROI1.

[0075] Furthermore, for example, if segmentation is performed in segmentation region ROI1 and the average value of the probability map M1 within segmentation region ROI1 is low, the setting function 353 can expand and set segmentation region ROI1, as shown in Figure 6B. That is, if the average value of the probability map M1 is low, the setting function 353 can determine that the initial segmentation region ROI1 is close to the end of the structure and expand and set segmentation region ROI1 again.

[0076] Here, when segmentation is performed in the segmentation region ROI1 and a probability map M1 within the segmentation region ROI1 is obtained, the setting function 353 can set an enlarged segmentation region ROI1 based on the gradient of the probability map M1, and at this time, it is also possible to change the orientation of the segmentation region ROI1. For example, as shown in Figure 6C, the setting function 353 can set an enlarged segmentation region ROI1 that has been rotated so that it covers the target structure with a single segmentation region.

[0077] Here, the setting function 353 can also use information such as anatomical feature points. That is, the setting function 353 can set the orientation and size of the segmentation region ROI1 using information on anatomical feature points in addition to the probability map. The example described above is merely one example, and the setting function 353 can flexibly set the segmentation region using information on likelihood of existence. For example, the setting function 353 can set the shape, size, and orientation of the segmentation region based on at least one of the following: the continuity of the structure, the distribution of the probability map of the structure, and anatomical feature points.

[0078] The above-described embodiment explains the case in which a segmentation region is set using a probability map of the structure to be segmented. However, the embodiment is not limited to this, and a probability map of a structure other than the target of segmentation may also be used when setting the segmentation region.

[0079] For example, when setting segmentation regions ROI2 to ROI4 as shown in the left diagram of Figure 5B, the setting function 353 also refers to the probability map of structures other than the structure (organ) corresponding to region R1 when setting the segmentation regions. In this case, in segmentation region ROI4 in the left diagram of Figure 5B, the probability of structures other than the structure (organ) corresponding to region R1 is high. Therefore, the setting function 353 sets only segmentation regions ROI2 and ROI3 and does not set segmentation region ROI4.

[0080] Furthermore, the above-described embodiment explains that segmentation regions can be set using multiple pieces of information regarding likelihood of existence. For example, segmentation regions can be set using anatomical feature points and probability maps, but discrepancies may occur between the information on anatomical feature points and the information on probability maps. For example, in a region that includes anatomical feature points of the liver, the liver may not be inferred as a classification result. In such cases, the processing function 354 lowers the threshold of the probability map and performs segmentation again. The setting function 353 sets the next segmentation region using the probability map obtained from the segmentation performed again.

[0081] Furthermore, in the embodiments described above, a case was explained in which it is determined whether or not there is a structure outside the segmentation region based on information regarding the likelihood of existence, and the next segmentation region is set based on the determination result. However, the embodiments are not limited to this, and the segmentation region may also be set based on the shape of the area of ​​the structure at the boundary of the segmentation region.

[0082] For example, in the left diagram of Figure 5A, at the upper boundary of segmentation region ROI1, the shape of region R1 closely matches the shape of segmentation region ROI1. That is, the organ corresponding to region R1 appears to be cut off by segmentation region ROI1, and it is highly likely that the organ does not fit within segmentation region ROI1 in that direction. Therefore, the setting function 353 may calculate the degree of agreement between the shape of the structure at the boundary of the segmentation region and the shape of the segmentation region, and set the segmentation region if the degree of agreement exceeds a threshold.

[0083] Furthermore, the above-described embodiment explained the case in which a cursor with an attached segmentation region is used when setting the initial segmentation region. However, the embodiment is not limited to this, and the initial segmentation region may be set by other methods.

[0084] For example, object detection may be applied to the initial setting of the segmentation region. In such cases, the operator uses a cursor that does not have a segmentation region attached to it to specify the structure to be segmented. The setting function 353 replaces the position specified by the cursor (e.g., a click point) with an anchor that classifies the foreground and background in object detection, and outputs a BBox (Bounding Box) from the click point to set it as the segmentation region.

[0085] Furthermore, for example, the operator's specification of location may not be used when setting the initial segmentation area. In other words, the information of the structures to be segmented may be obtained from other information rather than through operator specification. In such cases, the control function 351 obtains the structures to be segmented based on the input information of the structures.

[0086] For example, the control function 351 acquires information about the structures to be segmented based on scan conditions for collecting medical images and information entered by the operator (e.g., organ names). The setting function 353 sets the segmentation region for the medical image based on the segmentation target information acquired by the control function 351.

[0087] Here, the memory circuit 34 pre-stores correspondence information, which associates the setting position of the segmentation region with each organ. For example, the correspondence information is information in which the setting position for each organ is set using position information represented by the positions of anatomical feature points. The setting function 353 obtains the setting position of the first segmentation by obtaining the correspondence information of the segmentation target acquired by the control function 351. Furthermore, the setting function 353 detects anatomical feature points in the medical image by performing image processing on the medical image, and sets the segmentation region from the detected anatomical feature points and the setting position of the first segmentation.

[0088] Furthermore, although the above-described embodiment described the case of performing segmentation of a single structure (liver), the embodiment is not limited to this, and it may also be possible to perform segmentation to extract multiple structures. For example, when there are multiple structures such as ribs and vertebrae, one or more structures may be extracted depending on the operator's actions.

[0089] For example, if the operator sets the initial segmentation region to include multiple vertebral bodies, the segmentation may be performed to extract all vertebrae. On the other hand, if the operator sets the initial segmentation region to include only one vertebral body, the segmentation may be performed to extract only the specified vertebrae.

[0090] Furthermore, although we have described a case in which a segmentation region is set on a two-dimensional display image, the embodiments are not limited to this, and the segmentation region can also be displayed in three dimensions. Figure 7 is a diagram showing an example of a display according to the first embodiment. In Figure 7, the left side shows an MPR image of three orthogonal cross-sections, and the right side shows a three-dimensional image.

[0091] For example, as shown in Figure 7, the control function 351 can display the segmentation region in three dimensions by tilting the image in which the segmentation region is set in the MPR image of three orthogonal cross-sections. Furthermore, as shown in Figure 7, the control function 351 can also display the segmentation region in three dimensions by placing the segmentation region on the three-dimensional image. By displaying the segmentation region in three dimensions, the extracted region R1 can also be observed in three dimensions.

[0092] Furthermore, in the embodiments described above, an example was given in which the extracted structures are highlighted when displaying the segmentation results. However, the embodiments are not limited to this, and other information may also be displayed. Figure 8 is a diagram showing an example of the display of segmentation results according to the first embodiment.

[0093] For example, as shown in Figure 8, the control function 351 can highlight the organ specified by the cursor C1 and also display the segmentation classification results as icons or the like. In other words, in the medical image processing device 3, when the operator moves the cursor C1 and performs an operation to specify the target of segmentation (for example, by clicking), the target organ is highlighted and the classification result information is displayed.

[0094] As described above, according to the first embodiment, the control function 351 acquires information about the structure to be segmented. The acquisition function 352 acquires information about the likelihood of the structure's presence in the medical image based on the information about the structure. The setting function 353 sets a segmentation region for performing structure segmentation on a portion of the medical image based on the information about the likelihood of the structure's presence. The processing function 354 performs structure segmentation on the segmentation region of the medical image. Therefore, the medical image processing apparatus 3 according to the first embodiment can perform segmentation only for structures desired by the operator, enabling efficient and highly accurate segmentation.

[0095] For example, because the processing range can be narrowed down to only the target desired by the operator, localized bone segmentation or localized blood vessel segmentation can be performed, allowing for the removal of unnecessary parts in the 3D display and reducing the associated computational costs.

[0096] Furthermore, because independent regions such as fractured areas can be easily extracted, the medical image processing device 3 can automatically display fractured areas in a cross-sectional view that is easy to observe. The medical image processing device 3 can also extract a single vertebra and label it based on information about surrounding anatomical features.

[0097] Furthermore, for example, the medical image processing device 3 performs segmentation based on deep learning, enabling accurate segmentation even within regions where similar pixel values ​​are consecutive.

[0098] Furthermore, according to the first embodiment, the acquisition function 352 acquires at least one of the following as information regarding the likelihood of the existence of a structure: the continuity of the structure, a probability map, and anatomical feature points. Therefore, the medical image processing device 3 according to the first embodiment can accurately determine the presence or absence of a structure.

[0099] Furthermore, according to the first embodiment, the acquisition function 352 acquires at least one of the probability maps of structures and the probability map of structures other than structures as probability maps. Therefore, the medical image processing device 3 according to the first embodiment can appropriately determine the presence or absence of structures.

[0100] Furthermore, according to the first embodiment, the control function 351 acquires the structures to be segmented based on the operator's specified operation on the medical image. Therefore, the medical image processing device 3 according to the first embodiment can easily acquire information on the structures to be segmented.

[0101] Furthermore, according to the first embodiment, the control function 351 acquires the structures to be segmented based on the input information of the structures. Therefore, the medical image processing device 3 according to the first embodiment can acquire information on the structures to be segmented without the operator having to perform any specified operations.

[0102] Furthermore, according to the first embodiment, the setting function 353 sets the shape, size, and orientation of the segmentation region based on at least one of the following: the continuity of the structure, the distribution of the probability map of the structure, and anatomical feature points. Therefore, the medical image processing device 3 according to the first embodiment can set the segmentation region to suit various situations.

[0103] Furthermore, according to the first embodiment, the segmentation region is composed of a plurality of sub-regions, and the acquisition function 352 determines the connections between the sub-regions in the plurality of sub-regions based on at least one of the continuity of the structure, the distribution of the probability map of the structure, and anatomical feature points. Therefore, the medical image processing device 3 according to the first embodiment can determine the connections of structures in detail and can suppress the segmentation region from becoming too large.

[0104] Furthermore, according to the first embodiment, the control function 351 causes the segmentation region to be displayed on the medical image. Therefore, the medical image processing device 3 according to the first embodiment can allow the operator to confirm the set segmentation region.

[0105] (Other embodiments) The processing circuits described in each of the embodiments above may be configured by combining multiple independent processors, with each processor executing a program to realize each processing function. Furthermore, each processing function of the processing circuit may be implemented by appropriately distributing or integrating it across one or more processing circuits. Also, each processing function of the processing circuit may be implemented by a mixture of hardware such as circuits and software. While this example describes a case where the programs corresponding to each processing function are stored in a single memory circuit 34, the embodiments are not limited to this. For example, programs corresponding to each processing function may be stored in a distributed manner across multiple memory circuits, and the processing circuit may read and execute each program from each memory circuit.

[0106] In the embodiments described above, examples were given in which each part of this specification is implemented by the respective functions of the processing circuit, but the embodiments are not limited to these. For example, each part of this specification may be implemented not only by the respective functions described in the embodiments, but also by hardware alone, software alone, or a combination of hardware and software.

[0107] Furthermore, the term "processor" used in the above-described embodiment refers to circuits such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), or a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). Here, instead of storing the program in a memory circuit, the processor may be configured to directly incorporate the program into its circuitry. In this case, the processor realizes its function by reading and executing the program incorporated into the circuitry. Moreover, each processor in this embodiment is not limited to being configured as a single circuit; multiple independent circuits may be combined to form a single processor, and its function may be realized in this way.

[0108] Here, the medical image processing program executed by the processor is provided pre-installed in ROM (Read Only Memory) or memory circuits. Alternatively, this medical image processing program may be provided as a file in an installable or executable format on a computer-readable, non-transient storage medium such as a CD (Compact Disk)-ROM, FD (Flexible Disk), CD-R (Recordable), or DVD (Digital Versatile Disk). Furthermore, this medical image processing program may be stored on a computer connected to a network such as the Internet and provided or distributed by downloading it via the network. For example, this medical image processing program consists of modules containing the processing functions described above. In actual hardware, the CPU reads the medical image processing program from a storage medium such as ROM and executes it, loading each module onto the main memory and generating it in the main memory.

[0109] Furthermore, in the embodiments and modifications described above, each component of each illustrated device is a functional concept and does not necessarily have to be physically configured as shown. In other words, the specific form of distribution or integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed or integrated in any unit according to various loads and usage conditions. Moreover, each processing function performed by each device can be realized in whole or in any part by a CPU and a program that is analyzed and executed by the CPU, or by hardware using wired logic.

[0110] Furthermore, among the processes described in the embodiments and modifications described above, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

[0111] According to at least one embodiment described above, it is possible to efficiently perform highly accurate segmentation.

[0112] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0113] 3 Medical Image Processing Equipment 351 Control Functions 352 Acquisition function 353 Settings function 354 Processing Functions

Claims

1. A first acquisition unit that acquires information about the structure to be segmented, A second acquisition unit acquires information regarding the likelihood of the presence of the structure in a medical image based on the information regarding the structure, A setting unit sets a segmentation region for performing segmentation of the structure on a portion of the medical image based on information regarding the likelihood of the existence of the structure, A processing unit that performs segmentation of the structure on the segmentation region of the medical image, A medical image processing device equipped with [a specific feature].

2. The medical image processing apparatus according to claim 1, wherein the second acquisition unit acquires at least one of the continuity of the structure, a probability map, and anatomical feature points as information relating to the likelihood of existence of the structure.

3. The medical image processing apparatus according to claim 2, wherein the second acquisition unit acquires at least one of the probability maps of the structure and the probability maps of structures other than the structure as the probability map.

4. The medical image processing apparatus according to claim 1, wherein the first acquisition unit acquires the structure to be segmented based on an operation specified by the operator on the medical image.

5. The medical image processing apparatus according to claim 1, wherein the first acquisition unit acquires structures that are the target of segmentation based on input information of structures.

6. The medical image processing apparatus according to claim 1, wherein the setting unit sets the shape, size, and orientation of the segmentation region based on at least one of the continuity of the structure, the distribution of the probability map of the structure, and anatomical feature points.

7. The segmentation region is composed of multiple sub-regions, The medical image processing apparatus according to claim 6, wherein the second acquisition unit determines the connections between the subregions in the plurality of subregions based on at least one of the continuity of the structure, the distribution of the probability map of the structure, and anatomical feature points.

8. The medical image processing apparatus according to any one of claims 1 to 7, further comprising a display control unit for displaying the segmentation region on the medical image.

9. Obtain information about the structures to be segmented, Based on the information regarding the aforementioned structure, information regarding the likelihood of the presence of the aforementioned structure in the medical image is obtained. Based on information regarding the likelihood of the existence of the structure, a segmentation region is set for performing segmentation of the structure on a portion of the medical image. The segmentation of the structure is performed on the segmentation region of the medical image. A method that includes doing so.

10. Obtain information about the structures to be segmented, Based on the information regarding the aforementioned structure, information regarding the likelihood of the presence of the aforementioned structure in the medical image is obtained. Based on information regarding the likelihood of the existence of the structure, a segmentation region is set for performing segmentation of the structure on a portion of the medical image. The segmentation of the structure is performed on the segmentation region of the medical image. A program that instructs a computer to perform various processes.

Images