Image processing device, image processing method, and program

Abstract

To reduce a user's time and efforts required for manually editing a segmentation result of a medical image.SOLUTION: An image processing device includes an acquisition unit, an estimation unit, a determination unit, and a display control unit. The acquisition unit acquires medical image data. The estimation unit estimates a structure of an object organ extracted in the medical image data. The determination unit determines a certainty factor indicating the accuracy of an estimation result on the structure of the object organ depicted in the medical image data, for each region of the object organ. The display control unit causes at least one of the information on the necessity of editing by the user based on the certainty factor, and the information indicating a region where the user's editing is restricted to be displayed on a display image based on the medical image data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] In the medical field, diagnoses are performed using images acquired by various imaging devices (modalities), such as CT imaging diagnostic devices. Information on the volume and structure of various organs, such as blood vessels, in medical image data is used for diagnosis. To utilize this information, it is necessary to extract the contours of the relevant regions from the images. However, when this region extraction work is performed manually, it requires a great deal of effort from the operator, which is a problem. Therefore, various techniques for automatic or semi-automatic region extraction from images have been proposed to reduce the operator's effort.

[0003] For example, there is a technology for improving the accuracy of automatic extraction of blood vessel courses, as described in Non-Patent Document 1. There is also a technology for improving the accuracy of identifying non-blood flow areas within blood vessels, as described in Patent Document 1. There is also a technology for improving the accuracy of automatic recognition by using graph theory such as Dijkstra's algorithm or machine learning technology such as U-Net to determine the courses and areas of blood vessels.

[0004] Furthermore, a user may manually edit the results of automatic extraction of various organs such as blood vessels. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-200371 [Non-patent literature]

[0006] [Non-Patent Document 1] Automatic extraction of liver blood vessels based on graph structure analysis (https: / / www.jstage.jst.go.jp / article / tsjc / 2012 / 0 / 2012_67 / _pdf) Summary of the Invention [Problem to be solved by the invention]

[0007] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to reduce the user's effort when manually editing the segmentation results of medical images. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0008] The image processing apparatus according to the embodiment includes an acquisition unit, an extraction unit; A determination unit; an estimation unit; The medical image data acquisition unit acquires medical image data. The extraction unit extracts a vascular region in which the blood vessels of the target organ are depicted from the medical image data. The determination unit determines a degree of certainty, which is an index representing the accuracy of extraction of the vascular region extracted from the medical image data. The estimation unit Based on the confidence level, the background area in the medical image data where blood vessels are not depicted and the area of ​​each anatomical part included in the blood vessel area are identified. Estimate . Rules The decision unit determines the user's , by the estimation part Defines an area where editing of the estimation results is restricted. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of the overall configuration of a medical image processing system according to the first embodiment. [Figure 2] FIG. 2 is a schematic diagram illustrating a part of the volume data according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of segmentation according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of blood vessel running according to the first embodiment. [Figure 5] FIG. 5 is a diagram showing an example of an editing screen according to the first embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of the flow of the analysis process of the blood vessel structure according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing an example of an editing screen according to the first modification of the first embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a blood vessel cross-sectional image corresponding to the first cutting position in the SPR image of FIG. [Figure 9] FIG. 9 is a diagram showing an example of a blood vessel cross-sectional image corresponding to the second cutting position in the SPR image of FIG. [Figure 10] FIG. 10 is a diagram showing an example of a blood vessel cross-sectional image corresponding to the third cutting position in the SPR image of FIG. [Figure 11] FIG. 11 is a diagram illustrating an example of a search cost between nodes of a first branch of a blood vessel according to the second modification of the first embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of a search cost between nodes of a second branch of a blood vessel according to the second modification of the first embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of a search cost between nodes of a third branch of a blood vessel according to the second modification of the first embodiment. [Figure 14] FIG. 14 is a diagram showing an example of an editing screen according to the second modification of the first embodiment. [Figure 15] FIG. 15 is a diagram showing an example of medical image data obtained by capturing an image of the lumen of a blood vessel according to the second embodiment. [Figure 16] FIG. 16 is a diagram showing an example of an estimation result of the lumen region of a blood vessel according to the second embodiment. [Figure 17] FIG. 17 is a diagram showing an example of an editing screen according to the second embodiment. [Figure 18] FIG. 18 is a diagram showing an example of a first cutting position 10001 to a third cutting position 10003 on the editing screen according to the second embodiment. [Figure 19] FIG. 19 is a diagram showing an example of a blood vessel cross-sectional image corresponding to the first cutting position in FIG. [Figure 20]FIG. 20 is a diagram showing an example of a blood vessel cross-sectional image corresponding to the second cutting position in FIG. [Figure 21] FIG. 21 is a diagram showing an example of a blood vessel cross-sectional image corresponding to the third cutting position in FIG. [Figure 22] FIG. 22 is a diagram showing another example of the blood vessel cross-sectional image corresponding to the second cutting position in FIG. [Figure 23] FIG. 23 is a diagram showing an example of a table in which conditions for deducting points from certainty levels according to the second modification of the first and second embodiments are defined. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of an image processing device, an image processing method, and a program will be described in detail with reference to the drawings. However, the dimensions, materials, shapes, and relative arrangements of components described in the following embodiments are arbitrary and can be changed depending on the configuration of the device to which the present invention is applied or various conditions. Furthermore, the same reference numerals are used in the drawings to indicate identical or functionally similar elements.

[0011] (First embodiment) Fig. 1 is a block diagram showing an example of the configuration of a medical image processing system S according to the first embodiment. As shown in Fig. 1, the medical image processing system S includes an image processing device 100, a medical image diagnostic device 200, and a medical image storage device 500. The image processing device 100 is communicably connected to the medical image storage device 500 via a network 300 such as an in-hospital LAN (Local Area Network).

[0012] The medical image storage device 500 stores medical images captured by the medical image diagnostic device 200. The medical image storage device 500 is, for example, a PACS (Picture Archiving and Communication System) server device, and stores medical image data in a format conforming to DICOM (Digital Imaging and Communications in Medicine). The medical images include, but are not limited to, CT (Computed Tomography) image data, magnetic resonance image data, and ultrasound diagnostic image data. The medical image storage device 500 is realized by, for example, a computer device such as a DB (Database) server, and stores medical image data in semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or in memory circuits such as hard disks and optical disks.

[0013] The medical image diagnostic device 200 is, for example, a device that captures medical images of a subject, and may be, for example, an X-ray CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, an X-ray diagnostic device, an ultrasound diagnostic device, a PET (Positron Emission Tomography) device, a SPECT (Single Photon Emission Computed Tomography) device, or the like, but is not limited to these. The medical image diagnostic device 200 is also called a modality. Note that, although one medical image diagnostic device 200 is illustrated in FIG. 1, a plurality of medical image diagnostic devices 200 may be provided.

[0014] A medical image is an image of a subject captured by the medical image diagnostic apparatus 200. Examples of medical images include, but are not limited to, X-ray CT images, magnetic resonance images, and ultrasound images.

[0015] In this embodiment, a case will be described in which the medical image diagnostic apparatus 200 is an X-ray CT apparatus, as an example.

[0016] The image processing device 100 in this embodiment acquires medical image data from a medical image diagnostic device 200 or a medical image storage device 500. The medical image data is, for example, volume data of coronary arteries including blood vessels captured by the medical image diagnostic device 200, which is an X-ray CT device. Note that the medical image data is not limited to this example.

[0017] Furthermore, the image processing device 100 in this embodiment extracts vascular centerlines and segments vascular walls based on the acquired volume data, and presents an editing screen on which the user can edit the results. In this case, the image processing device 100 in this embodiment supports the user's editing work by displaying, among the segmentation results of vascular centerlines and vascular walls, areas where manual editing by the user should be restricted and areas where editing is recommended. The user in this embodiment is, for example, a doctor or a medical technician.

[0018] A vascular wall is an example of a vascular contour. In addition, a vascular region, a vascular contour, and a vascular core line are examples of a structure of a target organ in this embodiment. The structure of a target organ includes at least one of a vascular region, a vascular contour, and a vascular core line.

[0019] Here, an example of the configuration of the image processing device 100 according to this embodiment will be described.

[0020] The image processing device 100 is, for example, an information processing device such as a server device or a PC (Personal Computer), and includes a NW (network) interface 110, a storage circuitry 120, an input interface 130, a display 140, and a processing circuitry 150.

[0021] The NW interface 110 is connected to the processing circuit 150 and controls the transmission and communication of various data between the image processing device 100 and the medical image diagnostic device 200 and the medical image storage device 500. The NW interface 110 is realized by a network card, a network adapter, a NIC (Network Interface Controller), or the like.

[0022] The memory circuitry 120 stores in advance various types of information used by the processing circuitry 150. For example, the memory circuitry 120 stores medical image data acquired from the medical image diagnostic apparatus 200 or the medical image storage apparatus 500. The memory circuitry 120 also stores various programs. The memory circuitry 120 is, for example, a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or an integrated circuit storage device that stores various types of information. In addition to an HDD or an SSD, the memory circuitry 120 may also be a drive device that reads and writes various types of information from and to portable storage media such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a flash memory, or a semiconductor memory element such as a RAM (Random Access Memory).

[0023] The input interface 130 is realized by a trackball that accepts user operations, a switch button, a mouse, a keyboard, a touchpad that performs input operations by touching the operation surface, a touchscreen that integrates a display screen and a touchpad, a non-contact input circuit that uses an optical sensor, an audio input circuit, etc. The input interface 130 is connected to the processing circuit 150, and converts input operations received from the user into electrical signals and outputs them to the processing circuit 150.

[0024] In this specification, the input interface is not limited to an interface having physical operation parts such as a mouse, keyboard, etc. For example, an example of an input interface also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs this electrical signal to processing circuit 150.

[0025] The display 140 displays various types of information under the control of the processing circuitry 150. For example, the display 140 outputs an image interpretation viewer including medical images generated by the processing circuitry 150, a GUI (Graphical User Interface) for accepting various operations from a user, etc. The display 140 is an example of a display unit.

[0026] Specifically, the display 140 is a liquid crystal display, a CRT (Cathode Ray Tube) display, etc. The input interface 130 and the display 140 may be integrated together. For example, the input interface 130 and the display 140 may be realized by a touch panel.

[0027] The display 140 may be provided outside the image processing device 100. For example, a display of another PC or the like connected to the image processing device 100 via a network may be used as an example of the display unit.

[0028] The processing circuitry 150 is a processor that reads out programs from the storage circuitry 120 and executes them to realize functions corresponding to the programs. The processing circuitry 150 of this embodiment includes an acquisition function 151, an extraction and determination function 152, an area definition function 153, an image generation function 154, a display control function 155, a reception function 156, a model generation function 157, and an analysis function 158. The acquisition function 151 is an example of an acquisition unit. The extraction and determination function 152 is an example of an extraction unit, a determination unit, and an estimation unit. The area definition function 153 is an example of an area definition unit. The image generation function 154 is an example of an image generation unit. The display control function 155 is an example of a display control unit. The reception function 156 is an example of a reception unit. The model generation function 157 is an example of a model generation unit. The analysis function 158 is an example of an analysis unit.

[0029] Here, for example, each processing function of the processing circuitry 150, namely, an acquisition function 151, an extraction / determination function 152, an area definition function 153, an image generation function 154, a display control function 155, a reception function 156, a model generation function 157, and an analysis function 158, is stored in the storage circuitry 120 in the form of a computer-executable program. The processing circuitry 150 is a processor. For example, the processing circuitry 150 realizes a function corresponding to each program by reading the program from the storage circuitry 120 and executing it. In other words, the processing circuitry 150 in a state in which each program has been read has each function shown in the processing circuitry 150 of FIG. 1. 1 has been described as realizing the processing functions performed by the acquisition function 151, extraction / determination function 152, area definition function 153, image generation function 154, display control function 155, reception function 156, model generation function 157, and analysis function 158 in a single processor, but it is also possible to combine multiple independent processors to configure the processing circuit 150 and have each processor execute a program to realize the function. Also, in FIG. 1, it has been described as a single storage circuit 120 storing programs corresponding to each processing function, but it is also possible to configure multiple storage circuits to be distributed and have the processing circuit 150 read out corresponding programs from individual storage circuits.

[0030] In the above description, an example has been described in which a "processor" reads and executes a program corresponding to each function from a storage circuit, but the embodiment is not limited to this. The term "processor" refers to a circuit such as a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)). If the processor is, for example, a CPU, the processor realizes the function by reading and executing a program stored in a storage circuit. On the other hand, if the processor is an ASIC, instead of storing the program in the storage circuit 120, the function is directly incorporated as a logic circuit within the processor circuit. Note that each processor in this embodiment is not limited to being configured as a single circuit per processor, but may be configured as a single processor by combining multiple independent circuits to realize its function. Furthermore, the functions of the multiple components in FIG. 1 may be realized by integrating them into a single processor.

[0031] The acquisition function 151 acquires medical image data of an image of a subject from the medical image storage device 500 or the medical image diagnostic device 200 via the network 300 and the NW interface 110. As described above, the medical image data is, for example, volume data of coronary arteries including blood vessels imaged by the medical image diagnostic device 200, which is an X-ray CT device.

[0032] The extraction and determination function 152 estimates the structure of the target organ depicted in the medical image data, and determines the confidence level, which indicates the accuracy of the estimation result of the structure of the target organ depicted in the medical image data, for each region of the target organ.

[0033] In this embodiment, the extraction / determination function 152 includes an extraction function 161, a certainty factor determination function 162, and an estimation function 163. Note that the extraction function 161, the certainty factor determination function 162, and the estimation function 163 may be independent functions.

[0034] The extraction function 161 extracts a target organ from the acquired volume data by segmenting the volume data. More specifically, the extraction function 161 of this embodiment extracts vascular centerlines and vascular walls from the acquired volume data. The target organ in this embodiment is a blood vessel.

[0035] The extraction function 161 may extract only either the vascular center line or the vascular wall.

[0036] Furthermore, the reliability determination function 162 determines the reliability of the extraction of the extracted vascular centerline and vascular wall.

[0037] The certainty is an index that represents the accuracy of the extracted vascular center lines and vascular walls. For example, the accuracy of automatic extraction by the extraction function 161 may decrease depending on the condition of the blood vessel or its surroundings. The accuracy of the extracted vascular wall refers to the accuracy of the contour of the extracted vascular wall. The higher the region where the accuracy of automatic extraction by the extraction function 161 is likely to be high, the higher the certainty. Also, the lower the region where the accuracy of automatic extraction by the extraction function 161 is likely to be low, the lower the certainty. The certainty determination function 162 may determine the certainty of only one of the vascular center lines and the vascular walls.

[0038] Then, the estimation function 163 estimates the blood vessel structure using the certainty factor. In other words, the extraction and determination function 152 analyzes the acquired volume data and obtains the blood vessel structure and the certainty factor for each region of the blood vessel structure.

[0039] Fig. 2 is a schematic diagram illustrating a portion of volume data according to the first embodiment. A blood vessel 3001 shown in Fig. 2 branches into multiple blood vessels from the upper side to the lower side of Fig. 2. Furthermore, plaques 3002a and 3002b have caused stenosis in part of the lumen of the blood vessel 3001.

[0040] The extraction function 161 performs image segmentation of the acquired volume data into two categories: "blood vessels" and "background." Segmentation may use a deep learning method such as U-Net, or may use a method such as threshold determination or region search using pixel values. The extraction function 161 may also perform segmentation using other machine learning methods.

[0041] Furthermore, a trained model based on deep learning or other machine learning, such as U-Net, used for image segmentation may output a confidence level for each pixel along with the segmentation result. The trained model based on deep learning or other machine learning may be stored in, for example, the memory circuitry 120, and the extraction function 161 may read the trained model from the memory circuitry 120 and input the volume data to the trained model. Alternatively, the trained model may be incorporated into the extraction function 161 itself.

[0042] 3 is a diagram showing an example of segmentation according to the first embodiment. The extraction function 161 extracts, for example, a normal blood vessel region 3011, plaque regions 3012a and 3012b, and an extravascular background region 3013.

[0043] The plaque regions 3012a and 3012b are regions in which soft plaques in which cholesterol etc. have accumulated or hard plaques in which calcification etc. has progressed are depicted. Hard plaques in which calcification etc. has progressed contain a large amount of calcium.

[0044] FIG. 3 also shows the certainty determined by the certainty determination function 162 based on the segmentation by the extraction function 161. In FIG. 3, the extraction target is a blood vessel, so the certainty is expressed as a blood vessel certainty. In FIG. 3, the certainty is expressed as a numerical value between 0 and 1. The closer the numerical value is to 1, the higher the certainty, and the closer the numerical value is to 0, the lower the certainty. Note that the notation of the certainty is an example and is not limited to this.

[0045] In the example shown in FIG. 3, the normal blood vessel region 3011 is determined to have a high certainty, while the plaque regions 3012a and 3012b are determined to have a low certainty. In addition, in a background region 3013 outside the blood vessel 3001, the certainty as the blood vessel 3001 is low, for example, 0.2 to 0. In the background region 3013, the certainty of the background is high. Note that while FIG. 3 illustrates a range of certainty values ​​for each region, in reality, the certainty is set individually for each pixel included in each region. Note that, in general, the normal blood vessel region 3011 and the plaque regions 3012a and 3012b have different CT values, and therefore the extraction function 161 may distinguish between the normal blood vessel region 3011 and the plaque regions 3012a and 3012b based on the CT value.

[0046] The estimation function 163 sets a region with a blood vessel certainty of 0.2 or less as a background region 3013, and estimates the other regions, i.e., the normal blood vessel region 3011 and plaque regions 3012a and 3012b, as a blood vessel region 301. Note that the certainty threshold for distinguishing the background region 3013 from the blood vessel region 301 is not limited to a value of 0.2.

[0047] Note that areas with a low confidence level are not limited to plaque areas or calcium areas. For example, the confidence level also decreases in areas where the shape of the blood vessel 3001 is complex, where the width of the blood vessel 3001 is narrow relative to the resolution of the medical image diagnostic device 200, around bifurcations of the blood vessel 3001, and around treatment devices such as stents placed in the blood vessel 3001, because the automatic extraction of the blood vessel wall contour becomes more difficult. The confidence level may also be low depending on the condition of the blood vessel 3001 itself, such as tortuosity of the blood vessel due to myocardial infarction or arteriosclerosis. Furthermore, if the concentration of the contrast agent injected into the blood vessel 3001 for imaging is lower than the specified concentration range or higher than the specified concentration range, the contrast of the area on the medical image data becomes lower, making the automatic extraction of the blood vessel wall contour more difficult and lowering the confidence level.

[0048] Then, the estimation function 163 searches for the course of blood vessels using the blood vessel certainty factor. Specifically, first, the estimation function 163 sets a search start point 3014 within the blood vessel region 301. Note that the estimation function 163 may automatically acquire the search start point from medical image data according to the organ to be analyzed (brain, heart, etc.) or the region to be analyzed (cerebral arteries and veins, coronary arteries, etc.). For example, when the coronary arteries are the region to be analyzed, the estimation function 163 can extract the origins of the coronary arteries by image processing and use them as the search start point. Furthermore, the reception function 156, which will be described later, may receive an operation by the operator to manually specify the search start point.

[0049] Then, the estimation function 163 sets the vascular region 301 as the region for generating a graph. For example, the estimation function 163 sets graph nodes throughout the vascular region 301. Then, the estimation function 163 calculates the search cost between the nodes of the graph based on the vascular certainty. At this time, the estimation function 163 sets a higher search cost the lower the vascular certainty of the pixels between the nodes. In this embodiment, the value of "1-certainty between nodes" is defined as the "search cost." The estimation function 163 calculates the certainty between nodes, for example, from the certainty of both adjacent nodes. The certainty of a node is the certainty of the pixel where the node is set. Alternatively, the certainty of the node may be the average or median of the certainty of multiple pixels around the node.

[0050] Next, the estimation function 163 searches for a path from the search starting point to each node in the graph. The Dijkstra's algorithm or the like may be used for the search. As a result of the search, multiple paths with overlapping nodes are found, and the path with the longest search distance for completely overlapping paths is adopted as the vascular course of each blood vessel. The remaining path is then adopted as the vascular course of each blood vessel. Furthermore, the estimation function 163 divides the path into multiple sections. For each divided section, the maximum value of the search cost between nodes within the section is assigned as the unreliability of the section. In other words, in this embodiment, the unreliability is an index where a larger value indicates lower reliability, and the larger the maximum value of the search cost between nodes, the greater the unreliability. The search cost using the Dijkstra's algorithm is an example of a path search cost in this embodiment.

[0051] 4 is a diagram showing an example of a blood vessel running 3021 according to the first embodiment. In this embodiment, the running of a blood vessel center line 302 of a blood vessel 3001 is called a blood vessel running or a blood vessel structure.

[0052] The blood vessel 3001 shown in Fig. 4 branches into three. The extraction and determination function 152 acquires one route for each of the three branches and calculates the unreliability of each section on each route. In the example shown in Fig. 4, the unreliability around the plaque regions 3012a and 3012b is set high, and the unreliability is set particularly high (i.e., low reliability) for the section where the blood vessel 3001 is blocked by the particularly large plaque region 3012a (section with low blood vessel reliability). The estimation function 163 identifies the finally acquired route as the course of the blood vessel centerline 302.

[0053] The estimation function 163 may calculate the reliability of a vascular centerline based on the maximum value of the search cost between nodes, rather than calculating the unreliability as an index separate from the reliability. In this case, both the reliability for each pixel described in FIG. 3 and the reliability for each node shown in FIG. 4 are collectively referred to as reliability. Also, in FIG. 4, the unreliability is defined as the maximum value of the search cost between nodes, and the larger the value, the lower the reliability. In contrast, the estimation function 163 may use the reliability and the reliability as indexes whose values ​​are higher. For example, the estimation function 163 may calculate the reliability or the reliability so that the smaller the maximum value of the search cost between nodes, the higher the reliability or the reliability value. The estimation function 163 may calculate the reliability or the reliability based on a basis other than the search cost between nodes.

[0054] The search for blood vessel paths may be performed on the entire image without segmentation.

[0055] 1, the region defining function 153 defines a region in which editing by the user of the extraction results of the vascular centerline 302 and the vascular wall in the medical image data is restricted or a region in which editing by the user is recommended, based on the confidence level or the unconfidence level (for example, the maximum value of the search cost between nodes). Note that the region defining function 153 is an example of a defining unit.

[0056] "Editing restriction" includes "edit suppression" and "edit prohibition." "Edit suppression" refers to restricting the amount of editing by a user to within a specified range. Specifically, "edit suppression" may involve limiting the movable range to a specified distance from the initial position through manual operation by the user. The area defining function 153 may set the movable range according to the confidence level or the unconfidence level. For example, the area defining function 153 may set the movable range narrower when the confidence level is high and wider when the confidence level is low. The area defining function 153 may calculate the movable range by multiplying a predetermined reference value of the movable range by the reciprocal of the confidence level. The area defining function 153 may also define the movable range conditions using a predetermined threshold value. As an alternative method of suppression, instead of restricting the editing range, the amount of movement of the editing target in response to the user's editing operation may be reduced to limit the editing speed. In this case, the area defining function 153 may determine the degree of reduction in the amount of movement based on the confidence level.

[0057] Furthermore, "prohibiting editing" means setting the allowable movement range to 0. For example, the area definition function 153 may prohibit editing for a section where the search cost is equal to or less than a predetermined value and it is determined that editing by the user is unnecessary. Note that the above-described editing restrictions may be applied not only when the user directly edits the target area, but also to the interpolation process when editing an adjacent area. In this case, it is possible to alleviate the problem of the target area being deformed by interpolation when the adjacent area is edited.

[0058] The area in which editing by the user is restricted is an example of a first area in this embodiment. Also, the area in which editing by the user is recommended is an example of a second area in this embodiment. Also, among the areas in which editing by the user is restricted, an area in which editing by the user is prohibited is an example of a third area in this embodiment. Among the areas in which editing by the user is restricted, an area in which editing by the user is suppressed is an example of a fourth area in this embodiment.

[0059] In this embodiment, for example, the region defining function 153 defines, as a region where editing by the user is prohibited, a region where the vascular center line 302 and the vascular wall can be extracted with sufficiently high accuracy by the extraction and determination function 152 even without editing by the user. Also, the region defining function 153 defines, as a region where editing by the user is recommended, a region where the vascular center line 302 or the vascular wall may not have been extracted with sufficiently high accuracy by automatic processing by the extraction and determination function 152.

[0060] For example, the region defining function 153 defines, among the node sections of the vascular core line 302, a node section in which the maximum value of the search cost between nodes is equal to or less than a first threshold as a region in which the user is prohibited from editing the vascular core line 302. Furthermore, the region defining function 153 defines, among the node sections of the vascular core line 302, a node section in which the maximum value of the search cost between nodes is equal to or greater than a second threshold as a region in which the user is recommended to edit the vascular core line 302.

[0061] Furthermore, the region defining function 153, for example, defines a region of the blood vessel region 301 extracted from medical image data in which the blood vessel certainty is equal to or greater than a third threshold as a region in which the user is prohibited from editing the contour of the blood vessel 3001. Furthermore, the region defining function 153, for example, defines a region of the blood vessel region 301 extracted from medical image data in which the blood vessel certainty is equal to or less than a fourth threshold as a region in which the user is recommended to edit the contour of the blood vessel 3001. The values ​​of the first to fourth thresholds are not particularly limited.

[0062] The region defining function 153 may define only one of the regions where editing by the user is restricted and the regions where editing by the user is recommended. The region defining function 153 may also classify the vascular region 301 of medical image data into three categories: "editing prohibited," "editing recommended," and "other," or may define the recommended level of editing for the entire vascular region 301 continuously or in stages. For example, the region defining function 153 may express the recommended level of editing as a numerical value such as a percentage. The region defining function 153 may also define the recommended level of editing by classifying it into "low," "medium," "high," or in stages: "level 1," "level 2," and "level 3."

[0063] Returning to FIG. 1 , the image generation function 154 generates SPR (Stretched Multi Planar Reconstruction) image data of the blood vessel 3001 from the medical image data. For example, the image generation function 154 generates an SPR image, which is a three-dimensional image of the coronary artery, by three-dimensionally reconstructing the vascular region of the coronary artery in the coronary artery CT image data. Note that the SPR image data is one form of image data for display, and the format of the image data generated by the image generation function 154 is not limited to this. For example, CPR (Curved Planar Reconstruction) image data, MPR (Multi Planar Reconstruction) image data, or SVR (Shaded Volume Rendering) data may also be employed. The image generation function 154 may also generate two-dimensional image data as image data for display.

[0064] The display control function 155 displays the extracted vascular centerline 302 and vascular wall on an SPR image based on the SPR image data, along with information indicating areas where editing is prohibited or recommended. The display control function 155 may also display restrictions on editing or suppression of editing. The SPR image is an example of a display image in this embodiment.

[0065] More specifically, the display control function 155 displays an editing screen for the vascular centerline 302 and the vascular wall on the display 140. The editing screen is a screen in which the segmentation results of the vascular centerline 302 and the vascular wall are superimposed on the SPR image together with display of editing restrictions or recommendations. Note that only either the vascular centerline 302 or the vascular wall may be displayed on the editing screen.

[0066] Fig. 5 is a diagram showing an example of an editing screen according to the first embodiment. In the example of the editing screen shown in Fig. 5, an SPR image 303, a vascular center line 302 superimposed on the SPR image 303, a message 4001 indicating an area where editing is restricted, messages 4002a and 4002b indicating areas where confirmation is recommended, and a confirm button 1401 are displayed on the display 140.

[0067] 5, the display control function 155 masks the area where editing is restricted. A message 4001 indicating the masked area and the area where editing is restricted is an example of information indicating an area where editing is restricted. Messages 4002a and 4002b indicating an area where confirmation is recommended are also an example of information indicating an area where editing is recommended.

[0068] In the example shown in Fig. 5, the display control function 155 displays sections in which the search cost between nodes shown in Fig. 4 is 0.5 or more as areas where editing is recommended. Furthermore, the display control function 155 displays sections in which the search cost between nodes shown in Fig. 4 is 0.3 or less as areas where editing is restricted. Furthermore, for sections in which the search cost between nodes shown in Fig. 4 is greater than 0.3 and less than 0.5, the display control function 155 displays the vascular centerlines 302 estimated in Fig. 3 and Fig. 4 as they are. Note that the criteria for areas where editing is restricted and areas where editing is recommended are not limited to the example shown in Fig. 5.

[0069] Furthermore, the display manner of the areas where editing is restricted and areas where editing is recommended is not limited to the message and mask exemplified in FIG. 5, but may be various marks, color changes, etc.

[0070] Note that, instead of information indicating an area where editing is restricted, information indicating an area where editing is prohibited or information indicating an area where editing is restricted may be displayed on the editing screen.

[0071] The display control function 155 may also display the degree of recommendation for editing as a numerical value on the SPR image 303 on the editing screen. For example, the display control function 155 may display on the SPR image 303 the maximum value of the search cost for each node of the vascular core line 302 shown in FIG. 4. In this case, the larger the displayed maximum value of the search cost, the higher the degree of recommendation for editing. The display control function 155 may also display on the SPR image 303 the average value for each range of the vascular certainty factor for each pixel described in FIG. 3. The display control function 155 may display the average value of the vascular certainty factor for each pixel for each node of the vascular core line 302, or may display the average value of the vascular certainty factor for each pixel for each range based on other criteria. The display control function 155 may also display information on the SPR image 303 on the editing screen that indicates the recommendation level for editing in stages, using classifications such as "low," "medium," and "high," or "level 1," "level 2," and "level 3." The numerical value and classification indicating the degree to which editing is recommended are an example of information indicating whether or not editing by the user is necessary in this embodiment.

[0072] Furthermore, the display control function 155 may convert the value of the confidence level or the search cost and display the converted value on the SPR image 303, rather than displaying the value of the confidence level or the search cost as is. The content of the conversion process is not particularly limited. The conversion process may be executed by any one of the area definition function 153, the image generation function 154, or the display control function 155.

[0073] When the user operates and corrects the vascular center line 302 or the vascular wall segmentation results on the editing screen using a mouse or the like, the display control function 155 displays the corrected vascular center line 302 and vascular wall on the SPR image 303.

[0074] The confirm button 1401 is an image button that the user can press with a mouse or the like. When the user presses this button, the segmentation results of the vascular center line 302 and the vascular wall are confirmed. For example, the user operates and corrects the segmentation results of the vascular center line 302 or the vascular wall, and then confirms the correction results by pressing the confirm button 1401. Note that the editing screen is not limited to the example shown in FIG. 5, and a known UI (User Interface) may be adopted.

[0075] The display control function 155 also displays on the display 140 the confidence level or search cost and information representing the area edited by the user on a color map representing the vascular analysis results obtained by the analysis function 158 (described later). The information representing the area edited by the user is, for example, an image surrounding the portion of the vascular centerline 302 or vascular wall where the user has edited the vascular centerline 302 or vascular wall on the editing screen. The display manner of the information representing the area edited by the user is not limited to this. Furthermore, if the user edits the same portion of the vascular centerline 302 or vascular wall multiple times, the display manner may vary depending on the number of edits. Furthermore, the display control function 155 may display information representing not only whether or not a correction has been made, but also the degree of correction. For example, the information representing the degree of correction may be an index whose numerical value increases as the difference between the vascular centerline 302 or vascular wall before correction and the vascular centerline 302 or vascular wall after correction increases, or a color map whose color becomes darker.

[0076] Returning to FIG. 1, the reception function 156 receives various operations from the user via the input interface 130.

[0077] When the user performs an editing operation on the editing screen in an area where editing is restricted as defined by the area defining function 153, the reception function 156 does not accept the operation or converts the operation and accepts it. For example, when the user performs an operation to modify the vascular center line 302 or the vascular wall in an area where editing is prohibited, the reception function 156 does not accept the operation. Furthermore, when the user performs an operation to modify the vascular center line 302 or the vascular wall in an area where editing is restricted, the reception function 156 accepts the operation if it is included in the range set as the movable range.

[0078] Furthermore, when the user performs an operation to modify the vascular centerline 302 or the vascular wall in an area where editing is restricted, if the operation exceeds the range set as the movable range, the reception function 156 converts the operation into an operation up to the outer edge of the movable range and accepts the operation. Alternatively, when the user performs an operation to modify the vascular centerline 302 or the vascular wall in an area where editing is restricted, the reception function 156 may accept the operation by reducing the amount of movement of the editing target in response to the user's editing operation. For example, the reception function 156 may change the rate of reduction of the amount of movement depending on the confidence level.

[0079] Furthermore, the model generation function 157 generates a model for blood vessel analysis based on the determined blood vessel centerline 302 and the segmentation results of the blood vessel wall. The model for blood vessel analysis is, for example, a model for fluid structure analysis. Note that the method for blood vessel analysis is not limited to fluid structure analysis, and other analysis methods may be adopted.

[0080] The analysis function 158 uses a model for blood vessel analysis to perform blood vessel analysis of the blood vessel 3001. For example, the analysis function 158 uses a model for fluid structure analysis to perform fluid structure analysis.

[0081] The analysis function 158 generates a color map representing the vascular analysis results. The color map is, for example, image data that displays blood vessels depicted in SPR image data in different colors according to the values ​​of the analysis results. Note that the image data representing the vascular analysis results is not limited to a color map. Fluid parameters used as the values ​​of the analysis results displayed on the color map may include, for example, blood pressure, blood flow, and FFR (Fractional Flow Reserve) at each position of the blood vessel.

[0082] Furthermore, as a result of the fluid structure analysis performed by the analysis function 158, the accuracy of the segmentation results of the vascular centerline 302 or the vascular wall may be low in areas where the flow rate is too low compared to the vascular diameter or where the local pressure fluctuations are too large. In this case, the certainty determination function 162 or the estimation function 163 may modify the certainty according to the results of the fluid structure analysis. The display control function 155 may superimpose the certainty modified according to the results of the fluid structure analysis and whether or not the data has been edited by the user on a color map representing the vascular analysis results.

[0083] Next, the flow of the analysis process of the blood vessel structure executed by the image processing device 100 configured as above will be described.

[0084] FIG. 6 is a flowchart showing an example of the flow of the analysis process of the blood vessel structure according to the first embodiment.

[0085] First, the acquisition function 151 acquires volume data of the coronary arteries obtained by imaging the subject from the medical image storage device 500 or the medical image diagnostic device 200 (S101).

[0086] The extraction and determination function 152 then extracts the vascular centerline 302 and the vascular wall from the acquired volume data. In addition, the extraction and determination function 152 determines the confidence level of the vascular centerline 302 and the vascular wall in association with the extraction and estimation process (S102). The confidence level may be output by a trained model as described above, or may be a search cost using the Dijkstra algorithm or the like.

[0087] Then, the area defining function 153 defines an area where editing is restricted or an area where editing is recommended (S103). Note that the area defining function 153 may define both an area where editing is prohibited and an area where editing is suppressed as the area where editing is restricted, or may define only one of an area where editing is prohibited and an area where editing is suppressed. Furthermore, the area defining function 153 may define only one of an area where editing is restricted and an area where editing is recommended.

[0088] Then, the image generating function 154 generates SPR image data of the blood vessels from the acquired volume data (S104).

[0089] Then, the display control function 155 displays the extracted vascular centerline 302 and vascular wall on the SPR image together with information indicating areas where editing is prohibited or recommended on the display 140 (S105). For example, the display control function 155 displays the editing screen shown in FIG. 5 on the display 140.

[0090] Then, when the reception function 156 receives a user's correction operation on the editing screen (S106 "correction"), the display control function 155 displays the corrected vascular centerline 302 and vascular wall on the SPR image (S107). Note that the reception function 156 does not receive a user's correction operation in an area where editing is prohibited.

[0091] Here, if the reception function 156 receives a user operation to further correct the corrected vascular centerline 302 and vascular wall (S108 "re-correction"), the process returns to S107. At this time, the display control function 155 displays the area corrected by the user's previous operation on the editing screen in a different manner from the uncorrected area.

[0092] Furthermore, when the reception function 156 receives an operation to confirm the corrected vascular centerline 302 and vascular wall, for example, an operation to press the Confirm button 1401 on the editing screen (S108 "Confirm"), the model generation function 157 generates a model for vascular analysis based on the segmentation results of the confirmed vascular centerline 302 and vascular wall (S109). Also, in the process of S106, if the user performs a confirmation operation without making any correction operation (S106 "Confirm"), the process proceeds to S109.

[0093] Then, the analysis function 158 executes a fluid structure analysis of the blood vessel 3001 using the generated model for blood vessel analysis (S110).

[0094] Next, the analysis function 158 generates a color map that represents the results of the fluid structure analysis (S111).

[0095] Then, the display control function 155 superimposes the confidence level and whether or not the color map has been edited by the user on the color map, and displays the result on the display 140 (S112). Note that the confidence level displayed on the color map may be a confidence level corrected according to the result of the fluid structure analysis.

[0096] Then, when the reception function 156 receives a user operation to re-correct the vascular center line 302 and the vascular wall on the color map (S113 "re-correction"), the process returns to S107. In addition, the display control function 155 displays the area corrected by the user's previous operation on the editing screen in a different manner from the uncorrected area.

[0097] Furthermore, when the reception function 156 receives a confirmation operation from the user (S113 "Confirm"), the vascular center line 302 and vascular wall confirmed by the user are stored in the storage circuitry 120, and the processing of this flowchart ends.

[0098] As described above, the image processing device 100 of this embodiment estimates the structure of a target organ depicted in medical image data and determines a confidence level representing the accuracy of the estimation result of the structure of the target organ depicted in the medical image data for each region of the target organ. Furthermore, the image processing device 100 defines regions in which editing of the estimation result by the user is restricted based on the confidence level. At least one of information indicating whether editing is required or information indicating regions where editing is restricted is displayed on the display image based on the medical image data. Therefore, the image processing device 100 of this embodiment can reduce the user's effort when manually editing the segmentation results of medical images.

[0099] For example, as a comparative example, when a user manually edits a vascular center line or vascular wall after automatic extraction, if the user cannot grasp the accuracy of the automatic analysis, the user may have to recheck all blood vessels before editing, which is a waste of time. Furthermore, there is a possibility that the user may corrupt the results of highly accurate automatic extraction through manual editing, thereby reducing the accuracy of region extraction. In contrast, with the image processing device 100 of this embodiment, when a user manually edits, the user can easily grasp the regions to be edited based on information indicating whether editing is necessary or information indicating regions where editing is restricted, thereby reducing the effort required for manual editing by the user.

[0100] (Modification 1 of the first embodiment) In the editing screen of the first embodiment described above, as shown in FIG. 5, the vascular core line 302 is superimposed on the SPR image 303, but vascular contour information for specifying the vascular region, i.e., the segmentation result of the vascular wall, may be displayed on the SPR image 303.

[0101] FIG. 7 is a diagram showing an example of an editing screen according to Modification 1 of the first embodiment. As shown in FIG. 7, the display control function 155 may display vascular contour information 5001 on the SPR image 303. The vascular contour information 5001 may be, for example, boundary information of a region where the vascular certainty factor is equal to or greater than a certain value. For example, the display control function 155 displays boundary information of a region where the vascular certainty factor is greater than 0.2 as the vascular contour information 5001. Alternatively, the region defining function 153 may calculate the vascular contour information 5001 from pixel values ​​around the vascular centerline 302, for example. The vascular contour information 5001 is an example of the structure of a target organ in this modification.

[0102] On the editing screen shown in FIG. 7, editing restrictions on user operations are applied to the vascular center line 302 and the vascular contour information 5001.

[0103] Images of blood vessels in the cross-sectional direction may be displayed on the editing screen. Figures 8 to 10 are diagrams showing examples of blood vessel cross-sectional images corresponding to the first cutting position 5011, the second cutting position 5012, and the third cutting position 5013 in the SPR image 303 of Figure 7, respectively.

[0104] The first cutting position 5011 is an area where editing is restricted, and therefore, in Fig. 8, editing restrictions are imposed on the vascular center line 302 and the vascular wall contour for the entire cross section. The second cutting position 5012 is neither an area where editing is restricted nor an area where editing is recommended, and therefore, in Fig. 9, the vascular center line 302 and the vascular wall contour are displayed in an editable state. The third cutting position 5013 is an area where editing is recommended, and therefore, in Fig. 10, information recommending editing of the vascular center line 302 and the vascular wall contour for the entire vascular cross section, such as a message 4002b, is displayed.

[0105] (Modification 2 of the first embodiment) In addition, the process of restricting editing or determining recommended areas using the area definition function 153 may be performed on a route basis, i.e., on a branching basis of the blood vessel, rather than on a section basis in which the route of the blood vessel 3001, such as between each node, is divided into multiple sections.

[0106] For example, the region definition function 153 obtains the maximum search cost between nodes on each path and checks whether there are any sections on the path with low confidence in order to determine the range in which user editing should be restricted or recommended.

[0107] 11 to 13 are diagrams showing an example of search costs between nodes of the first to third branches of a blood vessel 3001 according to the second modification of the first embodiment.

[0108] The blood vessel 3001 shown in Figures 11 to 13 branches into three blood vessels 6001, 6002, and 6003. As shown in Figures 11 to 13, the maximum search costs of the three blood vessels 6001, 6002, and 6003 are "0.9", "0.3", and "0.5". The area defining function 153 defines routes with low maximum search costs, for example, routes with a maximum search cost of 0.3 or less, as areas where editing is restricted. Furthermore, the area defining function 153 defines routes with high maximum search costs, for example, routes with a maximum search cost of 0.6 or more, as areas where editing is recommended.

[0109] Fig. 14 is a diagram showing an example of an editing screen according to Modification 2 of the first embodiment. In the example shown in Fig. 14, editing restrictions or recommendations are displayed on a branch-by-branch basis. For example, because the maximum search cost of blood vessel 6001, which is the first branch, is 0.9, the display control function 155 displays a message 7002 recommending editing near blood vessel 6001. Furthermore, because the maximum search cost of blood vessel 6002, which is the second branch, is 0.3, the display control function 155 displays a message 7001 restricting editing near blood vessel 6002. Because the maximum search cost of blood vessel 6003, which is the third branch, is neither high nor low, it is not subject to editing restrictions or recommendations.

[0110] The display control function 155 may display the maximum search cost of each branch or the individual search costs between nodes on the editing screen.

[0111] (Modification 3 of the first embodiment) In the above-described first embodiment, an example has been described in which a plurality of relatively thin blood vessels 6001, 6002, and 6003 branch off from one thick blood vessel 3001, but the shape of the target blood vessel is not limited to this.

[0112] For example, the coronary arteries include three major blood vessels: the right coronary artery (RCA), the left circumflex artery (LCX), and the left anterior descending artery (LAD). In this case, the extraction function 161 automatically extracts the RCA, LCX, and LAD from medical image data. The confidence level determination function 162 and the estimation function 163 calculate confidence levels based on the search costs and other factors for the RCA, LCX, and LAD. The region definition function 153 determines whether to restrict or recommend user editing for each of the RCA, LCX, and LAD. If editing is recommended for any of the RCA, LCX, and LAD, the display control function 155 displays the recommended blood vessels on the editing screen.

[0113] For example, the display control function 155 may display a three-dimensional image including the RCA, LCX, and LAD, and a two-dimensional image that displays the RCA, LCX, and LAD individually, on the display 140. In this case, the display control function 155 may display, for example, two-dimensional images of blood vessels among the RCA, LCX, and LAD that are recommended for editing, and two-dimensional images of blood vessels that are neither prohibited nor recommended for editing, in an editable state. Note that the editing screen for each blood vessel may be a three-dimensional image.

[0114] Alternatively, the display control function 155 may display a list of names of blood vessels recommended for editing among the RCA, LCX, and LAD on the display 140. In this case, when the user selects the name of a blood vessel from the list, an editing screen for the selected blood vessel may be displayed.

[0115] Alternatively, the display control function 155 may display the automatically extracted names of the RCA, LCX, and LAD as a list on the display 140, and highlight only the names of the blood vessels recommended for editing among the RCA, LCX, and LAD. Note that the RCA, LCX, and LAD are examples of blood vessels, and the configuration of this modified example can also be applied to other blood vessels.

[0116] (Fourth modification of the first embodiment) In the first embodiment described above, an example was described in which information indicating editing restrictions or recommendations, confidence level, search cost, etc., is displayed on the vascular image on the editing screen, but the reason for the editing restrictions or recommendations may also be displayed on the vascular image.

[0117] For example, the confidence level decreases in areas where the blood vessel shape is complex, such as around a blood vessel branch. The confidence level determination function 162 may output, for example, information on the basis of the confidence level along with the confidence level. As an example, an output trained model that outputs information on the basis of the confidence level along with the confidence level may be used, or a rule-based method may be used. In addition, the display control function 155 displays the reason for restricting or recommending editing on the editing screen. For example, the display control function 155 may display a message such as "Due to the presence of a blood vessel branch, the accuracy of automatic extraction may be low" or "Due to the presence of a blood vessel branch, confirmation and editing are recommended" near the blood vessel branch.

[0118] (Fifth Modification of the First Embodiment) In the first embodiment described above, the fluid structure analysis process is executed in the flow of the blood vessel structure analysis process, but the fluid structure analysis process is not essential. Furthermore, the timing of executing the fluid structure analysis is not limited to the example shown in Fig. 6. For example, the fluid structure analysis may be executed in advance before the blood vessel structure editing screen is displayed, or may be executed after the blood vessel structure is finalized.

[0119] (Second embodiment) In the first embodiment described above, the estimation of the vascular center line 302 and the contour of the vascular wall that is the boundary between the inside and outside of the blood vessel was exemplified. In this second embodiment, the estimation of the shape of the lumen region of the blood vessel will be exemplified.

[0120] Similar to the first embodiment, the medical image processing system S of this embodiment includes an image processing device 100, a medical image diagnostic device 200, and a medical image storage device 500. The configuration of the image processing device 100 is similar to that of the first embodiment.

[0121] In the first embodiment, the image processing device 100 restricted or recommended manual editing based on the confidence level during blood vessel path search. In contrast, the image processing device 100 of this embodiment restricts or recommends manual editing based on the confidence level of the estimated blood vessel lumen region (= estimated lumen region). The contour of the blood vessel lumen is an example of the structure of the target organ in this modified example.

[0122] For example, the extraction function 161 of this embodiment extracts a blood vessel lumen region from medical image data. The certainty determination function 162 of this embodiment determines the certainty of each pixel or region of the extracted lumen region. In this embodiment, the certainty represents the accuracy of automatic extraction of the blood vessel lumen, and is therefore also referred to as lumen certainty.

[0123] The region defining function 153 defines a region with a high lumen certainty among regions inferred to be the lumen of a blood vessel as a region where editing is restricted, and also defines a region with a low lumen certainty among regions inferred to be the lumen of a blood vessel as a region where editing is recommended.

[0124] Fig. 15 is a diagram showing an example of medical image data obtained by capturing an image of the lumen of a blood vessel according to the second embodiment. The medical image data shown in Fig. 15 includes a boundary 8001 of a blood vessel region and plaque 8002.

[0125] The extraction function 161 of this embodiment performs segmentation of medical image data into three categories: "lumen," "plaque," and "background." As with the first embodiment, the segmentation method may be machine learning or threshold judgment, but in this embodiment, a method is used in which at least certainty maps of "lumen," "plaque," and "background" are estimated and used. That is, the extraction function 161 compares the certainty maps of "lumen," "plaque," and "background" obtained from the medical image data, and determines the region with the highest certainty of lumen as the estimated lumen region.

[0126] The display control function 155 of this embodiment displays the lumen region estimated by the extraction function 161 on the editing screen.

[0127] Fig. 16 is a diagram showing an example of the estimation result of the blood vessel lumen region according to the second embodiment. The image shown in Fig. 16 is the result of superimposing a boundary 8003 of the estimated lumen region segmented by the extraction function 161 onto the medical image data shown in Fig. 15. Because the estimated lumen region is determined by estimation, it may differ from the actual lumen shape, and may differ significantly from the actual lumen shape in areas where automatic extraction is difficult, such as around plaque 8002.

[0128] The region defining function 153 of this embodiment acquires the lumen certainty map from the extraction and determination function 152. Then, on the editing screen, a portion of the estimated lumen region where the region boundary surface has high accuracy is defined as a region where manual editing is restricted. The region defining function 153 determines the accuracy of the region boundary surface based on the difference in certainty between the inside and outside of the region interface. For example, if there is a large difference in lumen certainty on both sides of the boundary 8003 of the estimated lumen region, the region defining function 153 determines that the accuracy of the boundary 8003 is high. As an example, the region defining function 153 restricts editing of the boundary 8003 of the estimated lumen region when the difference in lumen certainty is 0.8 or more.

[0129] Furthermore, the region defining function 153 defines a portion of the estimated lumen region where the region boundary surface has low accuracy as a region for which editing is recommended. For example, if the difference in lumen certainty on both sides of the boundary 8003 of the estimated lumen region is small, the region defining function 153 determines that the accuracy of the boundary 8003 is low. As an example, the region defining function 153 recommends editing the boundary 8003 of the estimated lumen region when the difference in lumen certainty is 0.5 or less.

[0130] The display control function 155 of this embodiment displays information indicating areas where editing by the user is restricted and areas where editing is recommended on the editing screen for the boundary 8003 of the estimated lumen area.

[0131] FIG. 17 is a diagram showing an example of an editing screen according to the second embodiment.

[0132] The display control function 155 displays messages 9001a to 9001d indicating restrictions on editing in areas defined by the area defining function 153 as areas where editing by the user is restricted, relative to the boundary 8003 of the estimated lumen area. The display control function 155 may also display a mask image or the like in areas defined as areas where editing by the user is restricted. The display control function 155 also displays a message 9002 recommending editing in areas defined by the area defining function 153 as areas where editing by the user is recommended.

[0133] Furthermore, as described in the first modification of the first embodiment, the display control function 155 may display a blood vessel cross-sectional image on the editing screen.

[0134] Fig. 18 is a diagram showing an example of a first cutting position 10001 to a third cutting position 10003 on an editing screen according to the second embodiment. Figs. 19 to 21 are diagrams showing an example of a blood vessel cross-sectional image corresponding to the first cutting position 10001 to the third cutting position 10003 in Fig. 18. Each blood vessel cross-sectional image is a cross section obtained by cutting the first cutting position 10001 to the third cutting position 10003 shown in Fig. 18 at a right angle to the direction in which the blood vessel runs.

[0135] The setting of regions where editing is restricted and recommended may be performed partially or uniformly on a vascular cross section. For example, the region defining function 153 may restrict editing in a region within the same cross section where the difference in lumen certainty on both sides of the outer periphery of the cross section is greater than a fifth threshold, and may recommend editing in a region where the difference in lumen certainty on both sides of the outer periphery of the cross section is less than a sixth threshold. The values ​​of the fifth and sixth thresholds are not particularly limited.

[0136] As shown in Fig. 19, the cross section at the first cutting position 10001 has a difference in lumen certainty factor on both sides of the cross section outer periphery that is greater than the fifth threshold, so editing is restricted in the entire region. Also, as shown in Fig. 20, the cross section at the second cutting position 10002 has a mixture of regions where the difference in lumen certainty factor on both sides of the cross section outer periphery is greater than the fifth threshold and regions where the difference is less than the sixth threshold. Therefore, editing is restricted in some regions and recommended in other regions. Also, as shown in Fig. 21, the cross section at the third cutting position 10003 has a mixture of regions where editing is neither restricted nor recommended and regions where editing is recommended. Therefore, the display control function 155 displays a message 9002 recommending editing only in the recommended regions.

[0137] Alternatively, if there is a region in the periphery of a cross section in which editing is recommended, even if it is only a part of the periphery, the region defining function 153 may determine the entire periphery of the cross section as a region in which editing is recommended.

[0138] Fig. 22 is a diagram showing another example of a blood vessel cross-sectional image corresponding to the second cut position in Fig. 18. The cross section at the second cut position 10002 includes a mixture of an area where the difference in lumen certainty on both sides of the periphery of the cross section is greater than the fifth threshold and an area where the difference is less than the sixth threshold. Therefore, in the example shown in Fig. 22, the entire periphery of the cross section is set as an area where editing is recommended.

[0139] In this way, according to the image processing device 100 of this embodiment, even when the shape of the lumen region of a blood vessel is to be estimated, the user's effort in manually editing the segmentation results of a medical image can be reduced, as in the first embodiment.

[0140] Note that the region defining function 153 may use the certainty of the estimated lumen region rather than the accuracy of the boundary 8003 of the estimated lumen region when determining whether to restrict or recommend editing. The certainty of the estimated lumen region around the region to be edited may be obtained, and if the minimum value is equal to or greater than a seventh threshold, the region may be determined to be one for which editing is restricted, and if the minimum value is equal to or less than an eighth threshold, the region may be determined to be one for which editing is recommended. The seventh threshold for the minimum value of lumen certainty may be, for example, 0.8, and the eighth threshold for the minimum value of lumen certainty may be, for example, 0.5, but these values ​​are not limited to these.

[0141] Furthermore, the region defining function 153 may limit the region where editing is restricted or recommended to the periphery of plaque where it is difficult to estimate the lumen region. For example, the region defining function 153 acquires a lumen certainty map from the extraction and determination function 152, and sets the region with the highest plaque certainty as the estimated plaque region. The region defining function 153 may perform processing to set a region where editing is restricted or recommended only at the interface between the estimated plaque region and the estimated lumen region.

[0142] Note that when the extraction and determination function 152 uses a region boundary estimation method such as a graph cut method for segmenting the lumen region, boundary cutting cost information in the region boundary estimation method may be used as the blood vessel certainty. In this case, for example, the extraction and determination function 152 estimates a lumen certainty map for medical image data, then generates a graph in which the difference in certainty in the lumen certainty map is reflected in the cutting cost, and performs graph cutting. At this time, the extraction and determination function 152 sets a higher cost the smaller the difference in certainty. Then, the extraction and determination function 152 determines the inside of the cut boundary as the estimated lumen region.

[0143] When this method is adopted, the area definition function 153 obtains the cost of each boundary cut from the graph information when the graph cut is performed by the extraction / determination function 152, and sets boundaries with low costs as targets for editing restriction and boundaries with high costs as targets for editing recommendation.

[0144] (Modification 1 of the first and second embodiments) In the first and second embodiments described above, blood vessels are used as an example of the target organ, but the target organ is not limited to blood vessels. For example, the target organ may be any organ having a tubular structure, such as a lumen of a blood vessel, lymphatic vessel, ureter, esophagus, bronchus, digestive tract, or stomach.

[0145] (Modification 2 of the first and second embodiments) In the first and second embodiments described above, the extraction and determination function 152 simultaneously performs segmentation of the target area and determination of the confidence level, but the determination of the confidence level may be performed as a separate process after the segmentation of the target area is completed.

[0146] For example, the certainty determination function 162 may determine the certainty of each region of the target organ using a rule-based method. In the initial state, the certainty of all regions may be set to the same value, for example, "100," and the certainty determination function 162 may calculate the certainty of each region by calculating the certainty of regions that meet certainty deduction conditions.

[0147] FIG. 23 is a diagram showing an example of a table defining certainty factor deduction conditions according to Modification 2 of the first and second embodiments. The table stores conditions that reduce the accuracy of automatic blood vessel extraction, among the characteristics of blood vessels and their surrounding structures, in association with deduction points indicating the degree of reduction in accuracy. The table is stored in, for example, the storage circuitry 120. The certainty factor determination function 162 performs certainty factor deduction processing based on the table. Furthermore, the display control function 155 may display, on the editing screen, the conditions that each region falls under as reasons for the certainty factor for that region. The conditions and deduction points shown in FIG. 23 are merely examples and are not limited thereto. Note that if the target organ is not a blood vessel, the conditions will be different.

[0148] The method of calculating the confidence level in the rule-based method is not limited to the point deduction method. For example, a table defining the conditions for adding points to the confidence level may be used. Furthermore, the confidence levels of the entire region may not be the same in the initial state. For example, the confidence level determination function 162 may initially set the confidence level for each pixel calculated by the extraction function 161 or the search cost calculated by the estimation function 163, and then perform a rule-based point deduction or point addition.

[0149] (Modification 3 of the first and second embodiments) In the first and second embodiments described above, the display control function 155 displays information indicating the area where user editing is restricted and the recommended area on the editing screen, but the area where user editing is restricted does not have to be displayed. Even in this case, the user's operation in the area where user editing is restricted is restricted.

[0150] (Modification 4 of the first and second embodiments) In the first and second embodiments described above, the editing screen is displayed on the display 140 of the image processing device 100, but it may also be displayed on a display of another information processing device. In this case, the display of the other information processing device is an example of a display unit.

[0151] The various data handled in this specification are typically digital data.

[0152] According to at least one of the embodiments described above, it is possible to reduce the user's effort when manually editing the segmentation results of a medical image.

[0153] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0154] 100 Image processing device 120 Memory circuit 140 Display 150 Processing Circuit 151 Acquisition Function 152 Extraction / judgment function 153 Area definition function 154 Image generation function 155 Display Control Function 156 Reception Function 157 Model generation function 158 Analysis Function 161 Extraction function 162 Confidence Determination Function 163 Estimation Function 300 Network 301 Vascular area 302 Vascular Core Wire 3001 Blood vessels 3002a, 3002b plaque 3012a, 3012b Plaque area 3013 Background area 3014 Search starting point 3021 running

Claims

1. an acquisition unit that acquires medical image data; an extraction unit that extracts a vascular region in which blood vessels of a target organ are depicted from the medical image data; a determination unit that determines a degree of certainty, which is an index representing the accuracy of extraction of the vascular region extracted from the medical image data; an estimation unit that estimates a background region in which the blood vessels are not depicted in the medical image data and a region for each anatomical part included in the blood vessel region based on the certainty factor; a defining unit that defines an area in which editing of the estimation result by the estimation unit by a user is restricted based on the certainty; An image processing device comprising:

2. a display control unit that displays, on a display image based on the medical image data, at least one of information indicating whether the editing is necessary or information indicating an area to which the editing is restricted; The image processing device according to claim 1 , further comprising:

3. a region defining unit that defines, based on the distribution of the certainty factors in the structure of the target organ, at least one of a first region in which editing by a user is restricted and a second region in which editing by the user is recommended, in the structure of the target organ; the display control unit causes the display unit to superimpose on the display image the estimated structure of the target organ and information representing a range of the structure that corresponds to the first region or the second region. The image processing device according to claim 2 .

4. The vascular region includes a vascular core and a vascular wall, The first region includes at least one of a third region that prohibits the user from editing the extraction results of the vascular center line and the vascular wall, and a fourth region that limits a range in which the user can move the extracted positions of the vascular center line and the vascular wall by manual operation to a specified distance. The image processing device according to claim 3 .

5. a receiving unit that does not accept an editing operation by the user for a range included in the third area of ​​the structure of the target organ displayed on the display unit, accepts an editing operation by the user for a range included in the fourth area only within the range of the specified distance, and accepts an editing operation by the user for a range not included in the first area. The image processing device according to claim 4 .

6. a receiving unit that receives the editing operation by the user with respect to the range included in the fourth area by reducing the amount of movement of the editing object, The image processing device according to claim 4 .

7. the reception unit changes a rate of reduction of the movement amount based on the confidence level; The image processing device according to claim 6 .

8. The information indicating whether editing is necessary is a numerical value or a classification indicating the degree to which editing is recommended. The image processing device according to claim 2 .

9. the numerical value representing the degree of recommendation for editing is the confidence level or a numerical value calculated from the confidence level by a conversion process; The image processing device according to claim 8 .

10. The estimation unit estimates the course of the blood vessels depicted in the medical image data, The lower the confidence level, the higher the cost of path search for each section of the blood vessel in estimating the course of the blood vessel. The image processing device according to any one of claims 1 to 9.

11. The estimation unit estimates a contour of an outer wall of the blood vessel depicted in the medical image data, The confidence level represents the accuracy of the estimation of the lumen contour of the blood vessel for each pixel in the estimation of the lumen contour. The image processing device according to any one of claims 1 to 9.

12. The certainty factor is a lumen certainty factor that represents the accuracy of automatic extraction of the lumen of the blood vessel from the blood vessel region for the medical image data, the estimation unit estimates an outline of the lumen of the blood vessel depicted in the medical image data by a region boundary estimation method, and after estimating the lumen certainty factor, generates a lumen certainty factor map showing a distribution of the lumen certainty factor in the medical image data, generates a graph in which a difference in the lumen certainty factor on both sides of the outer periphery of the cross section of the blood vessel in the lumen certainty factor map is reflected in a cutting cost, and performs graph cutting; The system further includes an area defining unit that obtains the cutting cost of each cut boundary from graph information when a graph cut is executed, and defines boundaries with low cutting costs as targets for editing restriction by the user and boundaries with high cutting costs as targets for editing recommendation by the user. The image processing device according to any one of claims 1 to 9.

13. The anatomical sites included in the vascular region are the vascular core and the vascular wall. The image processing device according to any one of claims 1 to 9.

14. The estimation unit extracts at least one of the lumen of the blood vessel and plaque in the blood vessel from the medical image data. The image processing device according to any one of claims 1 to 8.

15. an acquisition step of acquiring medical image data; an extraction step of extracting a vascular region in which blood vessels of a target organ are depicted from the medical image data; a determination step of determining a degree of certainty, which is an index representing the accuracy of extraction of the vascular region extracted from the medical image data; an estimation step of estimating a background region in which the blood vessel is not depicted in the medical image data and a region for each anatomical part included in the blood vessel region based on the certainty factor; a defining step of defining an area in which editing of the estimation result by the estimating step by a user is restricted based on the confidence level; An image processing method comprising:

16. an acquisition step of acquiring medical image data; an extraction step of extracting a vascular region in which blood vessels of a target organ are depicted from the medical image data; a determination step of determining a degree of certainty, which is an index representing the accuracy of extraction of the vascular region extracted from the medical image data; an estimation step of estimating a background region in which the blood vessel is not depicted in the medical image data and a region for each anatomical part included in the blood vessel region based on the certainty factor; a defining step of defining an area in which editing of the estimation result by the estimating step by a user is restricted based on the certainty factor.

Images